You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Xingen Wang <ne...@gmail.com> on 2010/12/02 07:48:00 UTC
Some questions for Avro
Hi, Avro organizers,
I am currently trying Avro and try to make it as our rpc framework in
production, and I met some problems in using it, could you please find your
time and answer them for me?
1. Does the json schema support extends? e.g. First, I define an
general-purpose json schema named "Object", and in other json schema, try to
extends this "Object".
2. In the primitive types, I can't find one type that can represent any
objects. I tried "record", but you have to specify "name" and "fields" for
it.
If Avro support any of the above features, we can use it in our production.
Because we don't want to change these definition all the time.
--
Xingen Wang
--
Xingen Wang
--
Xingen Wang
Re: Some questions for Avro
Posted by Xingen Wang <ne...@gmail.com>.
Thanks, Doug,
I will try it in my project.
On Tue, Jan 11, 2011 at 3:27 AM, Doug Cutting <cu...@apache.org> wrote:
> On 12/07/2010 03:06 AM, Xingen Wang wrote:
>
>> I just want my rpc framework be able to transport any objects through
>> network without any modification in the existing schema definition.
>>
>> Can I achieve this objective with Avro
>>
>
> Perhaps. A way to do this is to define a general schema. If, for example,
> your objects were named records whose named fields could be null, boolean,
> long, double, string or a nested record value, then your schema might be
> something like:
>
> {"name":"data", "type":"record", "fields":[
> {"name":"value", "type":
> ["null",
> "boolean",
> "long",
> "double"
> "string",
> {"name":"record", "type":"record", "fields":[
> {"name":"fields", "type":
> {"type":"array", "items":
> {"name":"field", "type":"record", "fields":[
> {"name":"name", "type":"string"},
> {"name":"value", "type":"data"}
> ]}
> }
> }]}
> ]}
>
> Note that this is almost a schema for the JSON data model.
>
> Doug
>
> On Tue, Dec 7, 2010 at 9:12 AM, Jeff Hammerbacher <hammer@cloudera.com
>> <ma...@cloudera.com>> wrote:
>>
>> Hey Xingen,
>>
>> 1. Does the json schema support extends? e.g. First, I define an
>> general-purpose json schema named "Object", and in other json
>> schema, try to extends this "Object".
>>
>>
>> Avro IDL supports includes:
>> http://avro.apache.org/docs/current/idl.html#imports. It does not
>> support extensions, to the best of my knowledge. Perhaps you could
>> file a feature request at https://issues.apache.org/jira/browse/AVRO?
>>
>> 2. In the primitive types, I can't find one type that can
>> represent any objects. I tried "record", but you have to specify
>> "name" and "fields" for it.
>>
>>
>> You could make a union of all of the primitive types? I'm not sure
>> exactly what you're trying to accomplish with this question though.
>>
>> Thanks,
>> Jeff
>>
>>
>>
>>
>> --
>> Xingen Wang
>>
>
--
Xingen Wang
Re: Some questions for Avro
Posted by Doug Cutting <cu...@apache.org>.
On 12/07/2010 03:06 AM, Xingen Wang wrote:
> I just want my rpc framework be able to transport any objects through
> network without any modification in the existing schema definition.
>
> Can I achieve this objective with Avro
Perhaps. A way to do this is to define a general schema. If, for
example, your objects were named records whose named fields could be
null, boolean, long, double, string or a nested record value, then your
schema might be something like:
{"name":"data", "type":"record", "fields":[
{"name":"value", "type":
["null",
"boolean",
"long",
"double"
"string",
{"name":"record", "type":"record", "fields":[
{"name":"fields", "type":
{"type":"array", "items":
{"name":"field", "type":"record", "fields":[
{"name":"name", "type":"string"},
{"name":"value", "type":"data"}
]}
}
}]}
]}
Note that this is almost a schema for the JSON data model.
Doug
> On Tue, Dec 7, 2010 at 9:12 AM, Jeff Hammerbacher <hammer@cloudera.com
> <ma...@cloudera.com>> wrote:
>
> Hey Xingen,
>
> 1. Does the json schema support extends? e.g. First, I define an
> general-purpose json schema named "Object", and in other json
> schema, try to extends this "Object".
>
>
> Avro IDL supports includes:
> http://avro.apache.org/docs/current/idl.html#imports. It does not
> support extensions, to the best of my knowledge. Perhaps you could
> file a feature request at https://issues.apache.org/jira/browse/AVRO?
>
> 2. In the primitive types, I can't find one type that can
> represent any objects. I tried "record", but you have to specify
> "name" and "fields" for it.
>
>
> You could make a union of all of the primitive types? I'm not sure
> exactly what you're trying to accomplish with this question though.
>
> Thanks,
> Jeff
>
>
>
>
> --
> Xingen Wang
Re: Some questions for Avro
Posted by Xingen Wang <ne...@gmail.com>.
I just want my rpc framework be able to transport any objects through
network without any modification in the existing schema definition.
Can I achieve this objective with Avro
On Tue, Dec 7, 2010 at 9:12 AM, Jeff Hammerbacher <ha...@cloudera.com>wrote:
> Hey Xingen,
>
>
>> 1. Does the json schema support extends? e.g. First, I define an
>> general-purpose json schema named "Object", and in other json schema, try to
>> extends this "Object".
>>
>
> Avro IDL supports includes:
> http://avro.apache.org/docs/current/idl.html#imports. It does not support
> extensions, to the best of my knowledge. Perhaps you could file a feature
> request at https://issues.apache.org/jira/browse/AVRO?
>
>
>> 2. In the primitive types, I can't find one type that can represent any
>> objects. I tried "record", but you have to specify "name" and "fields" for
>> it.
>>
>
> You could make a union of all of the primitive types? I'm not sure exactly
> what you're trying to accomplish with this question though.
>
> Thanks,
> Jeff
>
--
Xingen Wang
Re: Some questions for Avro
Posted by Jeff Hammerbacher <ha...@cloudera.com>.
Hey Xingen,
> 1. Does the json schema support extends? e.g. First, I define an
> general-purpose json schema named "Object", and in other json schema, try to
> extends this "Object".
>
Avro IDL supports includes:
http://avro.apache.org/docs/current/idl.html#imports. It does not support
extensions, to the best of my knowledge. Perhaps you could file a feature
request at https://issues.apache.org/jira/browse/AVRO?
> 2. In the primitive types, I can't find one type that can represent any
> objects. I tried "record", but you have to specify "name" and "fields" for
> it.
>
You could make a union of all of the primitive types? I'm not sure exactly
what you're trying to accomplish with this question though.
Thanks,
Jeff