Adding extra fields into the schema

[Harsh J <qw...@gmail.com>]

I'm wondering if there's a way to add user-defined name:value pairs
into the Avro JSON schema itself?

For instance, while building my schema (by using setFields for a
collection of fields), for some fields I do the following: [Using Java
API]

fieldObject.schema().addProp("myproperty", "myvalue");

I see that it adds to the schema like this:
{
    "name" : "someFieldXYZ",
    "type" : {
      "type" : "bytes",
      "myproperty" : "myvalue"
    },
    "default" : "XYZ"
}

But when I re-parse this created schema and try to retrieve my
property back, I get a null. I do this for getting it back:

fieldObject.schema.getProp("myproperty");

Why am I getting a null here? How do I access what's I embedded into
the schema properly back again? Or is this plain WRONG a thing to do?

[Type resolution still works fine even after it getting modified like that].

-- 
Harsh J
www.harshj.com

Re: Adding extra fields into the schema

[Harsh J <qw...@gmail.com>]

Its pretty odd, the test case passes (Tried trunk though) but I'm
unable to add properties. Was this fixed for 1.4 or 1.3 also? I'm
using 1.3.3 as of now.

All I do is create a schema (with properties added to the schema
object for the chosen fields) at the InputFormat level (pre-Mapper),
place it in the JobConf with a toString and get it out again within my
Mapper and create a Schema from it. I lose my set properties, but the
rest remains perfectly intact. I'm only adding properties to non-union
fields of my record.

On Tue, Jul 20, 2010 at 12:10 AM, Doug Cutting <cu...@apache.org> wrote:
> This should work.
>
> There are existing tests for schema properties are in TestSchema.java. The
> checkProp() method is called to add a property to every type of schema,
> print the schema with that property, then re-parse it and check that the
> property is still there.
>
> Can you provide a complete test case that illustrates this failing?
>
> Thanks,
>
> Doug
>
> On 07/17/2010 10:45 AM, Harsh J wrote:
>>
>> I'm wondering if there's a way to add user-defined name:value pairs
>> into the Avro JSON schema itself?
>>
>> For instance, while building my schema (by using setFields for a
>> collection of fields), for some fields I do the following: [Using Java
>> API]
>>
>> fieldObject.schema().addProp("myproperty", "myvalue");
>>
>> I see that it adds to the schema like this:
>> {
>>     "name" : "someFieldXYZ",
>>     "type" : {
>>       "type" : "bytes",
>>       "myproperty" : "myvalue"
>>     },
>>     "default" : "XYZ"
>> }
>>
>> But when I re-parse this created schema and try to retrieve my
>> property back, I get a null. I do this for getting it back:
>>
>> fieldObject.schema.getProp("myproperty");
>>
>> Why am I getting a null here? How do I access what's I embedded into
>> the schema properly back again? Or is this plain WRONG a thing to do?
>>
>> [Type resolution still works fine even after it getting modified like
>> that].
>>
>



-- 
Harsh J
www.harshj.com

Re: Adding extra fields into the schema

[Doug Cutting <cu...@apache.org>]

This should work.

There are existing tests for schema properties are in TestSchema.java. 
The checkProp() method is called to add a property to every type of 
schema, print the schema with that property, then re-parse it and check 
that the property is still there.

Can you provide a complete test case that illustrates this failing?

Thanks,

Doug

On 07/17/2010 10:45 AM, Harsh J wrote:
> I'm wondering if there's a way to add user-defined name:value pairs
> into the Avro JSON schema itself?
>
> For instance, while building my schema (by using setFields for a
> collection of fields), for some fields I do the following: [Using Java
> API]
>
> fieldObject.schema().addProp("myproperty", "myvalue");
>
> I see that it adds to the schema like this:
> {
>      "name" : "someFieldXYZ",
>      "type" : {
>        "type" : "bytes",
>        "myproperty" : "myvalue"
>      },
>      "default" : "XYZ"
> }
>
> But when I re-parse this created schema and try to retrieve my
> property back, I get a null. I do this for getting it back:
>
> fieldObject.schema.getProp("myproperty");
>
> Why am I getting a null here? How do I access what's I embedded into
> the schema properly back again? Or is this plain WRONG a thing to do?
>
> [Type resolution still works fine even after it getting modified like that].
>

Re: Adding extra fields into the schema

[Harsh J <qw...@gmail.com>]

Okay, I think the schema builder ignores all of that. I'll go ahead
with maintaining what I need as a different json/xml element outside,
thank you :)

[Psst. But if there's a way, do let me know. I tend to jump quickly (:]

On Sat, Jul 17, 2010 at 11:15 PM, Harsh J <qw...@gmail.com> wrote:
> I'm wondering if there's a way to add user-defined name:value pairs
> into the Avro JSON schema itself?
>
> For instance, while building my schema (by using setFields for a
> collection of fields), for some fields I do the following: [Using Java
> API]
>
> fieldObject.schema().addProp("myproperty", "myvalue");
>
> I see that it adds to the schema like this:
> {
>    "name" : "someFieldXYZ",
>    "type" : {
>      "type" : "bytes",
>      "myproperty" : "myvalue"
>    },
>    "default" : "XYZ"
> }
>
> But when I re-parse this created schema and try to retrieve my
> property back, I get a null. I do this for getting it back:
>
> fieldObject.schema.getProp("myproperty");
>
> Why am I getting a null here? How do I access what's I embedded into
> the schema properly back again? Or is this plain WRONG a thing to do?
>
> [Type resolution still works fine even after it getting modified like that].
>
> --
> Harsh J
> www.harshj.com
>



-- 
Harsh J
www.harshj.com