field union default (in Java)

[John Kristian <jk...@linkedin.com>]

It seems tricky to add a nullable field with a default (for compatibility
with previous versions).  I find that the type of the default value must be
declared first in the union.  For example, this works:

{"name":"myField", "type":["int", "null"], "default": 3}

But this doesn't:

{"name":"myField", "type":["null", "int"], "default": 3}

When I resolve the latter from a writer's schema that doesn't declare the
field, I get the value null, not 3.  There's a similar problem with a null
default: the resolved value is zero, not null, if "int" comes first in the
union.

Am I approaching this wrong?  I was hoping the resolver would pick the type
that matches the default value.  It seems unambiguous, in this case.

I'm using the Java implementation.

- John Kristian

Re: field union default (in Java)

[Doug Cutting <cu...@apache.org>]

On 08/16/2010 01:08 PM, John Kristian wrote:
> It seems like it would better for schema parsing (or perhaps resolution) to
> fail if the default value doesn't match the first type in the union.  That
> would require people to clarify existing schemas.  Would it cause trouble in
> other ways?

Defaults can be ambiguous.  For example, {"a":1} as the default value 
for a union of a record with a field named "a" of type int and a map 
with long values, or "foo" for the default value of a union of string, 
bytes and an enum.  If you switched the order in the union, you'd get no 
warning.

However we could do a better job of detecting some erroneous default 
values earlier.  The resolver currently handles default values in 
ResolvingGrammarGenerator#encode().  We could add more checks there. 
For example, the NULL case there could throw an exception if the JSON 
node n is non-null.  Similarly, the INT case should throw an exception 
if JSON node is non-numeric (currently it calls n.getIntValue() which, 
unfortunately returns zero for non-numeric nodes).

Would you like to file an issue in Jira for this?  If you can, please 
provide a patch & unit tests.

Thanks,

Doug

Re: field union default (in Java)

[John Kristian <jk...@linkedin.com>]

Thanks for the reference.  I'll get used to it.  But it was an unpleasant
surprise.  I wish I had some warning earlier, before a surprising value came
out of schema resolution.  Mistakes that are found earlier are often cheaper
to correct.

It seems like it would better for schema parsing (or perhaps resolution) to
fail if the default value doesn't match the first type in the union.  That
would require people to clarify existing schemas.  Would it cause trouble in
other ways?

- John Kristian

> From: Doug Cutting <cu...@apache.org>
> Reply-To: <us...@avro.apache.org>
> Date: Mon, 16 Aug 2010 12:03:18 -0700
> To: <us...@avro.apache.org>
> Subject: Re: field union default (in Java)
> 
> On 08/16/2010 11:34 AM, John Kristian wrote:
>> It seems tricky to add a nullable field with a default (for compatibility
>> with previous versions).  I find that the type of the default value must be
>> declared first in the union.  For example, this works:
>> 
>> {"name":"myField", "type":["int", "null"], "default": 3}
>> 
>> But this doesn't:
>> 
>> {"name":"myField", "type":["null", "int"], "default": 3}
>> 
>> When I resolve the latter from a writer's schema that doesn't declare the
>> field, I get the value null, not 3.  There's a similar problem with a null
>> default: the resolved value is zero, not null, if "int" comes first in the
>> union.
> 
> John,
> 
> That's the intended, specified behavior:
> 
> "Default values for union fields correspond to the first schema in the
> union."
> 
> http://avro.apache.org/docs/current/spec.html#schema_complex
> 
> Doug

Re: field union default (in Java)

[Doug Cutting <cu...@apache.org>]

On 08/16/2010 11:34 AM, John Kristian wrote:
> It seems tricky to add a nullable field with a default (for compatibility
> with previous versions).  I find that the type of the default value must be
> declared first in the union.  For example, this works:
>
> {"name":"myField", "type":["int", "null"], "default": 3}
>
> But this doesn't:
>
> {"name":"myField", "type":["null", "int"], "default": 3}
>
> When I resolve the latter from a writer's schema that doesn't declare the
> field, I get the value null, not 3.  There's a similar problem with a null
> default: the resolved value is zero, not null, if "int" comes first in the
> union.

John,

That's the intended, specified behavior:

"Default values for union fields correspond to the first schema in the 
union."

http://avro.apache.org/docs/current/spec.html#schema_complex

Doug