optional array in avro

[Philipp <ph...@metrigo.de>]

I'm wondering whether or not it is possible to have an optional array.
Let's assume a schema like this:

{ "type": "record",
   "name": "test_avro",
   "fields" : [
     {"name": "test_field_1", "type": "long"},
     {"name": "subrecord", "type": [{
       "type": "record",
       "name": "subrecord_type",
       "fields":[{"name":"field_1", "type":"long"}]
       },"null"]
     },
     {"name": "simple_array",
     "type":{
         "type": "array",
         "items": "string"
       }
     }
   ]
}

Trying to write an avro record without "simple_array" would result in a 
NPE in the datafilewriter.
For subrecord it's just fine, but when I try to define the array as 
optional:

{"name": "simple_array",
     "type":[{
         "type": "array",
         "items": "string"
       }, "null"]


It does not result in a NPE but a runtime exception:
AvroRuntimeException: Not an array schema: 
[{"type":"array","items":"string"},"null"]

Thanks.

Re: optional array in avro

[Doug Cutting <cu...@apache.org>]

On 02/23/2012 09:04 AM, Philipp wrote:
> For subrecord it's just fine, but when I try to define the array as
> optional:
> 
> {"name": "simple_array",
>     "type":[{
>         "type": "array",
>         "items": "string"
>       }, "null"]
> 
> 
> It does not result in a NPE but a runtime exception:
> AvroRuntimeException: Not an array schema:
> [{"type":"array","items":"string"},"null"]

It's hard to tell exactly what's happening without a complete test case.

What version of Avro are you using?

Also, try specifying the "null" before the array schema in the union and
specify a default value of null, e.g.:

{"name": "simple_array",
 "type": ["null", {"type":"array", "items":"string"}],
 "default": null}

Doug