Using optional fields (nullable unions) with the C-API

[Markus Resch <ma...@adtech.de>]

Hey avro users,

I'm using a record like this

{
    "type" : "record", 
    "name" : "TestPacket", 
    "fields" : [
        { "name" : "Id" , "type" : "int"},
        { "name" : "Value" , "type" : ["string","null"]}
	]
}

But if I just not write anything to the 'Value' filed,
avro_writer_memory() refuses to validate it. How do you normally use the
optional fields with the C-API?

Regards,

	Markus


-- 
Markus Resch
Software Developer 
P: +49 6103-5715-0 | F: +49 6103-5715-111 |
ADTECH AG | Robert-Bosch-Str. 32 | 63303 Dreieich | Germany
www.adtech.com<http://www.adtech.com/>


ADTECH | A Division of Advertising.com Group - Residence of the Company:
Dreieich, Germany - Registration Office Offenbach HRB 34657
Chairman of the Board: Erhard Neumann - Chairman of the Supervisory
Board: Ned Brody

This message contains privileged and confidential information. Any
dissemination, distribution, copying or other use of this
message or any of its content (data, prices...) to any third parties may
only occur with ADTECH's prior consent.

Re: Using optional fields (nullable unions) with the C-API

[adityanahan <ad...@gmail.com>]

Hi guys, 

I am facing problem in encoding the nullable fields. 


Hi Doug, 

I thought i was using it correctly, but running into issues of using the
union with nullable fields. 
Doing something like: 

Schema: This is just snippet, the whole schema is not posted. 

{
            "name": "protocol",
            "type": [
                "string",
                "null"
            ]
        },
        {
            "name": "port_number",
            "type": [
                "long",
                "null"
            ],
            "doc": "Port"
        },
}
   acos_avro_check_i(avro_value_get_by_name(&writer_value, "port_number",
&field, NULL)); 
    acos_avro_check_i(avro_value_set_branch(&field, 1, &branch)); 
    acos_avro_check_i(avro_value_set_null(&branch)); 
    acos_avro_check_i(avro_value_set_int(&branch, 80)); 
    /*protocol*/ 
    acos_avro_check_i(avro_value_get_by_name(&writer_value, "protocol",
&field, NULL)); 
    acos_avro_check_i(avro_value_set_branch(&field, 1, &branch)); 
    acos_avro_check_i(avro_value_set_null(&branch)); 
    acos_avro_check_i(avro_value_set_string(&branch, "tcp")); 

Posting it. 

But getting error when i try to set the value for say set_int or set_string. 
I can't seem to understand why it is failing. 
Is this correct way to encode a union field or are there some better
examples i can look at ?



--
View this message in context: http://apache-avro.679487.n3.nabble.com/Using-optional-fields-nullable-unions-with-the-C-API-tp3525392p4037669.html
Sent from the Avro - Users mailing list archive at Nabble.com.

Re: Using optional fields (nullable unions) with the C-API

[adityanahan <ad...@gmail.com>]

Posting again.



--
View this message in context: http://apache-avro.679487.n3.nabble.com/Using-optional-fields-nullable-unions-with-the-C-API-tp3525392p4037667.html
Sent from the Avro - Users mailing list archive at Nabble.com.

Re: Using optional fields (nullable unions) with the C-API

[Douglas Creager <dc...@dcreager.net>]

> I'm using a record like this
> 
> {
>    "type" : "record", 
>    "name" : "TestPacket", 
>    "fields" : [
>        { "name" : "Id" , "type" : "int"},
>        { "name" : "Value" , "type" : ["string","null"]}
> 	]
> }
> 
> But if I just not write anything to the 'Value' filed,
> avro_writer_memory() refuses to validate it. How do you normally use the
> optional fields with the C-API?

Hi Markus

In Avro, an “optional” field is really just a common pattern where you've got a union, with a “null” for one branch, and some other type for the second branch.  And with all union types, you have to explicitly tell the Avro library which branch is active for a particular value.  (Think of the union as a container that must always contain exactly one value, but where the type of that value can change.)

So, if you're using the new value API, you'd use something like:

  avro_value_t  rec = /* given */;
  avro_value_t  field, branch;
  avro_value_get_by_name(rec, "Value", &field, NULL);
  avro_value_set_branch(&field, 1, &branch);
  avro_value_set_null(&branch);

and if you're using the old datum API:

  avro_datum_t  rec = /* given */;
  avro_datum_t  field, branch;
  avro_record_get(rec, "Value", &field);
  avro_union_set_discriminant(field, 1, &branch);
  /* no avro_null_set in the old API */

In set_branch/set_discriminant, the 1 refers to the index of the "null" type in the union.  I tend to use branch 0 for the null type in these kinds of unions, but that's just a matter of aesthetics.

cheers
–doug