Avro-c Generics Usage

[Brian Mcqueen <bm...@linkedin.com>]

In general I’ve got some lines coming in, and I’ll be parsing them into a c-struct.  Then I want to serialize it in avro.  I don’t want to write a custom implementation of the value interface unless I have to.  I see some text in the docs saying the generics should be good enough for most cases, but I don’t see how I could take my populated c struct, and an instantiation of a class and come out of it with something avro-encoded without me writing the machinery.

I see this flow in section 4.2. Creating value instances:


·         avro_generic_class_from_schema

·         avro_generic_value_new

·         avro_generic_value_free



And I see in the example where they are messing with longs:



·         avro_schema_long

·         avro_value_set_long



I can get my avro schema via this kind of thing:



·         avro_schema_from_json_literal



·         But how can I get the equivalent to avro_value_set_long? I don’t see a generic avro_value_set.


Give me some good news.  I want to go bang-boom and have my data serialized. I want to be able to mess only with the avro spec and have the serialization adapt by magic.

Re: Avro-c Generics Usage

[Brian Mcqueen <bm...@linkedin.com>]

I found a great c lib that is built on avro-c I'll put in this thread for future reference:

https://github.com/confluentinc/libserdes

and a related project showing how to use it:

https://github.com/edenhill/kafkacat

For my specific problem, there's no escaping the fact that I'll need to parse my input and properly build an avro value that corresponds to the schema.

I'm thinking I'll wrap the generics and expose them via lua so users can expect to handle arbitrary schemas.

I see one lua-avro out there and it seems way out-of-date and not likely worth resurrection.

On 7/11/20, 7:52 PM, "Brian Mcqueen" <bm...@linkedin.com> wrote:

    In general I’ve got some lines coming in, and I’ll be parsing them into a c-struct.  Then I want to serialize it in avro.  I don’t want to write a custom implementation of the value interface unless I have to.  I see some text in the docs saying the generics should be good enough for most cases, but I don’t see how I could take my populated c struct, and an instantiation of a class and come out of it with something avro-encoded without me writing the machinery.

    I see this flow in section 4.2. Creating value instances:

    ·         avro_generic_class_from_schema
    ·         avro_generic_value_new·         avro_generic_value_free And I see in the example where they are messing with longs: ·         avro_schema_long·         avro_value_set_long I can get my avro schema via this kind of thing:  ·         avro_schema_from_json_literal ·         But how can I get the equivalent to avro_value_set_long? I don’t see a generic avro_value_set. 
    Give me some good news.  I want to go bang-boom and have my data serialized. I want to be able to mess only with the avro spec and have the serialization adapt by magic.