You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by Vivek Kumar <vi...@eclipsetrading.com.INVALID> on 2024/01/11 03:11:32 UTC
Re: Avro schema evolution support in AVRO CPP
+dev
Regards,
Vivek Kumar
[http://www.eclipsetrading.com/logo.png]
Senior Software Developer
23/F One Hennessy
1 Hennessy Road
Wan Chai
Hong Kong
www.eclipsetrading.com<http://www.eclipsetrading.com/>
+852 2108 7352
Follow us today on our online platforms
[Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<https://www.linkedin.com/company/eclipse-trading>[Instagram]<https://www.instagram.com/eclipsetrading>
________________________________
From: Vivek Kumar <vi...@eclipsetrading.com>
Sent: Thursday, January 11, 2024 11:07 AM
To: user@avro.apache.org <us...@avro.apache.org>
Subject: Avro schema evolution support in AVRO CPP
Hi Avro team,
I am writing this email to check the support of Avro schema evolution in CPP - i.e. provide both the producer and consumer schema when decoding the data.
I can see that there's a resolvingDecoder function in AVRO CPP that takes two schemas. See
https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html#ReadingDifferentSchema
But there's a FIXME comment in this function. See https://issues.apache.org/jira/browse/AVRO-3720 and https://github.com/apache/avro/blob/main/lang/c%2B%2B/api/Decoder.hh#L218. Does this mean resolvingDecoder does not work properly? Could you please explain what scenarios are not covered by resolvingDecoder and how can we use it to support "Avro Schema Evolution" in c++?
Thanks
Regards,
Vivek Kumar
[http://www.eclipsetrading.com/logo.png]
Senior Software Developer
23/F One Hennessy
1 Hennessy Road
Wan Chai
Hong Kong
www.eclipsetrading.com<http://www.eclipsetrading.com/>
+852 2108 7352
Follow us today on our online platforms
[Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<https://www.linkedin.com/company/eclipse-trading>[Instagram]<https://www.instagram.com/eclipsetrading>
Re: Avro schema evolution support in AVRO CPP
Posted by John McClean <jm...@gmail.com>.
fwiw, I'm using it and it works fine, at least for my use cases.
J
On Fri, Jan 12, 2024 at 1:55 AM Martin Grigorov <mg...@apache.org>
wrote:
> Hi Vivek,
>
> I am not sure there is anyone to give you an exact answer. The C++ SDK has
> not been actively developed in the last few years.
> The best is to try it for your use cases and see if it works or not. The
> next step is to contribute Pull Requests for the missing functionalities!
>
> Martin
>
> On Thu, Jan 11, 2024 at 8:59 AM Vivek Kumar <
> vivek.kumar@eclipsetrading.com.invalid> wrote:
>
>> +dev
>>
>>
>> Regards,
>> Vivek Kumar
>>
>> [http://www.eclipsetrading.com/logo.png]
>>
>> Senior Software Developer
>> 23/F One Hennessy
>> 1 Hennessy Road
>> Wan Chai
>> Hong Kong
>> www.eclipsetrading.com<http://www.eclipsetrading.com/>
>> +852 2108 7352
>>
>> Follow us today on our online platforms
>> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
>> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
>> https://www.instagram.com/eclipsetrading>
>> ________________________________
>> From: Vivek Kumar <vi...@eclipsetrading.com>
>> Sent: Thursday, January 11, 2024 11:07 AM
>> To: user@avro.apache.org <us...@avro.apache.org>
>> Subject: Avro schema evolution support in AVRO CPP
>>
>> Hi Avro team,
>>
>> I am writing this email to check the support of Avro schema evolution in
>> CPP - i.e. provide both the producer and consumer schema when decoding the
>> data.
>>
>> I can see that there's a resolvingDecoder function in AVRO CPP that takes
>> two schemas. See
>>
>> https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html#ReadingDifferentSchema
>>
>> But there's a FIXME comment in this function. See
>> https://issues.apache.org/jira/browse/AVRO-3720 and
>> https://github.com/apache/avro/blob/main/lang/c%2B%2B/api/Decoder.hh#L218.
>> Does this mean resolvingDecoder does not work properly? Could you please
>> explain what scenarios are not covered by resolvingDecoder and how can we
>> use it to support "Avro Schema Evolution" in c++?
>>
>> Thanks
>>
>>
>> Regards,
>> Vivek Kumar
>>
>> [http://www.eclipsetrading.com/logo.png]
>>
>> Senior Software Developer
>> 23/F One Hennessy
>> 1 Hennessy Road
>> Wan Chai
>> Hong Kong
>> www.eclipsetrading.com<http://www.eclipsetrading.com/>
>> +852 2108 7352
>>
>> Follow us today on our online platforms
>> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
>> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
>> https://www.instagram.com/eclipsetrading>
>>
>
Re: Avro schema evolution support in AVRO CPP
Posted by Andrew Marlow <ma...@gmail.com>.
"In practice, it is very rare for schema evolution to change the order of
the fields." - I'll say. Since we are talking about a protocol that is
deliberately not self-describing we cannot just pluck out what we want -
how would such code get to it? This is why the standard advice in these
situations is to never reorder, remove or rename fields and to always add
new stuff to the end.
On Fri, 12 Jan 2024 at 13:19, Thiruvalluvan MG <th...@yahoo.com.invalid>
wrote:
> Out-of-order fields are not handled transparently in C++ if you are
> manually using the resolving decoder. (It's the same situation in Java as
> well).
> But, in C++ and in Java, if you generate code for the given Avro schema,
> the generated code takes care of the field ordering issue. Similarly, in
> both bindings, Generic data structures work properly with the field-order.
> If you are using the resolving decoder in your code directly, care must be
> exercises, If the reader schema and writer schema are both records and they
> have fields in different order (it is okay to insert or remove fields), the
> protocol is to first get the field-order array (which is essentially a
> permutation of the reader field ids 0 to n-1) from the resolving decoder
> and then read the fields of the reader schema in the order specified in the
> field-order array. This is done in order to avoid buffering by the decoder.
> Buffering can take a large number of allocations if the out-of-order fields
> is an array or map.
> In practice, it is very rare for schema evolution to change the order of
> the fields.
> Thanks
> Thiru On Friday, 12 January, 2024 at 03:24:11 pm IST, Martin Grigorov <
> mgrigorov@apache.org> wrote:
>
> Hi Vivek,
>
> I am not sure there is anyone to give you an exact answer. The C++ SDK has
> not been actively developed in the last few years.
> The best is to try it for your use cases and see if it works or not. The
> next step is to contribute Pull Requests for the missing functionalities!
>
> Martin
>
> On Thu, Jan 11, 2024 at 8:59 AM Vivek Kumar <
> vivek.kumar@eclipsetrading.com.invalid> wrote:
>
> > +dev
> >
> >
> > Regards,
> > Vivek Kumar
> >
> > [http://www.eclipsetrading.com/logo.png]
> >
> > Senior Software Developer
> > 23/F One Hennessy
> > 1 Hennessy Road
> > Wan Chai
> > Hong Kong
> > www.eclipsetrading.com<http://www.eclipsetrading.com/>
> > +852 2108 7352
> >
> > Follow us today on our online platforms
> > [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> > https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> > https://www.instagram.com/eclipsetrading>
> > ________________________________
> > From: Vivek Kumar <vi...@eclipsetrading.com>
> > Sent: Thursday, January 11, 2024 11:07 AM
> > To: user@avro.apache.org <us...@avro.apache.org>
> > Subject: Avro schema evolution support in AVRO CPP
> >
> > Hi Avro team,
> >
> > I am writing this email to check the support of Avro schema evolution in
> > CPP - i.e. provide both the producer and consumer schema when decoding
> the
> > data.
> >
> > I can see that there's a resolvingDecoder function in AVRO CPP that takes
> > two schemas. See
> >
> >
> https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html#ReadingDifferentSchema
> >
> > But there's a FIXME comment in this function. See
> > https://issues.apache.org/jira/browse/AVRO-3720 and
> >
> https://github.com/apache/avro/blob/main/lang/c%2B%2B/api/Decoder.hh#L218.
> > Does this mean resolvingDecoder does not work properly? Could you please
> > explain what scenarios are not covered by resolvingDecoder and how can we
> > use it to support "Avro Schema Evolution" in c++?
> >
> > Thanks
> >
> >
> > Regards,
> > Vivek Kumar
> >
> > [http://www.eclipsetrading.com/logo.png]
> >
> > Senior Software Developer
> > 23/F One Hennessy
> > 1 Hennessy Road
> > Wan Chai
> > Hong Kong
> > www.eclipsetrading.com<http://www.eclipsetrading.com/>
> > +852 2108 7352
> >
> > Follow us today on our online platforms
> > [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> > https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> > https://www.instagram.com/eclipsetrading>
> >
>
--
Regards,
Andrew Marlow
http://www.andrewpetermarlow.co.uk
Re: Avro schema evolution support in AVRO CPP
Posted by Andrew Marlow <ma...@gmail.com>.
"In practice, it is very rare for schema evolution to change the order of
the fields." - I'll say. Since we are talking about a protocol that is
deliberately not self-describing we cannot just pluck out what we want -
how would such code get to it? This is why the standard advice in these
situations is to never reorder, remove or rename fields and to always add
new stuff to the end.
On Fri, 12 Jan 2024 at 13:19, Thiruvalluvan MG <th...@yahoo.com.invalid>
wrote:
> Out-of-order fields are not handled transparently in C++ if you are
> manually using the resolving decoder. (It's the same situation in Java as
> well).
> But, in C++ and in Java, if you generate code for the given Avro schema,
> the generated code takes care of the field ordering issue. Similarly, in
> both bindings, Generic data structures work properly with the field-order.
> If you are using the resolving decoder in your code directly, care must be
> exercises, If the reader schema and writer schema are both records and they
> have fields in different order (it is okay to insert or remove fields), the
> protocol is to first get the field-order array (which is essentially a
> permutation of the reader field ids 0 to n-1) from the resolving decoder
> and then read the fields of the reader schema in the order specified in the
> field-order array. This is done in order to avoid buffering by the decoder.
> Buffering can take a large number of allocations if the out-of-order fields
> is an array or map.
> In practice, it is very rare for schema evolution to change the order of
> the fields.
> Thanks
> Thiru On Friday, 12 January, 2024 at 03:24:11 pm IST, Martin Grigorov <
> mgrigorov@apache.org> wrote:
>
> Hi Vivek,
>
> I am not sure there is anyone to give you an exact answer. The C++ SDK has
> not been actively developed in the last few years.
> The best is to try it for your use cases and see if it works or not. The
> next step is to contribute Pull Requests for the missing functionalities!
>
> Martin
>
> On Thu, Jan 11, 2024 at 8:59 AM Vivek Kumar <
> vivek.kumar@eclipsetrading.com.invalid> wrote:
>
> > +dev
> >
> >
> > Regards,
> > Vivek Kumar
> >
> > [http://www.eclipsetrading.com/logo.png]
> >
> > Senior Software Developer
> > 23/F One Hennessy
> > 1 Hennessy Road
> > Wan Chai
> > Hong Kong
> > www.eclipsetrading.com<http://www.eclipsetrading.com/>
> > +852 2108 7352
> >
> > Follow us today on our online platforms
> > [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> > https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> > https://www.instagram.com/eclipsetrading>
> > ________________________________
> > From: Vivek Kumar <vi...@eclipsetrading.com>
> > Sent: Thursday, January 11, 2024 11:07 AM
> > To: user@avro.apache.org <us...@avro.apache.org>
> > Subject: Avro schema evolution support in AVRO CPP
> >
> > Hi Avro team,
> >
> > I am writing this email to check the support of Avro schema evolution in
> > CPP - i.e. provide both the producer and consumer schema when decoding
> the
> > data.
> >
> > I can see that there's a resolvingDecoder function in AVRO CPP that takes
> > two schemas. See
> >
> >
> https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html#ReadingDifferentSchema
> >
> > But there's a FIXME comment in this function. See
> > https://issues.apache.org/jira/browse/AVRO-3720 and
> >
> https://github.com/apache/avro/blob/main/lang/c%2B%2B/api/Decoder.hh#L218.
> > Does this mean resolvingDecoder does not work properly? Could you please
> > explain what scenarios are not covered by resolvingDecoder and how can we
> > use it to support "Avro Schema Evolution" in c++?
> >
> > Thanks
> >
> >
> > Regards,
> > Vivek Kumar
> >
> > [http://www.eclipsetrading.com/logo.png]
> >
> > Senior Software Developer
> > 23/F One Hennessy
> > 1 Hennessy Road
> > Wan Chai
> > Hong Kong
> > www.eclipsetrading.com<http://www.eclipsetrading.com/>
> > +852 2108 7352
> >
> > Follow us today on our online platforms
> > [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> > https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> > https://www.instagram.com/eclipsetrading>
> >
>
--
Regards,
Andrew Marlow
http://www.andrewpetermarlow.co.uk
Re: Avro schema evolution support in AVRO CPP
Posted by Thiruvalluvan MG via user <us...@avro.apache.org>.
Out-of-order fields are not handled transparently in C++ if you are manually using the resolving decoder. (It's the same situation in Java as well).
But, in C++ and in Java, if you generate code for the given Avro schema, the generated code takes care of the field ordering issue. Similarly, in both bindings, Generic data structures work properly with the field-order.
If you are using the resolving decoder in your code directly, care must be exercises, If the reader schema and writer schema are both records and they have fields in different order (it is okay to insert or remove fields), the protocol is to first get the field-order array (which is essentially a permutation of the reader field ids 0 to n-1) from the resolving decoder and then read the fields of the reader schema in the order specified in the field-order array. This is done in order to avoid buffering by the decoder. Buffering can take a large number of allocations if the out-of-order fields is an array or map.
In practice, it is very rare for schema evolution to change the order of the fields.
Thanks
Thiru On Friday, 12 January, 2024 at 03:24:11 pm IST, Martin Grigorov <mg...@apache.org> wrote:
Hi Vivek,
I am not sure there is anyone to give you an exact answer. The C++ SDK has
not been actively developed in the last few years.
The best is to try it for your use cases and see if it works or not. The
next step is to contribute Pull Requests for the missing functionalities!
Martin
On Thu, Jan 11, 2024 at 8:59 AM Vivek Kumar <
vivek.kumar@eclipsetrading.com.invalid> wrote:
> +dev
>
>
> Regards,
> Vivek Kumar
>
> [http://www.eclipsetrading.com/logo.png]
>
> Senior Software Developer
> 23/F One Hennessy
> 1 Hennessy Road
> Wan Chai
> Hong Kong
> www.eclipsetrading.com<http://www.eclipsetrading.com/>
> +852 2108 7352
>
> Follow us today on our online platforms
> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> https://www.instagram.com/eclipsetrading>
> ________________________________
> From: Vivek Kumar <vi...@eclipsetrading.com>
> Sent: Thursday, January 11, 2024 11:07 AM
> To: user@avro.apache.org <us...@avro.apache.org>
> Subject: Avro schema evolution support in AVRO CPP
>
> Hi Avro team,
>
> I am writing this email to check the support of Avro schema evolution in
> CPP - i.e. provide both the producer and consumer schema when decoding the
> data.
>
> I can see that there's a resolvingDecoder function in AVRO CPP that takes
> two schemas. See
>
> https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html#ReadingDifferentSchema
>
> But there's a FIXME comment in this function. See
> https://issues.apache.org/jira/browse/AVRO-3720 and
> https://github.com/apache/avro/blob/main/lang/c%2B%2B/api/Decoder.hh#L218.
> Does this mean resolvingDecoder does not work properly? Could you please
> explain what scenarios are not covered by resolvingDecoder and how can we
> use it to support "Avro Schema Evolution" in c++?
>
> Thanks
>
>
> Regards,
> Vivek Kumar
>
> [http://www.eclipsetrading.com/logo.png]
>
> Senior Software Developer
> 23/F One Hennessy
> 1 Hennessy Road
> Wan Chai
> Hong Kong
> www.eclipsetrading.com<http://www.eclipsetrading.com/>
> +852 2108 7352
>
> Follow us today on our online platforms
> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> https://www.instagram.com/eclipsetrading>
>
Re: Avro schema evolution support in AVRO CPP
Posted by John McClean <jm...@gmail.com>.
fwiw, I'm using it and it works fine, at least for my use cases.
J
On Fri, Jan 12, 2024 at 1:55 AM Martin Grigorov <mg...@apache.org>
wrote:
> Hi Vivek,
>
> I am not sure there is anyone to give you an exact answer. The C++ SDK has
> not been actively developed in the last few years.
> The best is to try it for your use cases and see if it works or not. The
> next step is to contribute Pull Requests for the missing functionalities!
>
> Martin
>
> On Thu, Jan 11, 2024 at 8:59 AM Vivek Kumar <
> vivek.kumar@eclipsetrading.com.invalid> wrote:
>
>> +dev
>>
>>
>> Regards,
>> Vivek Kumar
>>
>> [http://www.eclipsetrading.com/logo.png]
>>
>> Senior Software Developer
>> 23/F One Hennessy
>> 1 Hennessy Road
>> Wan Chai
>> Hong Kong
>> www.eclipsetrading.com<http://www.eclipsetrading.com/>
>> +852 2108 7352
>>
>> Follow us today on our online platforms
>> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
>> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
>> https://www.instagram.com/eclipsetrading>
>> ________________________________
>> From: Vivek Kumar <vi...@eclipsetrading.com>
>> Sent: Thursday, January 11, 2024 11:07 AM
>> To: user@avro.apache.org <us...@avro.apache.org>
>> Subject: Avro schema evolution support in AVRO CPP
>>
>> Hi Avro team,
>>
>> I am writing this email to check the support of Avro schema evolution in
>> CPP - i.e. provide both the producer and consumer schema when decoding the
>> data.
>>
>> I can see that there's a resolvingDecoder function in AVRO CPP that takes
>> two schemas. See
>>
>> https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html#ReadingDifferentSchema
>>
>> But there's a FIXME comment in this function. See
>> https://issues.apache.org/jira/browse/AVRO-3720 and
>> https://github.com/apache/avro/blob/main/lang/c%2B%2B/api/Decoder.hh#L218.
>> Does this mean resolvingDecoder does not work properly? Could you please
>> explain what scenarios are not covered by resolvingDecoder and how can we
>> use it to support "Avro Schema Evolution" in c++?
>>
>> Thanks
>>
>>
>> Regards,
>> Vivek Kumar
>>
>> [http://www.eclipsetrading.com/logo.png]
>>
>> Senior Software Developer
>> 23/F One Hennessy
>> 1 Hennessy Road
>> Wan Chai
>> Hong Kong
>> www.eclipsetrading.com<http://www.eclipsetrading.com/>
>> +852 2108 7352
>>
>> Follow us today on our online platforms
>> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
>> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
>> https://www.instagram.com/eclipsetrading>
>>
>
Re: Avro schema evolution support in AVRO CPP
Posted by Thiruvalluvan MG <th...@yahoo.com.INVALID>.
Out-of-order fields are not handled transparently in C++ if you are manually using the resolving decoder. (It's the same situation in Java as well).
But, in C++ and in Java, if you generate code for the given Avro schema, the generated code takes care of the field ordering issue. Similarly, in both bindings, Generic data structures work properly with the field-order.
If you are using the resolving decoder in your code directly, care must be exercises, If the reader schema and writer schema are both records and they have fields in different order (it is okay to insert or remove fields), the protocol is to first get the field-order array (which is essentially a permutation of the reader field ids 0 to n-1) from the resolving decoder and then read the fields of the reader schema in the order specified in the field-order array. This is done in order to avoid buffering by the decoder. Buffering can take a large number of allocations if the out-of-order fields is an array or map.
In practice, it is very rare for schema evolution to change the order of the fields.
Thanks
Thiru On Friday, 12 January, 2024 at 03:24:11 pm IST, Martin Grigorov <mg...@apache.org> wrote:
Hi Vivek,
I am not sure there is anyone to give you an exact answer. The C++ SDK has
not been actively developed in the last few years.
The best is to try it for your use cases and see if it works or not. The
next step is to contribute Pull Requests for the missing functionalities!
Martin
On Thu, Jan 11, 2024 at 8:59 AM Vivek Kumar <
vivek.kumar@eclipsetrading.com.invalid> wrote:
> +dev
>
>
> Regards,
> Vivek Kumar
>
> [http://www.eclipsetrading.com/logo.png]
>
> Senior Software Developer
> 23/F One Hennessy
> 1 Hennessy Road
> Wan Chai
> Hong Kong
> www.eclipsetrading.com<http://www.eclipsetrading.com/>
> +852 2108 7352
>
> Follow us today on our online platforms
> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> https://www.instagram.com/eclipsetrading>
> ________________________________
> From: Vivek Kumar <vi...@eclipsetrading.com>
> Sent: Thursday, January 11, 2024 11:07 AM
> To: user@avro.apache.org <us...@avro.apache.org>
> Subject: Avro schema evolution support in AVRO CPP
>
> Hi Avro team,
>
> I am writing this email to check the support of Avro schema evolution in
> CPP - i.e. provide both the producer and consumer schema when decoding the
> data.
>
> I can see that there's a resolvingDecoder function in AVRO CPP that takes
> two schemas. See
>
> https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html#ReadingDifferentSchema
>
> But there's a FIXME comment in this function. See
> https://issues.apache.org/jira/browse/AVRO-3720 and
> https://github.com/apache/avro/blob/main/lang/c%2B%2B/api/Decoder.hh#L218.
> Does this mean resolvingDecoder does not work properly? Could you please
> explain what scenarios are not covered by resolvingDecoder and how can we
> use it to support "Avro Schema Evolution" in c++?
>
> Thanks
>
>
> Regards,
> Vivek Kumar
>
> [http://www.eclipsetrading.com/logo.png]
>
> Senior Software Developer
> 23/F One Hennessy
> 1 Hennessy Road
> Wan Chai
> Hong Kong
> www.eclipsetrading.com<http://www.eclipsetrading.com/>
> +852 2108 7352
>
> Follow us today on our online platforms
> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> https://www.instagram.com/eclipsetrading>
>
Re: Avro schema evolution support in AVRO CPP
Posted by Martin Grigorov <mg...@apache.org>.
Hi Vivek,
I am not sure there is anyone to give you an exact answer. The C++ SDK has
not been actively developed in the last few years.
The best is to try it for your use cases and see if it works or not. The
next step is to contribute Pull Requests for the missing functionalities!
Martin
On Thu, Jan 11, 2024 at 8:59 AM Vivek Kumar <
vivek.kumar@eclipsetrading.com.invalid> wrote:
> +dev
>
>
> Regards,
> Vivek Kumar
>
> [http://www.eclipsetrading.com/logo.png]
>
> Senior Software Developer
> 23/F One Hennessy
> 1 Hennessy Road
> Wan Chai
> Hong Kong
> www.eclipsetrading.com<http://www.eclipsetrading.com/>
> +852 2108 7352
>
> Follow us today on our online platforms
> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> https://www.instagram.com/eclipsetrading>
> ________________________________
> From: Vivek Kumar <vi...@eclipsetrading.com>
> Sent: Thursday, January 11, 2024 11:07 AM
> To: user@avro.apache.org <us...@avro.apache.org>
> Subject: Avro schema evolution support in AVRO CPP
>
> Hi Avro team,
>
> I am writing this email to check the support of Avro schema evolution in
> CPP - i.e. provide both the producer and consumer schema when decoding the
> data.
>
> I can see that there's a resolvingDecoder function in AVRO CPP that takes
> two schemas. See
>
> https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html#ReadingDifferentSchema
>
> But there's a FIXME comment in this function. See
> https://issues.apache.org/jira/browse/AVRO-3720 and
> https://github.com/apache/avro/blob/main/lang/c%2B%2B/api/Decoder.hh#L218.
> Does this mean resolvingDecoder does not work properly? Could you please
> explain what scenarios are not covered by resolvingDecoder and how can we
> use it to support "Avro Schema Evolution" in c++?
>
> Thanks
>
>
> Regards,
> Vivek Kumar
>
> [http://www.eclipsetrading.com/logo.png]
>
> Senior Software Developer
> 23/F One Hennessy
> 1 Hennessy Road
> Wan Chai
> Hong Kong
> www.eclipsetrading.com<http://www.eclipsetrading.com/>
> +852 2108 7352
>
> Follow us today on our online platforms
> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> https://www.instagram.com/eclipsetrading>
>
Re: Avro schema evolution support in AVRO CPP
Posted by Martin Grigorov <mg...@apache.org>.
Hi Vivek,
I am not sure there is anyone to give you an exact answer. The C++ SDK has
not been actively developed in the last few years.
The best is to try it for your use cases and see if it works or not. The
next step is to contribute Pull Requests for the missing functionalities!
Martin
On Thu, Jan 11, 2024 at 8:59 AM Vivek Kumar <
vivek.kumar@eclipsetrading.com.invalid> wrote:
> +dev
>
>
> Regards,
> Vivek Kumar
>
> [http://www.eclipsetrading.com/logo.png]
>
> Senior Software Developer
> 23/F One Hennessy
> 1 Hennessy Road
> Wan Chai
> Hong Kong
> www.eclipsetrading.com<http://www.eclipsetrading.com/>
> +852 2108 7352
>
> Follow us today on our online platforms
> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> https://www.instagram.com/eclipsetrading>
> ________________________________
> From: Vivek Kumar <vi...@eclipsetrading.com>
> Sent: Thursday, January 11, 2024 11:07 AM
> To: user@avro.apache.org <us...@avro.apache.org>
> Subject: Avro schema evolution support in AVRO CPP
>
> Hi Avro team,
>
> I am writing this email to check the support of Avro schema evolution in
> CPP - i.e. provide both the producer and consumer schema when decoding the
> data.
>
> I can see that there's a resolvingDecoder function in AVRO CPP that takes
> two schemas. See
>
> https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html#ReadingDifferentSchema
>
> But there's a FIXME comment in this function. See
> https://issues.apache.org/jira/browse/AVRO-3720 and
> https://github.com/apache/avro/blob/main/lang/c%2B%2B/api/Decoder.hh#L218.
> Does this mean resolvingDecoder does not work properly? Could you please
> explain what scenarios are not covered by resolvingDecoder and how can we
> use it to support "Avro Schema Evolution" in c++?
>
> Thanks
>
>
> Regards,
> Vivek Kumar
>
> [http://www.eclipsetrading.com/logo.png]
>
> Senior Software Developer
> 23/F One Hennessy
> 1 Hennessy Road
> Wan Chai
> Hong Kong
> www.eclipsetrading.com<http://www.eclipsetrading.com/>
> +852 2108 7352
>
> Follow us today on our online platforms
> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]<
> https://www.linkedin.com/company/eclipse-trading>[Instagram]<
> https://www.instagram.com/eclipsetrading>
>