Enum order change

[Arnaud BOS <ar...@gmail.com>]

Quick question: is reordering the values of an enum a backward compatible
change?

Per the Avro 1.8.2 documentation: "An enum is encoded by a int,
representing the zero-based position of the symbol in the schema."

If my enum is:

{"type": "enum", "name": "Foo", "symbols": ["A", "B", "C", "D"] }

And I update it to

{"type": "enum", "name": "Foo", "symbols": ["A", "B", "C", "E", "F", "D"] }


If a writer with the older schema will produce messages using enum value
"D" encoded as int 3.
Then a reader with the newer schema will consume messages with value 3 and
decoded it as enum value "E".

If this is the case, then reordering enums is just as non-backward
compatible as adding new values is non-forward compatible, is it correct?

Re: Enum order change

[Arnaud BOS <ar...@gmail.com>]

We're indeed using the writer's schema so we're not having the issue but I
was curious.

I was referring to schema resolution as pointed out, so I should have RTFM
more thoroughly I guess ¯\_(ツ)_/¯

Thank you both for your answers.

Le mer. 30 mai 2018 à 16:49, Bridger Howell <bh...@sofi.org> a écrit :

> > Then a reader with the newer schema will consume messages with value 3
> and decoded it as enum value "E".
>
> If you try to consume the old data treating it as if it were written with
> the newer schema, then yes that would cause issues, but that's normally how
> avro is used.
>
> For avro compatibility to work normally, it's expected that the writer's
> schema is available when reading the data, removing the sorts of issues you
> pointed out.
> The transformation of data written with one schema into a different schema
> is called schema resolution, and you can read more about it here:
> https://avro.apache.org/docs/current/spec.html#Schema+Resolution
>
> On Wed, May 30, 2018 at 6:35 AM, Arnaud BOS <ar...@gmail.com> wrote:
>
>> Quick question: is reordering the values of an enum a backward compatible
>> change?
>>
>> Per the Avro 1.8.2 documentation: "An enum is encoded by a int,
>> representing the zero-based position of the symbol in the schema."
>>
>> If my enum is:
>>
>> {"type": "enum", "name": "Foo", "symbols": ["A", "B", "C", "D"] }
>>
>> And I update it to
>>
>> {"type": "enum", "name": "Foo", "symbols": ["A", "B", "C", "E", "F", "D"] }
>>
>>
>> If a writer with the older schema will produce messages using enum value
>> "D" encoded as int 3.
>> Then a reader with the newer schema will consume messages with value 3
>> and decoded it as enum value "E".
>>
>> If this is the case, then reordering enums is just as non-backward
>> compatible as adding new values is non-forward compatible, is it correct?
>>
>
> The information contained in this email message is PRIVATE and intended
> only for the personal and confidential use of the recipient named above. If
> the reader of this message is not the intended recipient or an agent
> responsible for delivering it to the intended recipient, you are hereby
> notified that you have received this message in error and that any review,
> dissemination, distribution or copying of this message is strictly
> prohibited.  If you have received this communication in error, please
> notify us immediately by email, and delete the original message.
>

Re: Enum order change

[Bridger Howell <bh...@sofi.org>]

> Then a reader with the newer schema will consume messages with value 3
and decoded it as enum value "E".

If you try to consume the old data treating it as if it were written with
the newer schema, then yes that would cause issues, but that's normally how
avro is used.

For avro compatibility to work normally, it's expected that the writer's
schema is available when reading the data, removing the sorts of issues you
pointed out.
The transformation of data written with one schema into a different schema
is called schema resolution, and you can read more about it here:
https://avro.apache.org/docs/current/spec.html#Schema+Resolution

On Wed, May 30, 2018 at 6:35 AM, Arnaud BOS <ar...@gmail.com> wrote:

> Quick question: is reordering the values of an enum a backward compatible
> change?
>
> Per the Avro 1.8.2 documentation: "An enum is encoded by a int,
> representing the zero-based position of the symbol in the schema."
>
> If my enum is:
>
> {"type": "enum", "name": "Foo", "symbols": ["A", "B", "C", "D"] }
>
> And I update it to
>
> {"type": "enum", "name": "Foo", "symbols": ["A", "B", "C", "E", "F", "D"] }
>
>
> If a writer with the older schema will produce messages using enum value
> "D" encoded as int 3.
> Then a reader with the newer schema will consume messages with value 3 and
> decoded it as enum value "E".
>
> If this is the case, then reordering enums is just as non-backward
> compatible as adding new values is non-forward compatible, is it correct?
>

-- 


The information contained in this email message is PRIVATE and intended 
only for the personal and confidential use of the recipient named above. If 
the reader of this message is not the intended recipient or an agent 
responsible for delivering it to the intended recipient, you are hereby 
notified that you have received this message in error and that any review, 
dissemination, distribution or copying of this message is strictly 
prohibited.  If you have received this communication in error, please 
notify us immediately by email, and delete the original message.

Re: Enum order change

[Zoltan Farkas <zo...@yahoo.com>]

They will be compatible as long as you use the writer and reader schemas to decode the message...
(The symbol names are used to map between your writer and reader schema…)

—Z

> On May 30, 2018, at 8:35 AM, Arnaud BOS <ar...@gmail.com> wrote:
> 
> Quick question: is reordering the values of an enum a backward compatible change?
> 
> Per the Avro 1.8.2 documentation: "An enum is encoded by a int, representing the zero-based position of the symbol in the schema."
> 
> If my enum is: 
> {"type": "enum", "name": "Foo", "symbols": ["A", "B", "C", "D"] }
> And I update it to
> {"type": "enum", "name": "Foo", "symbols": ["A", "B", "C", "E", "F", "D"] }
> 
> If a writer with the older schema will produce messages using enum value "D" encoded as int 3.
> Then a reader with the newer schema will consume messages with value 3 and decoded it as enum value "E".
> 
> If this is the case, then reordering enums is just as non-backward compatible as adding new values is non-forward compatible, is it correct?