You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Klaus Kraft <kl...@gmail.com> on 2014/05/11 16:31:00 UTC
Handling data migration/schema evolution in Avro
Hi all,
I know that Avro supports schema evolution when it comes to adding,
deleting or renaming fields. However, I wondered whether it is possible to
deal with more complex evolution scenarios too, e.g. what if I want to move
a field to another record or what if I split one field into two fields or
merge two fields into one etc.
Is there any way to handle such scenarios in Avro? The most important
aspect is that I don't want to lose data. Is it possible to write explicit
migration classes in Java (thats the target programming language I am
using) or can't Avro deal with such scenarios?
Thanks in advance.
Re: Handling data migration/schema evolution in Avro
Posted by Lewis John Mcgibbney <le...@gmail.com>.
This seems more complex indeed. I've been reading as much as I can on this
topic recently and haven't seen anything testing such use cases...
For reference I thought it may be useful for other here for me to mention
the koji documentation on schema evolution... I found this a very helpful
resource.
I am sorry I don't have an answer right now... my use cases haven't come
this far down the schema evolution road yet!
On May 11, 2014 7:31 AM, "Klaus Kraft" <kl...@gmail.com> wrote:
> Hi all,
>
> I know that Avro supports schema evolution when it comes to adding,
> deleting or renaming fields. However, I wondered whether it is possible to
> deal with more complex evolution scenarios too, e.g. what if I want to move
> a field to another record or what if I split one field into two fields or
> merge two fields into one etc.
>
> Is there any way to handle such scenarios in Avro? The most important
> aspect is that I don't want to lose data. Is it possible to write explicit
> migration classes in Java (thats the target programming language I am
> using) or can't Avro deal with such scenarios?
>
> Thanks in advance.
>