Data migration using persistent Queues

[ipsxd <ma...@boostit.com>]

Let's imagine the following scenario. We have a queue (distributed in a
cluster and partitioned) with a persistent storage. We add Person ( name :
String ) to it. Then, because of business requirements we will add another
field let's say address which is also o String. What options do we have to
do data migration to support new model and be backwards compatible ?

Possible options that I found :

1. Create a new queue and migrate data programatically - the application
will use the new queue.
2. Store data in another format let's say JSON and provide backwards
compatibility at code level.
3. Is possible to use inheritance, but then we will have versioning in model
which is not desirable.

I think this problem arises also in the context of in memory queues.

Do you have other idea ?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Data migration using persistent Queues

[ipsxd <ma...@boostit.com>]

Yes, Ignite Queue. In the mean time I did some tests and it seems that adding
fields is not a problem but removing or doing some more complicated stuff
would be. 

Another option is to implement Externalizable and implement the logic there.
So I think a good approach would be a combination between methods. Additions
are backwards compatible so this feature will be used. For incompatible
changes migrations of data to other queues will be used. To have a
complicated code in serialization/deserialization is not desirable either as
this would affect performance. 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Data migration using persistent Queues

[ezhuravlev <e....@gmail.com>]

Sorry, here is the link for the BinaryObject doc:
https://apacheignite.readme.io/docs/binary-marshaller



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Data migration using persistent Queues

[ezhuravlev <e....@gmail.com>]

Hi,

Do you mean Ignite Queue? Or just your data structure that stored in simple
Ignite cache?

> Store data in another format let's say JSON and provide backwards
> compatibility at code level.
There is no need for this, Ignite internally stores everything as
BinaryObject and this format supports schema changes: Store data in another
format let's say JSON and provide backwards 
compatibility at code level.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/