You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@openjpa.apache.org by Todd Nine <to...@spidertracks.com> on 2014/03/09 18:54:24 UTC
Utilizing a slice for online migration
Hi all,
We're migrating from a Cassandra based JPA adapter to Amazon's RDS. In
order to do this, we want a no downtime migration. To do this, we would
need the following flow.
1) Start writing to both systems. Cassandra is still the authoritative
record, and records are replicated to RDS. All queries will still be served
from Cassandra on read.
2) In the background, read all records from Cassandra and write them to
RDS. Use update timestamps to ensure we don't overwrite newly updated data.
3) Switch our read path (probably with a new deployment configured to only
read from RDS)
I've been looking at the Slices documentation, and it seems like defining a
Data Replication Policy will perform the dual writes I need in steps 1) and
2). Does anyone have any experience with using the policy in this manner?
Thanks in advance,
Todd
Re: Utilizing a slice for online migration
Posted by Pinaki Poddar <pp...@apache.org>.
Yes, you are on the right track.
I have considered that Slice can be used for data migration. Read data from
one Slice and if the data is touched, write it to another.
Nowadays, I do not spend enough time on OpenJPA to help with you some real
code/prototype. But with little digging (especially where Slice writes the
same record to multiple databases for @Replicated types) -- you can use it
for online migration.
Regards --
-----
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context: http://openjpa.208410.n2.nabble.com/Utilizing-a-slice-for-online-migration-tp7586062p7586727.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Utilizing a slice for online migration
Posted by Todd Nine <to...@spidertracks.com>.
Hi Rick,
Unfortunately I can't seem to use slice, since the Hector JPA plugin is
not a JDBC based plugin. I'm now at the point of having 2 entity managers,
1 for Cassandra via Hector JPA, and another with MySQL. Do you know of
anyone that has successfully implemented a proxy Entity Manager that can
delegate to other entity managers? Our objects are always in a detached
state, so we don't have to worry about transactions across both system.
We're utilizing optimistic concurrency for all of our entities so this also
should give us a bit of relief.
Thanks in advance,
Todd
On 11 March 2014 07:04, Rick Curtis <cu...@gmail.com> wrote:
> Todd -
>
> > Does anyone have any experience with using the policy in this manner?
> My general feeling is that not many people are using slice as there has
> been very little mailing list traffic regarding it's usage. Be sure to let
> us know how it goes!
>
> Thanks,
> Rick
>
>
> On Sun, Mar 9, 2014 at 12:54 PM, Todd Nine <to...@spidertracks.com> wrote:
>
> > Hi all,
> > We're migrating from a Cassandra based JPA adapter to Amazon's RDS. In
> > order to do this, we want a no downtime migration. To do this, we would
> > need the following flow.
> >
> > 1) Start writing to both systems. Cassandra is still the authoritative
> > record, and records are replicated to RDS. All queries will still be
> served
> > from Cassandra on read.
> >
> > 2) In the background, read all records from Cassandra and write them to
> > RDS. Use update timestamps to ensure we don't overwrite newly updated
> > data.
> >
> > 3) Switch our read path (probably with a new deployment configured to
> only
> > read from RDS)
> >
> >
> > I've been looking at the Slices documentation, and it seems like
> defining a
> > Data Replication Policy will perform the dual writes I need in steps 1)
> and
> > 2). Does anyone have any experience with using the policy in this
> manner?
> >
> > Thanks in advance,
> > Todd
> >
>
>
>
> --
> *Rick Curtis*
>
Re: Utilizing a slice for online migration
Posted by Rick Curtis <cu...@gmail.com>.
> Your post seems to be in rather poor taste for a open source developer.
I disagree as my post had the best interest of the developer in mind. From
my point of view, slice doesn't come up on the list very often and when it
does, there isn't anyone with experience to help. OpenJPA has very, very
few people supporting this community these days and I don't want to steer
people toward a feature that they won't get much/any help with. It sucks,
but that's the way it is.
On Sat, May 31, 2014 at 2:11 AM, Pinaki Poddar <pp...@apache.org> wrote:
> Hello Rick,
> Your post seems to be in rather poor taste for a open source developer.
> Instead of encouraging users to use a module from a project of which you
> are a member and the module has been in operation for last four years and
> have been successfully used by others, your answer seem to be: "Nobody uses
> Slice, so do not bother". It is not only factually incorrect, it is
> inappropriate.
>
>
>
>
>
>
>
> -----
> Pinaki Poddar
> Chair, Apache OpenJPA Project
> --
> View this message in context:
> http://openjpa.208410.n2.nabble.com/Utilizing-a-slice-for-online-migration-tp7586062p7586726.html
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
--
*Rick Curtis*
Re: Utilizing a slice for online migration
Posted by Pinaki Poddar <pp...@apache.org>.
Hello Rick,
Your post seems to be in rather poor taste for a open source developer.
Instead of encouraging users to use a module from a project of which you
are a member and the module has been in operation for last four years and
have been successfully used by others, your answer seem to be: "Nobody uses
Slice, so do not bother". It is not only factually incorrect, it is
inappropriate.
-----
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context: http://openjpa.208410.n2.nabble.com/Utilizing-a-slice-for-online-migration-tp7586062p7586726.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Utilizing a slice for online migration
Posted by Rick Curtis <cu...@gmail.com>.
Todd -
> Does anyone have any experience with using the policy in this manner?
My general feeling is that not many people are using slice as there has
been very little mailing list traffic regarding it's usage. Be sure to let
us know how it goes!
Thanks,
Rick
On Sun, Mar 9, 2014 at 12:54 PM, Todd Nine <to...@spidertracks.com> wrote:
> Hi all,
> We're migrating from a Cassandra based JPA adapter to Amazon's RDS. In
> order to do this, we want a no downtime migration. To do this, we would
> need the following flow.
>
> 1) Start writing to both systems. Cassandra is still the authoritative
> record, and records are replicated to RDS. All queries will still be served
> from Cassandra on read.
>
> 2) In the background, read all records from Cassandra and write them to
> RDS. Use update timestamps to ensure we don't overwrite newly updated
> data.
>
> 3) Switch our read path (probably with a new deployment configured to only
> read from RDS)
>
>
> I've been looking at the Slices documentation, and it seems like defining a
> Data Replication Policy will perform the dual writes I need in steps 1) and
> 2). Does anyone have any experience with using the policy in this manner?
>
> Thanks in advance,
> Todd
>
--
*Rick Curtis*