You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "Durity, Sean R" <SE...@homedepot.com> on 2019/06/17 13:36:47 UTC
RE: [EXTERNAL] Re: Cassandra migration from 1.25 to 3.x
The advice so far is exactly correct for an in-place kind of upgrade. The blog post you mentioned is different. They decided to jump versions in Cassandra by standing up a new cluster and using a dual-write/dual-read process for their app. They also wrote code to read and interpret sstables in order to migrate existing data. Getting that right with compaction running, data consistency, etc. it not easy. That is what Cassandra does, of course. They had to reverse engineer that process.
I would not personally take that path as it seems a more difficult way to go -- for the DBA/admin. It is a nice path for the development team, though. They only had to look at their reads and writes (already encapsulated in a DAO) for the dual clusters. In a multi-upgrade scenario, drivers and statements probably have to get upgraded at several steps along the way (including a move from Thrift to CQL, probably). More app testing is required each upgrade. So, the decision has to be based on which resources you have and trust (app dev and testing + Cassandra upgrades or data migration and testing). Once you have automated/semi-automated Cassandra upgrades in place, that is an easier path, but that company obviously hadn't invested there.
Sean Durity
-----Original Message-----
From: Michael Shuler <ms...@pbandjelly.org> On Behalf Of Michael Shuler
Sent: Monday, June 17, 2019 8:26 AM
To: user@cassandra.apache.org
Subject: [EXTERNAL] Re: Cassandra migration from 1.25 to 3.x
First and foremost, read NEWS.txt from your current version to the version you wish to upgrade to. There are too may details that you many need to be aware of. For instance, in the 2.0.0 Upgrading notes:
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_cassandra_blob_cassandra-2D3.11_NEWS.txt-23L1169-2DL1178&d=DwIDaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=CPWy2XBaDFHzVDapNO4E7kLIFFfbkRd8KqftSrypjSU&s=D3Y18E9gxewpushCMETjHt9cS8lKvLMrhUdhPriF4Dk&e=
I assume you meant 1.2.5, so you're first step is to upgrade to at least
1.2.9 (I would suggest using latest 1.2.x, which is 1.2.19). Then you can to to 2.0.x and up.
Practicing on a scratch cluster is valuable experience. Reading the upgrade notes in NEWS.txt is a must.
--
Kind regards,
Michael
On 6/17/19 3:34 AM, Anurag Sharma wrote:
> Thanks Alex,
>
> I came across some interesting and efficient ways of upgrading from
> 1.x to 3.x as described in the blog here
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__medium.com_-40ro
> eezilkha_database-2Dmigration-2Dat-2Dscale-2Dae85c14c3621&d=DwIDaQ&c=M
> tgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=
> CPWy2XBaDFHzVDapNO4E7kLIFFfbkRd8KqftSrypjSU&s=7ekpEXHT1Qm_xL9l6_1Kty32
> fDDerlB_PgO1-4K1-VQ&e= > and others. Was curious if someone has
> open-sourced their custom utility. :D
>
> Regards
> Anurag
>
> On Mon, Jun 17, 2019 at 1:27 PM Oleksandr Shulgin
> <oleksandr.shulgin@zalando.de <ma...@zalando.de>> wrote:
>
> On Mon, Jun 17, 2019 at 9:30 AM Anurag Sharma
> <anurag.rp.sharma@gmail.com <ma...@gmail.com>> wrote:
>
>
> We are upgrading Cassandra from 1.25 to 3.X. Just curious if
> there is any recommended open source utility for the same.
>
>
> Hi,
>
> The "recommended open source utility" is the Apache Cassandra
> itself. ;-)
>
> Given the huge difference between the major versions, though, you
> will need a decent amount of planning and preparation to
> successfully complete such a migration. Most likely you will want
> to do it in small steps, first upgrading to the latest minor version
> in the 1.x series, then making a jump to 2.x, then to 3.0, and only
> then to 3.x if you really mean to. On each upgrade step, be sure to
> examine the release notes carefully to understand if there is any
> impact for your cluster and/or client applications. Do have a test
> system with preferably identical setup and configuration and execute
> the upgrade steps there first to verify your expectations.
>
> Good luck!
> --
> Alex
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
________________________________
The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
Re: [EXTERNAL] Re: Cassandra migration from 1.25 to 3.x
Posted by Anurag Sharma <an...@gmail.com>.
Thanks Michael and Sean. That was very useful.
Regards
Anurag
On Mon, Jun 17, 2019 at 7:07 PM Durity, Sean R <SE...@homedepot.com>
wrote:
> The advice so far is exactly correct for an in-place kind of upgrade. The
> blog post you mentioned is different. They decided to jump versions in
> Cassandra by standing up a new cluster and using a dual-write/dual-read
> process for their app. They also wrote code to read and interpret sstables
> in order to migrate existing data. Getting that right with compaction
> running, data consistency, etc. it not easy. That is what Cassandra does,
> of course. They had to reverse engineer that process.
>
> I would not personally take that path as it seems a more difficult way to
> go -- for the DBA/admin. It is a nice path for the development team,
> though. They only had to look at their reads and writes (already
> encapsulated in a DAO) for the dual clusters. In a multi-upgrade scenario,
> drivers and statements probably have to get upgraded at several steps along
> the way (including a move from Thrift to CQL, probably). More app testing
> is required each upgrade. So, the decision has to be based on which
> resources you have and trust (app dev and testing + Cassandra upgrades or
> data migration and testing). Once you have automated/semi-automated
> Cassandra upgrades in place, that is an easier path, but that company
> obviously hadn't invested there.
>
> Sean Durity
>
> -----Original Message-----
> From: Michael Shuler <ms...@pbandjelly.org> On Behalf Of Michael Shuler
> Sent: Monday, June 17, 2019 8:26 AM
> To: user@cassandra.apache.org
> Subject: [EXTERNAL] Re: Cassandra migration from 1.25 to 3.x
>
> First and foremost, read NEWS.txt from your current version to the version
> you wish to upgrade to. There are too may details that you many need to be
> aware of. For instance, in the 2.0.0 Upgrading notes:
>
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_cassandra_blob_cassandra-2D3.11_NEWS.txt-23L1169-2DL1178&d=DwIDaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=CPWy2XBaDFHzVDapNO4E7kLIFFfbkRd8KqftSrypjSU&s=D3Y18E9gxewpushCMETjHt9cS8lKvLMrhUdhPriF4Dk&e=
>
> I assume you meant 1.2.5, so you're first step is to upgrade to at least
> 1.2.9 (I would suggest using latest 1.2.x, which is 1.2.19). Then you can
> to to 2.0.x and up.
>
> Practicing on a scratch cluster is valuable experience. Reading the
> upgrade notes in NEWS.txt is a must.
>
> --
> Kind regards,
> Michael
>
> On 6/17/19 3:34 AM, Anurag Sharma wrote:
> > Thanks Alex,
> >
> > I came across some interesting and efficient ways of upgrading from
> > 1.x to 3.x as described in the blog here
> > <https://urldefense.proofpoint.com/v2/url?u=https-3A__medium.com_-40ro
> > eezilkha_database-2Dmigration-2Dat-2Dscale-2Dae85c14c3621&d=DwIDaQ&c=M
> > tgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=
> > CPWy2XBaDFHzVDapNO4E7kLIFFfbkRd8KqftSrypjSU&s=7ekpEXHT1Qm_xL9l6_1Kty32
> > fDDerlB_PgO1-4K1-VQ&e= > and others. Was curious if someone has
> > open-sourced their custom utility. :D
> >
> > Regards
> > Anurag
> >
> > On Mon, Jun 17, 2019 at 1:27 PM Oleksandr Shulgin
> > <oleksandr.shulgin@zalando.de <ma...@zalando.de>>
> wrote:
> >
> > On Mon, Jun 17, 2019 at 9:30 AM Anurag Sharma
> > <anurag.rp.sharma@gmail.com <ma...@gmail.com>>
> wrote:
> >
> >
> > We are upgrading Cassandra from 1.25 to 3.X. Just curious if
> > there is any recommended open source utility for the same.
> >
> >
> > Hi,
> >
> > The "recommended open source utility" is the Apache Cassandra
> > itself. ;-)
> >
> > Given the huge difference between the major versions, though, you
> > will need a decent amount of planning and preparation to
> > successfully complete such a migration. Most likely you will want
> > to do it in small steps, first upgrading to the latest minor version
> > in the 1.x series, then making a jump to 2.x, then to 3.0, and only
> > then to 3.x if you really mean to. On each upgrade step, be sure to
> > examine the release notes carefully to understand if there is any
> > impact for your cluster and/or client applications. Do have a test
> > system with preferably identical setup and configuration and execute
> > the upgrade steps there first to verify your expectations.
> >
> > Good luck!
> > --
> > Alex
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>
>
> ________________________________
>
> The information in this Internet Email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this Email
> by anyone else is unauthorized. If you are not the intended recipient, any
> disclosure, copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful. When addressed
> to our clients any opinions or advice contained in this Email are subject
> to the terms and conditions expressed in any applicable governing The Home
> Depot terms of business or client engagement letter. The Home Depot
> disclaims all responsibility and liability for the accuracy and content of
> this attachment and for any damages or losses arising from any
> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
> items of a destructive nature, which may be contained in this attachment
> and shall not be liable for direct, indirect, consequential or special
> damages in connection with this e-mail message or its attachment.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>