You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco)" <hv...@cisco.com> on 2017/08/01 21:24:34 UTC
Cassandra Data migration from 2.2.3 to 3.7
Please help me understand data migration path from 2.2.3 to 3.7.
1. Tried to take snapshot and load the snapshot using sstableloader; but it is throwing Streaming Exception.
Please advise.
Thanks,
Harika
[http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
Harika Vangapelli
Engineer - IT
hvangape@cisco.com<ma...@cisco.com>
Tel:
Cisco Systems, Inc.
United States
cisco.com
[http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
Re: Cassandra Data migration from 2.2.3 to 3.7
Posted by Jeff Jirsa <jj...@gmail.com>.
Anytime you copy sstables around (sstableloader is excluded here), make sure you don't copy sstables from 2 nodes to the same directories without checking that the names don't collide or you'll lose one of the sstables
--
Jeff Jirsa
> On Aug 2, 2017, at 10:38 AM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com> wrote:
>
> Jeff, what is the meaning of this line you mentioned in below email..
> ‘making sure you dont overwrite any sstables with the same name’
>
> <image001.png>
>
> Harika Vangapelli
> Engineer - IT
> hvangape@cisco.com
> Tel:
> Cisco Systems, Inc.
>
>
>
> United States
> cisco.com
> <image002.gif>Think before you print.
> This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
> Please click here for Company Registration Information.
>
> From: Jeff Jirsa [mailto:jjirsa@gmail.com]
> Sent: Tuesday, August 1, 2017 5:13 PM
> To: cassandra <us...@cassandra.apache.org>
> Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
>
> Roughly that approach should work, modulo copying data in/out to the right paths, and making sure you dont overwrite any sstables with the same name.
>
> (You dont need to refresh after the upgradesstables unless you've got 2 3.7 clusters, and you may need to refresh before the upgradesstables if the 3.7 cluster in step 3 is already running)
>
>
> On Tue, Aug 1, 2017 at 3:44 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com> wrote:
> Jeff, I tried the below steps for just 3 rows of data, It looks to be working. But Confirm me this is right approach.
> 1. Taking snapshot (using nodetool snapshot) in the existing cluster (2.2 cluster)
>
> 2. Bring the snapshot to new cluster and copy in keyspce/table (2.2->3.7)
>
> 3. Nodetool upgrade sstbales on new cluster (3.7)
>
> 4. Use noedtool refresh -- <keyspace> <table> (3.7)
>
> Does this approach work?
>
> <image001.png>
>
> Harika Vangapelli
> Engineer - IT
> hvangape@cisco.com
> Tel:
> Cisco Systems, Inc.
>
>
>
> United States
> cisco.com
>
> <image002.gif>Think before you print.
> This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
> Please click here for Company Registration Information.
>
> From: Jeff Jirsa [mailto:jjirsa@gmail.com]
> Sent: Tuesday, August 1, 2017 3:36 PM
>
> To: user@cassandra.apache.org
> Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
>
> You can't stream across versions, so you either need to use the native protocol (do writes with the normal INSERT operations) or you have to get to the right sstable version
>
> That means one of the three of these:
>
> - export to CSV (copy to/copy from, or Brian Hess' Cassandra-loader on github)
> - export to CSV and use CQLSSTableWriter to generate sstables you can bulk load
> - create a minimal cluster, load the sstables in, and run upgradesstables to rev the version and then stop that cluster and stream them into the new cluster
>
>
> --
> Jeff Jirsa
>
>
> On Aug 1, 2017, at 3:30 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com> wrote:
>
> Jeff, Thanks for initial pointer. I want to add some more information to it.
>
> This is to migrate from one cluster (2.2 version) to 3.7 . The client who is in 2.2 cluster they don’t need down time.
>
> So How the data is migrated to new cluster without down time with different versions?
>
> Thanks,
> Harika
>
> <image001.png>
>
> Harika Vangapelli
> Engineer - IT
> hvangape@cisco.com
> Tel:
> Cisco Systems, Inc.
>
>
>
> United States
> cisco.com
>
> <image002.gif>Think before you print.
> This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
> Please click here for Company Registration Information.
>
> From: Jeff Jirsa [mailto:jjirsa@gmail.com]
> Sent: Tuesday, August 1, 2017 2:48 PM
> To: user@cassandra.apache.org
> Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
>
> If you upgrade in place:
>
> Stop each node one at a time
> Swap binaries
> Start each node
> Run upgradesstables when all nodes are bounced
>
> If you want to take data from 2.2 and put it into a 3.x cluster, you'll need to run upgradesstables on it offline somehow (like clone that 2.2 cluster/schema and run it offline with 3.x upgradesstables), then sstableloader it into place
>
> Note that 3.7 is probably not the version you want - 3.11 is probably much more stable and solves some serious bugs (or 3.0.14 if you don't need features from 3.7 ). The one exception to this would be if you're using a 3.7 lts branch (like instaclustr had at one point - and may still have).
>
>
> --
> Jeff Jirsa
>
>
> On Aug 1, 2017, at 2:24 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com> wrote:
>
> Please help me understand data migration path from 2.2.3 to 3.7.
>
> 1. Tried to take snapshot and load the snapshot using sstableloader; but it is throwing Streaming Exception.
>
> Please advise.
>
> Thanks,
> Harika
>
>
> <image001.png>
>
> Harika Vangapelli
> Engineer - IT
> hvangape@cisco.com
> Tel:
> Cisco Systems, Inc.
>
>
>
> United States
> cisco.com
>
> <image002.gif>Think before you print.
> This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
> Please click here for Company Registration Information.
>
>
RE: Cassandra Data migration from 2.2.3 to 3.7
Posted by "Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco)" <hv...@cisco.com>.
Jeff, what is the meaning of this line you mentioned in below email..
‘making sure you dont overwrite any sstables with the same name’
[http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
Harika Vangapelli
Engineer - IT
hvangape@cisco.com<ma...@cisco.com>
Tel:
Cisco Systems, Inc.
United States
cisco.com
[http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
From: Jeff Jirsa [mailto:jjirsa@gmail.com]
Sent: Tuesday, August 1, 2017 5:13 PM
To: cassandra <us...@cassandra.apache.org>
Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
Roughly that approach should work, modulo copying data in/out to the right paths, and making sure you dont overwrite any sstables with the same name.
(You dont need to refresh after the upgradesstables unless you've got 2 3.7 clusters, and you may need to refresh before the upgradesstables if the 3.7 cluster in step 3 is already running)
On Tue, Aug 1, 2017 at 3:44 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com>> wrote:
Jeff, I tried the below steps for just 3 rows of data, It looks to be working. But Confirm me this is right approach.
1. Taking snapshot (using nodetool snapshot) in the existing cluster (2.2 cluster)
2. Bring the snapshot to new cluster and copy in keyspce/table (2.2->3.7)
3. Nodetool upgrade sstbales on new cluster (3.7)
4. Use noedtool refresh -- <keyspace> <table> (3.7)
Does this approach work?
[http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
Harika Vangapelli
Engineer - IT
hvangape@cisco.com<ma...@cisco.com>
Tel:
Cisco Systems, Inc.
United States
cisco.com<http://cisco.com>
[http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
From: Jeff Jirsa [mailto:jjirsa@gmail.com<ma...@gmail.com>]
Sent: Tuesday, August 1, 2017 3:36 PM
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
You can't stream across versions, so you either need to use the native protocol (do writes with the normal INSERT operations) or you have to get to the right sstable version
That means one of the three of these:
- export to CSV (copy to/copy from, or Brian Hess' Cassandra-loader on github)
- export to CSV and use CQLSSTableWriter to generate sstables you can bulk load
- create a minimal cluster, load the sstables in, and run upgradesstables to rev the version and then stop that cluster and stream them into the new cluster
--
Jeff Jirsa
On Aug 1, 2017, at 3:30 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com>> wrote:
Jeff, Thanks for initial pointer. I want to add some more information to it.
This is to migrate from one cluster (2.2 version) to 3.7 . The client who is in 2.2 cluster they don’t need down time.
So How the data is migrated to new cluster without down time with different versions?
Thanks,
Harika
<image001.png>
Harika Vangapelli
Engineer - IT
hvangape@cisco.com<ma...@cisco.com>
Tel:
Cisco Systems, Inc.
United States
cisco.com<http://cisco.com>
<image002.gif>Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
From: Jeff Jirsa [mailto:jjirsa@gmail.com]
Sent: Tuesday, August 1, 2017 2:48 PM
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
If you upgrade in place:
Stop each node one at a time
Swap binaries
Start each node
Run upgradesstables when all nodes are bounced
If you want to take data from 2.2 and put it into a 3.x cluster, you'll need to run upgradesstables on it offline somehow (like clone that 2.2 cluster/schema and run it offline with 3.x upgradesstables), then sstableloader it into place
Note that 3.7 is probably not the version you want - 3.11 is probably much more stable and solves some serious bugs (or 3.0.14 if you don't need features from 3.7 ). The one exception to this would be if you're using a 3.7 lts branch (like instaclustr had at one point - and may still have).
--
Jeff Jirsa
On Aug 1, 2017, at 2:24 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com>> wrote:
Please help me understand data migration path from 2.2.3 to 3.7.
1. Tried to take snapshot and load the snapshot using sstableloader; but it is throwing Streaming Exception.
Please advise.
Thanks,
Harika
<image001.png>
Harika Vangapelli
Engineer - IT
hvangape@cisco.com<ma...@cisco.com>
Tel:
Cisco Systems, Inc.
United States
cisco.com<http://cisco.com>
<image002.gif>Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
Re: Cassandra Data migration from 2.2.3 to 3.7
Posted by Jeff Jirsa <jj...@gmail.com>.
Roughly that approach should work, modulo copying data in/out to the right
paths, and making sure you dont overwrite any sstables with the same name.
(You dont need to refresh after the upgradesstables unless you've got 2 3.7
clusters, and you may need to refresh before the upgradesstables if the 3.7
cluster in step 3 is already running)
On Tue, Aug 1, 2017 at 3:44 PM, Harika Vangapelli -T (hvangape - AKRAYA INC
at Cisco) <hv...@cisco.com> wrote:
> Jeff, I tried the below steps for just 3 rows of data, It looks to be
> working. But Confirm me this is right approach.
>
> 1. Taking snapshot (using nodetool snapshot) in the existing
> cluster (2.2 cluster)
>
> 2. Bring the snapshot to new cluster and copy in keyspce/table
> (2.2->3.7)
>
> 3. Nodetool upgrade sstbales on new cluster (3.7)
>
> 4. Use noedtool refresh -- <keyspace> <table> (3.7)
>
> Does this approach work?
>
>
>
> [image:
> http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
>
>
>
> *Harika Vangapelli*
>
> Engineer - IT
>
> hvangape@cisco.com
>
> Tel:
>
> *Cisco Systems, Inc.*
>
>
>
>
> United States
> cisco.com
>
>
>
> [image: http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think
> before you print.
>
> This email may contain confidential and privileged material for the sole
> use of the intended recipient. Any review, use, distribution or disclosure
> by others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by
> reply email and delete all copies of this message.
>
> Please click here
> <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for
> Company Registration Information.
>
>
>
> *From:* Jeff Jirsa [mailto:jjirsa@gmail.com]
> *Sent:* Tuesday, August 1, 2017 3:36 PM
>
> *To:* user@cassandra.apache.org
> *Subject:* Re: Cassandra Data migration from 2.2.3 to 3.7
>
>
>
> You can't stream across versions, so you either need to use the native
> protocol (do writes with the normal INSERT operations) or you have to get
> to the right sstable version
>
>
>
> That means one of the three of these:
>
>
>
> - export to CSV (copy to/copy from, or Brian Hess' Cassandra-loader on
> github)
>
> - export to CSV and use CQLSSTableWriter to generate sstables you can bulk
> load
>
> - create a minimal cluster, load the sstables in, and run upgradesstables
> to rev the version and then stop that cluster and stream them into the new
> cluster
>
>
>
> --
>
> Jeff Jirsa
>
>
>
>
> On Aug 1, 2017, at 3:30 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at
> Cisco) <hv...@cisco.com> wrote:
>
> Jeff, Thanks for initial pointer. I want to add some more information to
> it.
>
>
>
> This is to migrate from one cluster (2.2 version) to 3.7 . The client who
> is in 2.2 cluster they don’t need down time.
>
>
>
> So How the data is migrated to new cluster without down time with
> different versions?
>
>
>
> Thanks,
>
> Harika
>
>
>
> <image001.png>
>
>
>
> *Harika Vangapelli*
>
> Engineer - IT
>
> hvangape@cisco.com
>
> Tel:
>
> *Cisco Systems, Inc.*
>
>
>
>
> United States
> cisco.com
>
>
>
> <image002.gif>Think before you print.
>
> This email may contain confidential and privileged material for the sole
> use of the intended recipient. Any review, use, distribution or disclosure
> by others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by
> reply email and delete all copies of this message.
>
> Please click here
> <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for
> Company Registration Information.
>
>
>
> *From:* Jeff Jirsa [mailto:jjirsa@gmail.com <jj...@gmail.com>]
> *Sent:* Tuesday, August 1, 2017 2:48 PM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Cassandra Data migration from 2.2.3 to 3.7
>
>
>
> If you upgrade in place:
>
>
>
> Stop each node one at a time
>
> Swap binaries
>
> Start each node
>
> Run upgradesstables when all nodes are bounced
>
>
>
> If you want to take data from 2.2 and put it into a 3.x cluster, you'll
> need to run upgradesstables on it offline somehow (like clone that 2.2
> cluster/schema and run it offline with 3.x upgradesstables), then
> sstableloader it into place
>
>
>
> Note that 3.7 is probably not the version you want - 3.11 is probably much
> more stable and solves some serious bugs (or 3.0.14 if you don't need
> features from 3.7 ). The one exception to this would be if you're using a
> 3.7 lts branch (like instaclustr had at one point - and may still have).
>
>
>
>
> --
>
> Jeff Jirsa
>
>
>
>
> On Aug 1, 2017, at 2:24 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at
> Cisco) <hv...@cisco.com> wrote:
>
> Please help me understand data migration path from 2.2.3 to 3.7.
>
>
>
> 1. Tried to take snapshot and load the snapshot using
> sstableloader; but it is throwing Streaming Exception.
>
> Please advise.
>
>
>
> Thanks,
>
> Harika
>
>
>
>
>
> <image001.png>
>
>
>
> *Harika Vangapelli*
>
> Engineer - IT
>
> hvangape@cisco.com
>
> Tel:
>
> *Cisco Systems, Inc.*
>
>
>
>
> United States
> cisco.com
>
>
>
> <image002.gif>Think before you print.
>
> This email may contain confidential and privileged material for the sole
> use of the intended recipient. Any review, use, distribution or disclosure
> by others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by
> reply email and delete all copies of this message.
>
> Please click here
> <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for
> Company Registration Information.
>
>
>
>
Re: Cassandra Data migration from 2.2.3 to 3.7
Posted by Jon Haddad <jo...@gmail.com>.
Just curious, why go to 3.7? 3.11 has hundreds of bug fixes that 3.7 doesn’t and will continue to receive fixes.
> On Aug 1, 2017, at 3:44 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com> wrote:
>
> Jeff, I tried the below steps for just 3 rows of data, It looks to be working. But Confirm me this is right approach.
> 1. Taking snapshot (using nodetool snapshot) in the existing cluster (2.2 cluster)
> 2. Bring the snapshot to new cluster and copy in keyspce/table (2.2->3.7)
> 3. Nodetool upgrade sstbales on new cluster (3.7)
> 4. Use noedtool refresh -- <keyspace> <table> (3.7)
> Does this approach work?
>
> <image001.png>
>
> Harika Vangapelli
> Engineer - IT
> hvangape@cisco.com <ma...@cisco.com>
> Tel:
> Cisco Systems, Inc.
>
>
>
> United States
> cisco.com <http://cisco.com/>
> <image002.gif>Think before you print.
> This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
> Please click here <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
>
> From: Jeff Jirsa [mailto:jjirsa@gmail.com]
> Sent: Tuesday, August 1, 2017 3:36 PM
> To: user@cassandra.apache.org
> Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
>
> You can't stream across versions, so you either need to use the native protocol (do writes with the normal INSERT operations) or you have to get to the right sstable version
>
> That means one of the three of these:
>
> - export to CSV (copy to/copy from, or Brian Hess' Cassandra-loader on github)
> - export to CSV and use CQLSSTableWriter to generate sstables you can bulk load
> - create a minimal cluster, load the sstables in, and run upgradesstables to rev the version and then stop that cluster and stream them into the new cluster
>
>
> --
> Jeff Jirsa
>
>
> On Aug 1, 2017, at 3:30 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hvangape@cisco.com <ma...@cisco.com>> wrote:
>
> Jeff, Thanks for initial pointer. I want to add some more information to it.
>
> This is to migrate from one cluster (2.2 version) to 3.7 . The client who is in 2.2 cluster they don’t need down time.
>
> So How the data is migrated to new cluster without down time with different versions?
>
> Thanks,
> Harika
>
> <image001.png>
>
> Harika Vangapelli
> Engineer - IT
> hvangape@cisco.com <ma...@cisco.com>
> Tel:
> Cisco Systems, Inc.
>
>
>
> United States
> cisco.com <http://cisco.com/>
>
> <image002.gif>Think before you print.
> This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
> Please click here <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
>
> From: Jeff Jirsa [mailto:jjirsa@gmail.com <ma...@gmail.com>]
> Sent: Tuesday, August 1, 2017 2:48 PM
> To: user@cassandra.apache.org <ma...@cassandra.apache.org>
> Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
>
> If you upgrade in place:
>
> Stop each node one at a time
> Swap binaries
> Start each node
> Run upgradesstables when all nodes are bounced
>
> If you want to take data from 2.2 and put it into a 3.x cluster, you'll need to run upgradesstables on it offline somehow (like clone that 2.2 cluster/schema and run it offline with 3.x upgradesstables), then sstableloader it into place
>
> Note that 3.7 is probably not the version you want - 3.11 is probably much more stable and solves some serious bugs (or 3.0.14 if you don't need features from 3.7 ). The one exception to this would be if you're using a 3.7 lts branch (like instaclustr had at one point - and may still have).
>
>
> --
> Jeff Jirsa
>
>
> On Aug 1, 2017, at 2:24 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hvangape@cisco.com <ma...@cisco.com>> wrote:
>
> Please help me understand data migration path from 2.2.3 to 3.7.
>
> 1. Tried to take snapshot and load the snapshot using sstableloader; but it is throwing Streaming Exception.
> Please advise.
>
> Thanks,
> Harika
>
>
> <image001.png>
>
> Harika Vangapelli
> Engineer - IT
> hvangape@cisco.com <ma...@cisco.com>
> Tel:
> Cisco Systems, Inc.
>
>
>
> United States
> cisco.com <http://cisco.com/>
>
> <image002.gif>Think before you print.
> This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
> Please click here <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
RE: Cassandra Data migration from 2.2.3 to 3.7
Posted by "Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco)" <hv...@cisco.com>.
Jeff, I tried the below steps for just 3 rows of data, It looks to be working. But Confirm me this is right approach.
1. Taking snapshot (using nodetool snapshot) in the existing cluster (2.2 cluster)
2. Bring the snapshot to new cluster and copy in keyspce/table (2.2->3.7)
3. Nodetool upgrade sstbales on new cluster (3.7)
4. Use noedtool refresh -- <keyspace> <table> (3.7)
Does this approach work?
[http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
Harika Vangapelli
Engineer - IT
hvangape@cisco.com<ma...@cisco.com>
Tel:
Cisco Systems, Inc.
United States
cisco.com
[http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
From: Jeff Jirsa [mailto:jjirsa@gmail.com]
Sent: Tuesday, August 1, 2017 3:36 PM
To: user@cassandra.apache.org
Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
You can't stream across versions, so you either need to use the native protocol (do writes with the normal INSERT operations) or you have to get to the right sstable version
That means one of the three of these:
- export to CSV (copy to/copy from, or Brian Hess' Cassandra-loader on github)
- export to CSV and use CQLSSTableWriter to generate sstables you can bulk load
- create a minimal cluster, load the sstables in, and run upgradesstables to rev the version and then stop that cluster and stream them into the new cluster
--
Jeff Jirsa
On Aug 1, 2017, at 3:30 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com>> wrote:
Jeff, Thanks for initial pointer. I want to add some more information to it.
This is to migrate from one cluster (2.2 version) to 3.7 . The client who is in 2.2 cluster they don’t need down time.
So How the data is migrated to new cluster without down time with different versions?
Thanks,
Harika
<image001.png>
Harika Vangapelli
Engineer - IT
hvangape@cisco.com<ma...@cisco.com>
Tel:
Cisco Systems, Inc.
United States
cisco.com<http://cisco.com>
<image002.gif>Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
From: Jeff Jirsa [mailto:jjirsa@gmail.com]
Sent: Tuesday, August 1, 2017 2:48 PM
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
If you upgrade in place:
Stop each node one at a time
Swap binaries
Start each node
Run upgradesstables when all nodes are bounced
If you want to take data from 2.2 and put it into a 3.x cluster, you'll need to run upgradesstables on it offline somehow (like clone that 2.2 cluster/schema and run it offline with 3.x upgradesstables), then sstableloader it into place
Note that 3.7 is probably not the version you want - 3.11 is probably much more stable and solves some serious bugs (or 3.0.14 if you don't need features from 3.7 ). The one exception to this would be if you're using a 3.7 lts branch (like instaclustr had at one point - and may still have).
--
Jeff Jirsa
On Aug 1, 2017, at 2:24 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com>> wrote:
Please help me understand data migration path from 2.2.3 to 3.7.
1. Tried to take snapshot and load the snapshot using sstableloader; but it is throwing Streaming Exception.
Please advise.
Thanks,
Harika
<image001.png>
Harika Vangapelli
Engineer - IT
hvangape@cisco.com<ma...@cisco.com>
Tel:
Cisco Systems, Inc.
United States
cisco.com<http://cisco.com>
<image002.gif>Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
Re: Cassandra Data migration from 2.2.3 to 3.7
Posted by Jeff Jirsa <jj...@gmail.com>.
You can't stream across versions, so you either need to use the native protocol (do writes with the normal INSERT operations) or you have to get to the right sstable version
That means one of the three of these:
- export to CSV (copy to/copy from, or Brian Hess' Cassandra-loader on github)
- export to CSV and use CQLSSTableWriter to generate sstables you can bulk load
- create a minimal cluster, load the sstables in, and run upgradesstables to rev the version and then stop that cluster and stream them into the new cluster
--
Jeff Jirsa
> On Aug 1, 2017, at 3:30 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com> wrote:
>
> Jeff, Thanks for initial pointer. I want to add some more information to it.
>
> This is to migrate from one cluster (2.2 version) to 3.7 . The client who is in 2.2 cluster they don’t need down time.
>
> So How the data is migrated to new cluster without down time with different versions?
>
> Thanks,
> Harika
>
> <image001.png>
>
> Harika Vangapelli
> Engineer - IT
> hvangape@cisco.com
> Tel:
> Cisco Systems, Inc.
>
>
>
> United States
> cisco.com
> <image002.gif>Think before you print.
> This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
> Please click here for Company Registration Information.
>
> From: Jeff Jirsa [mailto:jjirsa@gmail.com]
> Sent: Tuesday, August 1, 2017 2:48 PM
> To: user@cassandra.apache.org
> Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
>
> If you upgrade in place:
>
> Stop each node one at a time
> Swap binaries
> Start each node
> Run upgradesstables when all nodes are bounced
>
> If you want to take data from 2.2 and put it into a 3.x cluster, you'll need to run upgradesstables on it offline somehow (like clone that 2.2 cluster/schema and run it offline with 3.x upgradesstables), then sstableloader it into place
>
> Note that 3.7 is probably not the version you want - 3.11 is probably much more stable and solves some serious bugs (or 3.0.14 if you don't need features from 3.7 ). The one exception to this would be if you're using a 3.7 lts branch (like instaclustr had at one point - and may still have).
>
>
> --
> Jeff Jirsa
>
>
> On Aug 1, 2017, at 2:24 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com> wrote:
>
> Please help me understand data migration path from 2.2.3 to 3.7.
>
> 1. Tried to take snapshot and load the snapshot using sstableloader; but it is throwing Streaming Exception.
> Please advise.
>
> Thanks,
> Harika
>
>
> <image001.png>
>
> Harika Vangapelli
> Engineer - IT
> hvangape@cisco.com
> Tel:
> Cisco Systems, Inc.
>
>
>
> United States
> cisco.com
>
> <image002.gif>Think before you print.
> This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
> Please click here for Company Registration Information.
>
RE: Cassandra Data migration from 2.2.3 to 3.7
Posted by "Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco)" <hv...@cisco.com>.
Jeff, Thanks for initial pointer. I want to add some more information to it.
This is to migrate from one cluster (2.2 version) to 3.7 . The client who is in 2.2 cluster they don’t need down time.
So How the data is migrated to new cluster without down time with different versions?
Thanks,
Harika
[http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
Harika Vangapelli
Engineer - IT
hvangape@cisco.com<ma...@cisco.com>
Tel:
Cisco Systems, Inc.
United States
cisco.com
[http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
From: Jeff Jirsa [mailto:jjirsa@gmail.com]
Sent: Tuesday, August 1, 2017 2:48 PM
To: user@cassandra.apache.org
Subject: Re: Cassandra Data migration from 2.2.3 to 3.7
If you upgrade in place:
Stop each node one at a time
Swap binaries
Start each node
Run upgradesstables when all nodes are bounced
If you want to take data from 2.2 and put it into a 3.x cluster, you'll need to run upgradesstables on it offline somehow (like clone that 2.2 cluster/schema and run it offline with 3.x upgradesstables), then sstableloader it into place
Note that 3.7 is probably not the version you want - 3.11 is probably much more stable and solves some serious bugs (or 3.0.14 if you don't need features from 3.7 ). The one exception to this would be if you're using a 3.7 lts branch (like instaclustr had at one point - and may still have).
--
Jeff Jirsa
On Aug 1, 2017, at 2:24 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com>> wrote:
Please help me understand data migration path from 2.2.3 to 3.7.
1. Tried to take snapshot and load the snapshot using sstableloader; but it is throwing Streaming Exception.
Please advise.
Thanks,
Harika
<image001.png>
Harika Vangapelli
Engineer - IT
hvangape@cisco.com<ma...@cisco.com>
Tel:
Cisco Systems, Inc.
United States
cisco.com<http://cisco.com>
<image002.gif>Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.
Re: Cassandra Data migration from 2.2.3 to 3.7
Posted by Jeff Jirsa <jj...@gmail.com>.
If you upgrade in place:
Stop each node one at a time
Swap binaries
Start each node
Run upgradesstables when all nodes are bounced
If you want to take data from 2.2 and put it into a 3.x cluster, you'll need to run upgradesstables on it offline somehow (like clone that 2.2 cluster/schema and run it offline with 3.x upgradesstables), then sstableloader it into place
Note that 3.7 is probably not the version you want - 3.11 is probably much more stable and solves some serious bugs (or 3.0.14 if you don't need features from 3.7 ). The one exception to this would be if you're using a 3.7 lts branch (like instaclustr had at one point - and may still have).
--
Jeff Jirsa
> On Aug 1, 2017, at 2:24 PM, Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco) <hv...@cisco.com> wrote:
>
> Please help me understand data migration path from 2.2.3 to 3.7.
>
> 1. Tried to take snapshot and load the snapshot using sstableloader; but it is throwing Streaming Exception.
> Please advise.
>
> Thanks,
> Harika
>
>
> <image001.png>
>
> Harika Vangapelli
> Engineer - IT
> hvangape@cisco.com
> Tel:
> Cisco Systems, Inc.
>
>
>
> United States
> cisco.com
> <image002.gif>Think before you print.
> This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
> Please click here for Company Registration Information.
>