You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Ioannis Zafiropoulos <jo...@gmail.com> on 2017/08/15 20:33:31 UTC
Migrate from DSE (Datastax) to Apache Cassandra
Hi all,
We have setup a new cluster DSE 5.1.2 (with Cassandra 3.11.0.1758) and we
want to migrate it to Apache Cassandra 3.11.0 without loosing schema or
data.
Anybody, has done it before?
Obviously we are going to test this, but it would be nice to hear if
somebody else has gone through with the procedure.
Thank you!
Re: Migrate from DSE (Datastax) to Apache Cassandra
Posted by kurt greaves <ku...@instaclustr.com>.
Haven't done it for 5.1 but went smoothly for earlier versions. If you're
not using any of the additional features of DSE, it should be OK. Just
change any custom replication strategies before migrating and also make
sure your yaml options are compatible.
Re: Migrate from DSE (Datastax) to Apache Cassandra
Posted by Ioannis Zafiropoulos <jo...@gmail.com>.
Ok found a solution for this problem.
I deleted the system's keyspace directory and restarted COSS and it was
rebuilt.
rm -rf /var/lib/cassandra/data/system
A bit drastic but I'll test it also on a multi-node cluster.
On Thu, Aug 17, 2017 at 3:57 PM, Ioannis Zafiropoulos <jo...@gmail.com>
wrote:
> Thanks Felipe and Erick,
>
> Yes, your comment helped a lot, I was able to resolve that by:
> ALTER KEYSPACE dse_system WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor':'1'};
>
> Another problem I had was with CentOS release 6.7 (Final)
> I was getting glibc 2.14 not found.
> Based on this <https://issues.apache.org/jira/browse/CASSANDRA-13072> I
> switched jna-4.4.0.jar with jna-4.2.2.jar and it worked.
>
> I just started COSS for the first time successfully, I am able to connect
> and work on the DB.
> It would be a perfect success if it was not for an exception that bugs me
> every time I start Cassandra:
>
> DEBUG [SSTableBatchOpen:1] 2017-08-17 14:36:50,477 SSTableReader.java:506
> - Opening /cassandra/disk01/system/local-7ad54392bcdd35a684174e047860b377/mc-217-big
> (0.598KiB)
> DEBUG [SSTableBatchOpen:2] 2017-08-17 14:36:50,477 SSTableReader.java:506
> - Opening /cassandra/disk01/system/local-7ad54392bcdd35a684174e047860b377/mc-155-big
> (0.139KiB)
> ERROR [SSTableBatchOpen:2] 2017-08-17 14:36:50,489
> DebuggableThreadPoolExecutor.java:239 - Error in ThreadPoolExecutor
> java.lang.RuntimeException: Unknown column server_id during deserialization
> at org.apache.cassandra.db.SerializationHeader$Component.
> toHeader(SerializationHeader.java:309) ~[apache-cassandra-3.11.0.jar:
> 3.11.0]
> at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:513)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
> ~[apache-cassandra-3.11.0.jar:3.11.0]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_65]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_65]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_65]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_65]
> at org.apache.cassandra.concurrent.NamedThreadFactory.
> lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.0.jar:3.11.0]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_65]
>
> I looked at another DSE installation and system.local table has indeed a
> 'server_id' column.
> On my COSS testbed this column disappeared from the table as soon as I
> started for the first time COSS.
> I tried to sstablescrub, sstableupgrade but it didn't go away.
>
> I don't know if I should worry or how to fix it, any ideas?
>
>
>
> On Wed, Aug 16, 2017 at 1:38 PM, Felipe Esteves <
> felipe.esteves@b2wdigital.com> wrote:
>
>> Ioannis,
>> As some people already said, there's one or two keyspaces that uses
>> EverywhereStrategy, dse_system is one of them, if I'm not wrong.
>> You must remember to change them to a community strategy or it will fail.
>>
>>
>>
>>
>>
>
Re: Migrate from DSE (Datastax) to Apache Cassandra
Posted by Ioannis Zafiropoulos <jo...@gmail.com>.
Thanks Felipe and Erick,
Yes, your comment helped a lot, I was able to resolve that by:
ALTER KEYSPACE dse_system WITH replication = {'class': 'SimpleStrategy',
'replication_factor':'1'};
Another problem I had was with CentOS release 6.7 (Final)
I was getting glibc 2.14 not found.
Based on this <https://issues.apache.org/jira/browse/CASSANDRA-13072> I
switched jna-4.4.0.jar with jna-4.2.2.jar and it worked.
I just started COSS for the first time successfully, I am able to connect
and work on the DB.
It would be a perfect success if it was not for an exception that bugs me
every time I start Cassandra:
DEBUG [SSTableBatchOpen:1] 2017-08-17 14:36:50,477 SSTableReader.java:506 -
Opening
/cassandra/disk01/system/local-7ad54392bcdd35a684174e047860b377/mc-217-big
(0.598KiB)
DEBUG [SSTableBatchOpen:2] 2017-08-17 14:36:50,477 SSTableReader.java:506 -
Opening
/cassandra/disk01/system/local-7ad54392bcdd35a684174e047860b377/mc-155-big
(0.139KiB)
ERROR [SSTableBatchOpen:2] 2017-08-17 14:36:50,489
DebuggableThreadPoolExecutor.java:239 - Error in ThreadPoolExecutor
java.lang.RuntimeException: Unknown column server_id during deserialization
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:309)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:513)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:396)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
org.apache.cassandra.io.sstable.format.SSTableReader$5.run(SSTableReader.java:561)
~[apache-cassandra-3.11.0.jar:3.11.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_65]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_65]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_65]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_65]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
[apache-cassandra-3.11.0.jar:3.11.0]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_65]
I looked at another DSE installation and system.local table has indeed a
'server_id' column.
On my COSS testbed this column disappeared from the table as soon as I
started for the first time COSS.
I tried to sstablescrub, sstableupgrade but it didn't go away.
I don't know if I should worry or how to fix it, any ideas?
On Wed, Aug 16, 2017 at 1:38 PM, Felipe Esteves <
felipe.esteves@b2wdigital.com> wrote:
> Ioannis,
> As some people already said, there's one or two keyspaces that uses
> EverywhereStrategy, dse_system is one of them, if I'm not wrong.
> You must remember to change them to a community strategy or it will fail.
>
>
>
>
>
Re: Migrate from DSE (Datastax) to Apache Cassandra
Posted by Felipe Esteves <fe...@b2wdigital.com>.
Ioannis,
As some people already said, there's one or two keyspaces that uses
EverywhereStrategy, dse_system is one of them, if I'm not wrong.
You must remember to change them to a community strategy or it will fail.
--
Re: Migrate from DSE (Datastax) to Apache Cassandra
Posted by Ioannis Zafiropoulos <jo...@gmail.com>.
We use NetworkTopologyStrategy as the replication strategy.
The only DSE specific features we use (left untouched by default) are:
authenticator: com.datastax.bdp.cassandra.auth.DseAuthenticator
authorizer: com.datastax.bdp.cassandra.auth.DseAuthorizer
role_manager: com.datastax.bdp.cassandra.auth.DseRoleManager
So I hope by changing these to the COSS recommended ones, before the
migration, DSE will be able to switch to them on its own (?)
and then do the final transition of tarball installation.
Thank you all for your answers!
On Tue, Aug 15, 2017 at 10:42 PM, Jon Haddad <jo...@gmail.com>
wrote:
> I agree with Jeff, it’s not necessary to launch a new cluster for this
> operation.
>
> On Aug 15, 2017, at 7:39 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>
> Or just alter the key space replication strategy and remove the DSE
> specific strategies in favor of network topology strategy
>
>
> --
> Jeff Jirsa
>
>
> On Aug 15, 2017, at 7:26 PM, Erick Ramirez <fl...@gmail.com> wrote:
>
> Ioannis, it's not a straightforward process to migrate from DSE to COSS.
> There are some parts of DSE which are not recognised by COSS, e.g.
> EverywhereStrategy for replication only known to DSE.
>
> You are better off standing up a new COSS 3.11 cluster and restore app
> keyspaces to the new cluster. Cheers!
>
> On Wed, Aug 16, 2017 at 6:33 AM, Ioannis Zafiropoulos <jo...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> We have setup a new cluster DSE 5.1.2 (with Cassandra 3.11.0.1758) and we
>> want to migrate it to Apache Cassandra 3.11.0 without loosing schema or
>> data.
>>
>> Anybody, has done it before?
>>
>> Obviously we are going to test this, but it would be nice to hear if
>> somebody else has gone through with the procedure.
>>
>> Thank you!
>>
>
>
>
Re: Migrate from DSE (Datastax) to Apache Cassandra
Posted by Jon Haddad <jo...@gmail.com>.
I agree with Jeff, it’s not necessary to launch a new cluster for this operation.
> On Aug 15, 2017, at 7:39 PM, Jeff Jirsa <jj...@gmail.com> wrote:
>
> Or just alter the key space replication strategy and remove the DSE specific strategies in favor of network topology strategy
>
>
> --
> Jeff Jirsa
>
>
> On Aug 15, 2017, at 7:26 PM, Erick Ramirez <flightctlr@gmail.com <ma...@gmail.com>> wrote:
>
>> Ioannis, it's not a straightforward process to migrate from DSE to COSS. There are some parts of DSE which are not recognised by COSS, e.g. EverywhereStrategy for replication only known to DSE.
>>
>> You are better off standing up a new COSS 3.11 cluster and restore app keyspaces to the new cluster. Cheers!
>>
>> On Wed, Aug 16, 2017 at 6:33 AM, Ioannis Zafiropoulos <johnzaf@gmail.com <ma...@gmail.com>> wrote:
>> Hi all,
>>
>> We have setup a new cluster DSE 5.1.2 (with Cassandra 3.11.0.1758) and we want to migrate it to Apache Cassandra 3.11.0 without loosing schema or data.
>>
>> Anybody, has done it before?
>>
>> Obviously we are going to test this, but it would be nice to hear if somebody else has gone through with the procedure.
>>
>> Thank you!
>>
Re: Migrate from DSE (Datastax) to Apache Cassandra
Posted by Jeff Jirsa <jj...@gmail.com>.
Or just alter the key space replication strategy and remove the DSE specific strategies in favor of network topology strategy
--
Jeff Jirsa
> On Aug 15, 2017, at 7:26 PM, Erick Ramirez <fl...@gmail.com> wrote:
>
> Ioannis, it's not a straightforward process to migrate from DSE to COSS. There are some parts of DSE which are not recognised by COSS, e.g. EverywhereStrategy for replication only known to DSE.
>
> You are better off standing up a new COSS 3.11 cluster and restore app keyspaces to the new cluster. Cheers!
>
>> On Wed, Aug 16, 2017 at 6:33 AM, Ioannis Zafiropoulos <jo...@gmail.com> wrote:
>> Hi all,
>>
>> We have setup a new cluster DSE 5.1.2 (with Cassandra 3.11.0.1758) and we want to migrate it to Apache Cassandra 3.11.0 without loosing schema or data.
>>
>> Anybody, has done it before?
>>
>> Obviously we are going to test this, but it would be nice to hear if somebody else has gone through with the procedure.
>>
>> Thank you!
>
Re: Migrate from DSE (Datastax) to Apache Cassandra
Posted by Erick Ramirez <fl...@gmail.com>.
Ioannis, it's not a straightforward process to migrate from DSE to COSS.
There are some parts of DSE which are not recognised by COSS, e.g.
EverywhereStrategy for replication only known to DSE.
You are better off standing up a new COSS 3.11 cluster and restore app
keyspaces to the new cluster. Cheers!
On Wed, Aug 16, 2017 at 6:33 AM, Ioannis Zafiropoulos <jo...@gmail.com>
wrote:
> Hi all,
>
> We have setup a new cluster DSE 5.1.2 (with Cassandra 3.11.0.1758) and we
> want to migrate it to Apache Cassandra 3.11.0 without loosing schema or
> data.
>
> Anybody, has done it before?
>
> Obviously we are going to test this, but it would be nice to hear if
> somebody else has gone through with the procedure.
>
> Thank you!
>