Combining two clusters/keyspaces into single cluster

[Arlington Albertson <ar...@gmail.com>]

Hey Folks,

I've been looking through various documentations, but I'm either
overlooking something obvious or not wording it correctly, but the gist of
my problem is this:

I have two cassandra clusters, with two separate keyspaces on EC2. We'll
call them as follows:

*cluster1* (DC name, cluster name, etc...)
*keyspace1* (only exists on cluster1)

*cluster2* (DC name, cluster name, etc...)
*keyspace2* (only exists on cluster2)

I need to perform the following:
- take keyspace2, and add it to cluster1 so that all nodes can serve the
traffic
- needs to happen "live" so that I can repoint new instances to the
cluster1 endpoints and they'll just start working, and no longer directly
use cluster2
- eventually, tear down cluster2 (easy with a `nodetool decommission` after
verifying all seeds have been changed, etc...)

This doc seems to be the closest I've found thus far:
https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_add_dc_to_cluster_t.html

Is that the appropriate guide for this and I'm just over thinking it? Or is
there something else I should be looking at?

Also, this is DSC C* 2.1.13.

TIA!

-AA

Re: Combining two clusters/keyspaces into single cluster

[Jan Kesten <j....@enercast.de>]

Hi,

one way I think might work (but not tested in any way by me and there 
will be some lag / stale data):

- create the keyspace2 von cluster1
- use nodetool flush and snapshot on cluster2, remember the timestamp
- use sstableloader to write all sstables from cluster2 snapshot to cluster1
- you can repeat last two steps and use sstableload only on tables with 
mtime > timestamp to add the differencens to cluster1
- shutdown cluster2 when done

Of course, data written by old clients to cluster2 wont be available in 
cluster1 until loading that data into it.

Just my 2 cents :)

Jan


Am 22.04.2016 um 01:15 schrieb Arlington Albertson:
> Hey Folks,
>
> I've been looking through various documentations, but I'm either 
> overlooking something obvious or not wording it correctly, but the 
> gist of my problem is this:
>
> I have two cassandra clusters, with two separate keyspaces on EC2. 
> We'll call them as follows:
>
> *cluster1* (DC name, cluster name, etc...)
> *keyspace1* (only exists on cluster1)
>
> *cluster2* (DC name, cluster name, etc...)
> *keyspace2*(only exists on cluster2)
>
> I need to perform the following:
> - take keyspace2, and add it to cluster1 so that all nodes can serve 
> the traffic
> - needs to happen "live" so that I can repoint new instances to the 
> cluster1 endpoints and they'll just start working, and no longer 
> directly use cluster2
> - eventually, tear down cluster2 (easy with a `nodetool decommission` 
> after verifying all seeds have been changed, etc...)
>
> This doc seems to be the closest I've found thus far:
> https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_add_dc_to_cluster_t.html
>
> Is that the appropriate guide for this and I'm just over thinking it? 
> Or is there something else I should be looking at?
>
> Also, this is DSC C* 2.1.13.
>
> TIA!
>
> -AA

Re: Combining two clusters/keyspaces into single cluster

[Jens Rantil <je...@tink.se>]

Hi Arlington,

I don't think that's possible to do live simply within the realm of
Cassandra since the nodes are in separate Cassandra clusters. What you
could do is do double writing from your application while doing Cassandra
Bulk Loading described here: http://stackoverflow.com/a/12528029/260805.

Cheers,
Jens

On Fri, Apr 22, 2016 at 1:15 AM Arlington Albertson <
arlingtonalbertson@gmail.com> wrote:

> Hey Folks,
>
> I've been looking through various documentations, but I'm either
> overlooking something obvious or not wording it correctly, but the gist of
> my problem is this:
>
> I have two cassandra clusters, with two separate keyspaces on EC2. We'll
> call them as follows:
>
> *cluster1* (DC name, cluster name, etc...)
> *keyspace1* (only exists on cluster1)
>
> *cluster2* (DC name, cluster name, etc...)
> *keyspace2* (only exists on cluster2)
>
> I need to perform the following:
> - take keyspace2, and add it to cluster1 so that all nodes can serve the
> traffic
> - needs to happen "live" so that I can repoint new instances to the
> cluster1 endpoints and they'll just start working, and no longer directly
> use cluster2
> - eventually, tear down cluster2 (easy with a `nodetool decommission`
> after verifying all seeds have been changed, etc...)
>
> This doc seems to be the closest I've found thus far:
>
> https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_add_dc_to_cluster_t.html
>
> Is that the appropriate guide for this and I'm just over thinking it? Or
> is there something else I should be looking at?
>
> Also, this is DSC C* 2.1.13.
>
> TIA!
>
> -AA
>
-- 

Jens Rantil
Backend Developer @ Tink

Tink AB, Wallingatan 5, 111 60 Stockholm, Sweden
For urgent matters you can reach me at +46-708-84 18 32.