Migrating a column family from one cluster to another

[Bryan Fernandez <bf...@gmail.com>]

Greetings,

We presently have a single six node cluster, but are looking to distribute
our data model across multiple clusters. What would be the recommended
approach to migrating a few column families from a six node cluster to a
three node cluster? I am under the impression that a snapshot of a column
family on a single node would potentially be only 1/6 of the data present
in the column family across the cluster. I guess what we are looking for is
something like a mysql table dump that would could import into a new
cluster with an arbitrary node count.  What's the best practice for this
type of a data migration?

Thanks,

Bryan

Re: Migrating a column family from one cluster to another

[Rob Coli <rc...@palominodb.com>]

On Fri, May 18, 2012 at 1:41 PM, Poziombka, Wade L
<wa...@intel.com> wrote:
> How does counters affect this?  Why would be different?

Oh, actually this is an obsolete caution as of Cassandra 0.8beta1 :

https://issues.apache.org/jira/browse/CASSANDRA-1938

Sorry! :)

=Rob
PS - for historical reference, before this ticket the counts were
based on the ip address of the nodes and things would be hosed if you
did the copy-all-the-sstables operations. it is easy for me to forget
that almost no one was using cassandra counters before 0.8, heh.

-- 
=Robert Coli
AIM&GTALK - rcoli@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

Re: Migrating a column family from one cluster to another

["Poziombka, Wade L" <wa...@intel.com>]

How does counters affect this?  Why would be different?  

Sent from my iPhone

On May 18, 2012, at 15:40, "Rob Coli" <rc...@palominodb.com> wrote:

> On Thu, May 17, 2012 at 9:37 AM, Bryan Fernandez <bf...@gmail.com> wrote:
>> What would be the recommended
>> approach to migrating a few column families from a six node cluster to a
>> three node cluster?
> 
> The easiest way (if you are not using counters) is :
> 
> 1) make sure all filenames of sstables are unique [1]
> 2) copy all sstablefiles from the 6 nodes to all 3 nodes
> 3) run a "cleanup" compaction on the 3 nodes
> 
> =Rob
> [1] https://issues.apache.org/jira/browse/CASSANDRA-1983
> 
> -- 
> =Robert Coli
> AIM&GTALK - rcoli@palominodb.com
> YAHOO - rcoli.palominob
> SKYPE - rcoli_palominodb

Re: Migrating a column family from one cluster to another

[Jonathan Ellis <jb...@gmail.com>]

Better: use bin/sstableloader, which will copy exactly the right
ranges of data to the new cluster.

On Fri, May 18, 2012 at 3:39 PM, Rob Coli <rc...@palominodb.com> wrote:
> On Thu, May 17, 2012 at 9:37 AM, Bryan Fernandez <bf...@gmail.com> wrote:
>> What would be the recommended
>> approach to migrating a few column families from a six node cluster to a
>> three node cluster?
>
> The easiest way (if you are not using counters) is :
>
> 1) make sure all filenames of sstables are unique [1]
> 2) copy all sstablefiles from the 6 nodes to all 3 nodes
> 3) run a "cleanup" compaction on the 3 nodes
>
> =Rob
> [1] https://issues.apache.org/jira/browse/CASSANDRA-1983
>
> --
> =Robert Coli
> AIM&GTALK - rcoli@palominodb.com
> YAHOO - rcoli.palominob
> SKYPE - rcoli_palominodb



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Migrating a column family from one cluster to another

[Rob Coli <rc...@palominodb.com>]

On Thu, May 17, 2012 at 9:37 AM, Bryan Fernandez <bf...@gmail.com> wrote:
> What would be the recommended
> approach to migrating a few column families from a six node cluster to a
> three node cluster?

The easiest way (if you are not using counters) is :

1) make sure all filenames of sstables are unique [1]
2) copy all sstablefiles from the 6 nodes to all 3 nodes
3) run a "cleanup" compaction on the 3 nodes

=Rob
[1] https://issues.apache.org/jira/browse/CASSANDRA-1983

-- 
=Robert Coli
AIM&GTALK - rcoli@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb