You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Lanny Ripple <la...@spotright.com> on 2013/07/23 21:32:46 UTC
Decommission an entire DC
Hi,
We have a multi-dc setup using DC1:2, DC2:2. We want to get rid of DC1.
We're in the position where we don't need to save any of the data on DC1.
We know we'll lose a (tiny. already checked) bit of data but our
processing is such that we'll recover over time.
How do we drop DC1 and just move forward with DC2? Using nodetool
decommision or removetoken looks like we'll eventually end up with a single
DC1 node containing the entire dc's data which would be slow and costly.
We've speculated that setting DC1:0 or removing it from the schema would do
the trick but without finding any hits during searching on that idea I
hesitate to just do it. We can drop DC1s data but have to keep a working
ring in DC2.
Re: Decommission an entire DC
Posted by Robert Coli <rc...@eventbrite.com>.
On Wed, Jul 24, 2013 at 12:26 PM, Lanny Ripple <la...@spotright.com> wrote:
> That one is documented --
> http://www.datastax.com/documentation/cassandra/1.2/index.html#cassandra/operations/ops_add_dc_to_cluster_t.html
>
That doc is incomplete, IMO. It (and the doc for adding a node) should
contain an instruction to run repair. Rebuild, like bootstrap, streams from
only one replica. Due to CASSANDRA-2434 [1], if you do not repair after
rebuild you may be missing data in your new DC when read at QUORUM_LOCAL. I
have bcc:ed docs@datastax for their information. :)
=Rob
[1] https://issues.apache.org/jira/browse/CASSANDRA-2434
Re: Decommission an entire DC
Posted by Lanny Ripple <la...@spotright.com>.
That one is documented --
http://www.datastax.com/documentation/cassandra/1.2/index.html#cassandra/operations/ops_add_dc_to_cluster_t.html
On Wed, Jul 24, 2013 at 3:33 AM, Cyril Scetbon <cy...@free.fr>wrote:
> And if we want to add a new DC ? I suppose we should add all nodes and
> alter the replication factor of the keyspace after that, but if anyone can
> confirm it and maybe give me some tips ?
> FYI ,we have 2 DCs with between 10 and 20 nodes in each and a 2To database
> (local replication factor included)
>
> thanks
> --
> Cyril SCETBON
>
> On Jul 24, 2013, at 12:04 AM, Omar Shibli <om...@eyeviewdigital.com> wrote:
>
> All you need to do is to decrease the replication factor of DC1 to 0, and
> then decommission the nodes one by one,
> I've tried this before and it worked with no issues.
>
> Thanks,
>
> On Tue, Jul 23, 2013 at 10:32 PM, Lanny Ripple <la...@spotright.com>wrote:
>
>> Hi,
>>
>> We have a multi-dc setup using DC1:2, DC2:2. We want to get rid of DC1.
>> We're in the position where we don't need to save any of the data on DC1.
>> We know we'll lose a (tiny. already checked) bit of data but our
>> processing is such that we'll recover over time.
>>
>> How do we drop DC1 and just move forward with DC2? Using nodetool
>> decommision or removetoken looks like we'll eventually end up with a single
>> DC1 node containing the entire dc's data which would be slow and costly.
>>
>> We've speculated that setting DC1:0 or removing it from the schema would
>> do the trick but without finding any hits during searching on that idea I
>> hesitate to just do it. We can drop DC1s data but have to keep a working
>> ring in DC2.
>>
>>
>
>
Re: Decommission an entire DC
Posted by Cyril Scetbon <cy...@free.fr>.
And if we want to add a new DC ? I suppose we should add all nodes and alter the replication factor of the keyspace after that, but if anyone can confirm it and maybe give me some tips ?
FYI ,we have 2 DCs with between 10 and 20 nodes in each and a 2To database (local replication factor included)
thanks
--
Cyril SCETBON
On Jul 24, 2013, at 12:04 AM, Omar Shibli <om...@eyeviewdigital.com> wrote:
> All you need to do is to decrease the replication factor of DC1 to 0, and then decommission the nodes one by one,
> I've tried this before and it worked with no issues.
>
> Thanks,
>
> On Tue, Jul 23, 2013 at 10:32 PM, Lanny Ripple <la...@spotright.com> wrote:
> Hi,
>
> We have a multi-dc setup using DC1:2, DC2:2. We want to get rid of DC1. We're in the position where we don't need to save any of the data on DC1. We know we'll lose a (tiny. already checked) bit of data but our processing is such that we'll recover over time.
>
> How do we drop DC1 and just move forward with DC2? Using nodetool decommision or removetoken looks like we'll eventually end up with a single DC1 node containing the entire dc's data which would be slow and costly.
>
> We've speculated that setting DC1:0 or removing it from the schema would do the trick but without finding any hits during searching on that idea I hesitate to just do it. We can drop DC1s data but have to keep a working ring in DC2.
>
>
Re: Decommission an entire DC
Posted by Omar Shibli <om...@eyeviewdigital.com>.
All you need to do is to decrease the replication factor of DC1 to 0, and
then decommission the nodes one by one,
I've tried this before and it worked with no issues.
Thanks,
On Tue, Jul 23, 2013 at 10:32 PM, Lanny Ripple <la...@spotright.com> wrote:
> Hi,
>
> We have a multi-dc setup using DC1:2, DC2:2. We want to get rid of DC1.
> We're in the position where we don't need to save any of the data on DC1.
> We know we'll lose a (tiny. already checked) bit of data but our
> processing is such that we'll recover over time.
>
> How do we drop DC1 and just move forward with DC2? Using nodetool
> decommision or removetoken looks like we'll eventually end up with a single
> DC1 node containing the entire dc's data which would be slow and costly.
>
> We've speculated that setting DC1:0 or removing it from the schema would
> do the trick but without finding any hits during searching on that idea I
> hesitate to just do it. We can drop DC1s data but have to keep a working
> ring in DC2.
>
>