Best strategy to increase cluster size and keep nodes balanced

[Filippo Diotalevi <fi...@ntoklo.com>]

What's the best strategy to increase the cluster size from 3 nodes to 7 nodes, while keeping the nodes balanced? 
The datastax documentation at http://www.datastax.com/docs/1.0/operations/cluster_management seems to suggest that it's best practice to add one node at a time. Is it the only approach?

Can we add 4 new nodes, reset the tokens, and expect the cluster to rebalance correctly?  

Thanks,
-- 
Filippo Diotalevi

Re: Best strategy to increase cluster size and keep nodes balanced

[aaron morton <aa...@thelastpickle.com>]

Unless you really need to consider moving to 6, it will be easier. 

That said, if you want to get to 7 I would:

* bring the new nodes in with tokens selected for 7.
* move the old nodes to new 7-node tokens. 
* cleanup on the old nodes

There is a way to expedite things by copying files around, _but_ if this is the first time you have grown the ring I would do it the normal documented way. 

I would make the changes one node at a time. There are a couple of reasons, but here are two: 

* if something goes wrong the impact will be less and the cleanup will be less. IMHO With a cluster of machines it's a good idea to make changes one at a time, so that if / when things go wrong you get the benefit of the other machines working. 
* bootstrapping new nodes will have some performance impact on the existing nodes. With fewer existing nodes it will have a larger impact. 

Hope that helps.

-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 21/08/2012, at 12:57 AM, Filippo Diotalevi <fi...@ntoklo.com> wrote:

> What's the best strategy to increase the cluster size from 3 nodes to 7 nodes, while keeping the nodes balanced? 
> The datastax documentation at http://www.datastax.com/docs/1.0/operations/cluster_management seems to suggest that it's best practice to add one node at a time. Is it the only approach?
> 
> Can we add 4 new nodes, reset the tokens, and expect the cluster to rebalance correctly?  
> 
> Thanks,
> -- 
> Filippo Diotalevi
> 
>