You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by amulya rattan <ta...@gmail.com> on 2011/09/04 11:58:40 UTC
load balance issue
Hi there,
I had a 3 nodes ring, added a 4th one, and moved others to appropriate
tokens..doing nodetool ring shows:
127.0.0.1 datacenter1 rack1 Up Normal 348.82 MB
25.00% 0
127.0.0.2 datacenter1 rack1 Up Normal 349.81 MB
25.00% 42535295865117307932921825928971026432
127.0.0.3 datacenter1 rack1 Up Normal 437.04 MB
25.00% 85070591730234615865843651857942052864
127.0.0.4 datacenter1 rack1 Up Normal 261.74 MB
25.00% 127605887595351923798765477786913079296
Although it shows 25% on all nodes, the data distribution seems totally
unbalanced, how did that happen?
Also if I happen to have a thousand node cluster, and add another 100 nodes,
is it ok to start moving all nodes to their respective new tokens
simultaneously or one-by-one?
Any response is appreciated.
~Amulya
Re: load balance issue
Posted by amulya rattan <ta...@gmail.com>.
This is golden! thanks a heap guys
On Mon, Sep 5, 2011 at 6:07 PM, Nick Bailey <ni...@datastax.com> wrote:
> You can place each of the 4 new nodes exactly in the middle of 2 of
> the current nodes. This way each node will still be responsible for
> the same amount of data but your old nodes did not move.
>
> On Mon, Sep 5, 2011 at 2:56 PM, amulya rattan <ta...@gmail.com>
> wrote:
> > Ah, missed that. Thanks for the pointer.
> > While we are at it, the doc says that if I am doubling the strength of my
> > cluster, and I assign calculated tokens to the new nodes, i don't need to
> do
> > the nodetool move for old nodes. Won't I have to assign the old nodes
> with
> > their new respective tokens too? If I had 4 nodes and I add 4 more, the
> > token distribution for each node changes. Then how could I get away with
> not
> > assigning old nodes with new tokens?
> > ~Amulya
> >
> > On Mon, Sep 5, 2011 at 4:12 AM, Sylvain Lebresne <sy...@datastax.com>
> > wrote:
> >>
> >> Have you done step 6 of the 'To add nodes to a Cassandra cluster' of
> >> http://www.datastax.com/docs/0.8/operations/clustering#adding-capacity,
> >> aka, run nodetool cleanup on the previously existing nodes ?
> >>
> >> --
> >> Sylvain
> >>
> >> On Sun, Sep 4, 2011 at 11:58 AM, amulya rattan <ta...@gmail.com>
> >> wrote:
> >> > Hi there,
> >> > I had a 3 nodes ring, added a 4th one, and moved others to
> appropriate
> >> > tokens..doing nodetool ring shows:
> >> > 127.0.0.1 datacenter1 rack1 Up Normal 348.82 MB
> >> > 25.00% 0
> >> > 127.0.0.2 datacenter1 rack1 Up Normal 349.81 MB
> >> > 25.00% 42535295865117307932921825928971026432
> >> > 127.0.0.3 datacenter1 rack1 Up Normal 437.04 MB
> >> > 25.00% 85070591730234615865843651857942052864
> >> > 127.0.0.4 datacenter1 rack1 Up Normal 261.74 MB
> >> > 25.00% 127605887595351923798765477786913079296
> >> > Although it shows 25% on all nodes, the data distribution seems
> totally
> >> > unbalanced, how did that happen?
> >> > Also if I happen to have a thousand node cluster, and add another 100
> >> > nodes,
> >> > is it ok to start moving all nodes to their respective new tokens
> >> > simultaneously or one-by-one?
> >> > Any response is appreciated.
> >> > ~Amulya
> >
> >
>
Re: load balance issue
Posted by Nick Bailey <ni...@datastax.com>.
You can place each of the 4 new nodes exactly in the middle of 2 of
the current nodes. This way each node will still be responsible for
the same amount of data but your old nodes did not move.
On Mon, Sep 5, 2011 at 2:56 PM, amulya rattan <ta...@gmail.com> wrote:
> Ah, missed that. Thanks for the pointer.
> While we are at it, the doc says that if I am doubling the strength of my
> cluster, and I assign calculated tokens to the new nodes, i don't need to do
> the nodetool move for old nodes. Won't I have to assign the old nodes with
> their new respective tokens too? If I had 4 nodes and I add 4 more, the
> token distribution for each node changes. Then how could I get away with not
> assigning old nodes with new tokens?
> ~Amulya
>
> On Mon, Sep 5, 2011 at 4:12 AM, Sylvain Lebresne <sy...@datastax.com>
> wrote:
>>
>> Have you done step 6 of the 'To add nodes to a Cassandra cluster' of
>> http://www.datastax.com/docs/0.8/operations/clustering#adding-capacity,
>> aka, run nodetool cleanup on the previously existing nodes ?
>>
>> --
>> Sylvain
>>
>> On Sun, Sep 4, 2011 at 11:58 AM, amulya rattan <ta...@gmail.com>
>> wrote:
>> > Hi there,
>> > I had a 3 nodes ring, added a 4th one, and moved others to appropriate
>> > tokens..doing nodetool ring shows:
>> > 127.0.0.1 datacenter1 rack1 Up Normal 348.82 MB
>> > 25.00% 0
>> > 127.0.0.2 datacenter1 rack1 Up Normal 349.81 MB
>> > 25.00% 42535295865117307932921825928971026432
>> > 127.0.0.3 datacenter1 rack1 Up Normal 437.04 MB
>> > 25.00% 85070591730234615865843651857942052864
>> > 127.0.0.4 datacenter1 rack1 Up Normal 261.74 MB
>> > 25.00% 127605887595351923798765477786913079296
>> > Although it shows 25% on all nodes, the data distribution seems totally
>> > unbalanced, how did that happen?
>> > Also if I happen to have a thousand node cluster, and add another 100
>> > nodes,
>> > is it ok to start moving all nodes to their respective new tokens
>> > simultaneously or one-by-one?
>> > Any response is appreciated.
>> > ~Amulya
>
>
Re: load balance issue
Posted by amulya rattan <ta...@gmail.com>.
Ah, missed that. Thanks for the pointer.
While we are at it, the doc says that if I am doubling the strength of my
cluster, and I assign calculated tokens to the new nodes, i don't need to do
the nodetool move for old nodes. Won't I have to assign the old nodes with
their new respective tokens too? If I had 4 nodes and I add 4 more, the
token distribution for each node changes. Then how could I get away with not
assigning old nodes with new tokens?
~Amulya
On Mon, Sep 5, 2011 at 4:12 AM, Sylvain Lebresne <sy...@datastax.com>wrote:
> Have you done step 6 of the 'To add nodes to a Cassandra cluster' of
> http://www.datastax.com/docs/0.8/operations/clustering#adding-capacity,
> aka, run nodetool cleanup on the previously existing nodes ?
>
> --
> Sylvain
>
> On Sun, Sep 4, 2011 at 11:58 AM, amulya rattan <ta...@gmail.com>
> wrote:
> > Hi there,
> > I had a 3 nodes ring, added a 4th one, and moved others to appropriate
> > tokens..doing nodetool ring shows:
> > 127.0.0.1 datacenter1 rack1 Up Normal 348.82 MB
> > 25.00% 0
> > 127.0.0.2 datacenter1 rack1 Up Normal 349.81 MB
> > 25.00% 42535295865117307932921825928971026432
> > 127.0.0.3 datacenter1 rack1 Up Normal 437.04 MB
> > 25.00% 85070591730234615865843651857942052864
> > 127.0.0.4 datacenter1 rack1 Up Normal 261.74 MB
> > 25.00% 127605887595351923798765477786913079296
> > Although it shows 25% on all nodes, the data distribution seems totally
> > unbalanced, how did that happen?
> > Also if I happen to have a thousand node cluster, and add another 100
> nodes,
> > is it ok to start moving all nodes to their respective new tokens
> > simultaneously or one-by-one?
> > Any response is appreciated.
> > ~Amulya
>
Re: load balance issue
Posted by Sylvain Lebresne <sy...@datastax.com>.
Have you done step 6 of the 'To add nodes to a Cassandra cluster' of
http://www.datastax.com/docs/0.8/operations/clustering#adding-capacity,
aka, run nodetool cleanup on the previously existing nodes ?
--
Sylvain
On Sun, Sep 4, 2011 at 11:58 AM, amulya rattan <ta...@gmail.com> wrote:
> Hi there,
> I had a 3 nodes ring, added a 4th one, and moved others to appropriate
> tokens..doing nodetool ring shows:
> 127.0.0.1 datacenter1 rack1 Up Normal 348.82 MB
> 25.00% 0
> 127.0.0.2 datacenter1 rack1 Up Normal 349.81 MB
> 25.00% 42535295865117307932921825928971026432
> 127.0.0.3 datacenter1 rack1 Up Normal 437.04 MB
> 25.00% 85070591730234615865843651857942052864
> 127.0.0.4 datacenter1 rack1 Up Normal 261.74 MB
> 25.00% 127605887595351923798765477786913079296
> Although it shows 25% on all nodes, the data distribution seems totally
> unbalanced, how did that happen?
> Also if I happen to have a thousand node cluster, and add another 100 nodes,
> is it ok to start moving all nodes to their respective new tokens
> simultaneously or one-by-one?
> Any response is appreciated.
> ~Amulya