You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Oleg Dulin <ol...@gmail.com> on 2014/10/28 11:35:29 UTC
EC2 Snitch load imbalance
I have a setup with 6 cassandra nodes (1.2.18), using RandomPartition,
not using vnodes -- this is a legacy cluster.
We went from 3 nodes to 6 in the last few days to add capacity.
However, there appears to be an imbalance:
Datacenter: us-east
==========
Replicas: 2
Address Rack Status State Load Owns
Token
113427455640312821154458202477256070484
x.x.x.73 1d Up Normal 154.64 GB 33.33%
85070591730234615865843651857942052863
x.x.x.251 1a Up Normal 62.26 GB 16.67%
28356863910078205288614550619314017621
x.x.x.238 1b Up Normal 243.7 GB 50.00%
56713727820156410577229101238628035242
x.x.x.25 1a Up Normal 169.3 GB 33.33% 210
x.x.x.162 1b Up Normal 118.24 GB 50.00%
141784319550391026443072753096570088105
x.x.x.208 1d Up Normal 226.85 GB 16.67%
113427455640312821154458202477256070484
What is the cause of this imbalance ? How can I rectify it ?
Regards,
Oleg
Re: EC2 Snitch load imbalance
Posted by Robert Coli <rc...@eventbrite.com>.
On Tue, Oct 28, 2014 at 3:35 AM, Oleg Dulin <ol...@gmail.com> wrote:
> I have a setup with 6 cassandra nodes (1.2.18), using RandomPartition, not
> using vnodes -- this is a legacy cluster.
>
> We went from 3 nodes to 6 in the last few days to add capacity. However,
> there appears to be an imbalance:
>
The first step to check is if the ranges are even.
Did you pick initial_token for these new nodes?
=Rob
Re: EC2 Snitch load imbalance
Posted by Oleg Dulin <ol...@gmail.com>.
Thanks Mark.
The output in my original post is with keyspace specified.
On 2014-10-28 12:00:15 +0000, Mark Reddy said:
> Oleg,
>
> If you are running nodetool status, be sure to specify the keyspace
> also. If you don't specify the keyspace the results will be nonsense.
>
> https://issues.apache.org/jira/browse/CASSANDRA-7173
>
>
> Regards,
> Mark
>
> On 28 October 2014 10:35, Oleg Dulin <ol...@gmail.com> wrote:
> I have a setup with 6 cassandra nodes (1.2.18), using RandomPartition,
> not using vnodes -- this is a legacy cluster.
>
> We went from 3 nodes to 6 in the last few days to add capacity.
> However, there appears to be an imbalance:
>
> Datacenter: us-east
> ==========
> Replicas: 2
>
> Address Rack Status State Load Owns
> Token
>
> 113427455640312821154458202477256070484
> x.x.x.73 1d Up Normal 154.64 GB 33.33%
> 85070591730234615865843651857942052863
> x.x.x.251 1a Up Normal 62.26 GB 16.67%
> 28356863910078205288614550619314017621
> x.x.x.238 1b Up Normal 243.7 GB 50.00%
> 56713727820156410577229101238628035242
> x.x.x.25 1a Up Normal 169.3 GB 33.33% 210
> x.x.x.162 1b Up Normal 118.24 GB 50.00%
> 141784319550391026443072753096570088105
> x.x.x.208 1d Up Normal 226.85 GB 16.67%
> 113427455640312821154458202477256070484
>
>
> What is the cause of this imbalance ? How can I rectify it ?
>
> Regards,
> Oleg
Re: EC2 Snitch load imbalance
Posted by Mark Reddy <ma...@gmail.com>.
Oleg,
If you are running nodetool status, be sure to specify the keyspace also.
If you don't specify the keyspace the results will be nonsense.
https://issues.apache.org/jira/browse/CASSANDRA-7173
Regards,
Mark
On 28 October 2014 10:35, Oleg Dulin <ol...@gmail.com> wrote:
> I have a setup with 6 cassandra nodes (1.2.18), using RandomPartition, not
> using vnodes -- this is a legacy cluster.
>
> We went from 3 nodes to 6 in the last few days to add capacity. However,
> there appears to be an imbalance:
>
> Datacenter: us-east
> ==========
> Replicas: 2
>
> Address Rack Status State Load Owns
> Token
>
> 113427455640312821154458202477256070484
> x.x.x.73 1d Up Normal 154.64 GB 33.33%
> 85070591730234615865843651857942052863
> x.x.x.251 1a Up Normal 62.26 GB 16.67%
> 28356863910078205288614550619314017621
> x.x.x.238 1b Up Normal 243.7 GB 50.00%
> 56713727820156410577229101238628035242
> x.x.x.25 1a Up Normal 169.3 GB 33.33%
> 210
> x.x.x.162 1b Up Normal 118.24 GB 50.00%
> 141784319550391026443072753096570088105
> x.x.x.208 1d Up Normal 226.85 GB 16.67%
> 113427455640312821154458202477256070484
>
>
> What is the cause of this imbalance ? How can I rectify it ?
>
> Regards,
> Oleg
>
>
>