You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Jabbar <aj...@gmail.com> on 2013/01/29 17:29:17 UTC
Uneven CPU load on a 4 node cluster
Hello,
I've been testing a four identical node cassanda 1.2 cluster for a number
of days. I have written a c# client using cassandra sharp() which inserts
data into a table.
The keyspace difinition is
CREATE KEYSPACE "data"
WITH REPLICATION = {'class' : 'NetworkTopologyStrategy', 'dc1' : 3};
The table definition is
CREATE TABLE datapoints (
siteid bigint,
time timestamp,
channel int,
data float,
PRIMARY KEY ((siteid, channel),time)
)
I am finding that the CPU load on one of the servers stays at ~90% whilst
the load on the other servers stays < 40%. All the servers are supposed to
be identical.
The client library I am using does load balancing between all nodes.
I have also used the cassandra stress tool as follows
cassandra-stress -d 192.168.21.7,192.168.21.9,192.168.21.12,192.168.21.14
--replication-factor 3 -n 10000000 -t 100
and have found that it behaves similarly.
Can somebody explain why this happens?
--
Thanks
A Jabbar Azam
Re: Uneven CPU load on a 4 node cluster
Posted by Andreas Wagner <an...@googlemail.com>.
Hi all,
I'm experiencing very similar effects. Did you (or anyone for that
matter) have/solvethis issue?
I have a 3 node cluster with vnodes having the same #tokens (256).
Infact, all nodes are configured identical and share similar/same
hardware. Cassandra.yaml settings are fairly standard - nothing fancy.
According to "nodetool status" command everything is perfectly balanced.
Running "cassandra-stress -d node_ip1,node_ip2,node_ip3" causes a heavy
load on node_ip1, while node_ip2/3 are almost idle. Data, however, seems
to be distributed evenly. I even get "UnavailableException" for some
keys to be inserted on node_ip1.
I also tried a second run with the scheduling set to "roundrobin" and
made use of the standard "throttlingoption". Unfortunately, nothing changed.
Could someone please provide some pointersand/or insights what I'm doing
wrong?
Thanks so much!
Andreas
Re: Uneven CPU load on a 4 node cluster
Posted by Jabbar <aj...@gmail.com>.
The high CPU node got replaced and now I'm not getting abnormally high CPU
from one node. They all are evenly balanced now.
On 29 January 2013 16:29, Jabbar <aj...@gmail.com> wrote:
> Hello,
>
> I've been testing a four identical node cassanda 1.2 cluster for a number
> of days. I have written a c# client using cassandra sharp() which inserts
> data into a table.
>
> The keyspace difinition is
>
> CREATE KEYSPACE "data"
> WITH REPLICATION = {'class' : 'NetworkTopologyStrategy', 'dc1' : 3};
>
>
> The table definition is
>
> CREATE TABLE datapoints (
> siteid bigint,
> time timestamp,
> channel int,
> data float,
> PRIMARY KEY ((siteid, channel),time)
> )
>
>
> I am finding that the CPU load on one of the servers stays at ~90% whilst
> the load on the other servers stays < 40%. All the servers are supposed to
> be identical.
>
> The client library I am using does load balancing between all nodes.
>
> I have also used the cassandra stress tool as follows
>
> cassandra-stress -d 192.168.21.7,192.168.21.9,192.168.21.12,192.168.21.14
> --replication-factor 3 -n 10000000 -t 100
>
> and have found that it behaves similarly.
>
> Can somebody explain why this happens?
>
>
>
>
> --
> Thanks
>
> A Jabbar Azam
>
--
Thanks
A Jabbar Azam
Re: Uneven CPU load on a 4 node cluster
Posted by Jabbar <aj...@gmail.com>.
Forgot to mention that I also used
ALTER KEYSPACE "Keyspace1" WITH REPLICATION =
{ 'class' : 'SimpleStrategy', 'replication_factor' : 3 };
To change the replication factor for Keyspace1. For some reason the command
line doesn't me to change the replication factor. I get the following error
Unable to create stress keyspace: Keyspace names must be case-insensitively
unique ("Keyspace1" conflicts with "Keyspace1")
On 29 January 2013 16:29, Jabbar <aj...@gmail.com> wrote:
> Hello,
>
> I've been testing a four identical node cassanda 1.2 cluster for a number
> of days. I have written a c# client using cassandra sharp() which inserts
> data into a table.
>
> The keyspace difinition is
>
> CREATE KEYSPACE "data"
> WITH REPLICATION = {'class' : 'NetworkTopologyStrategy', 'dc1' : 3};
>
>
> The table definition is
>
> CREATE TABLE datapoints (
> siteid bigint,
> time timestamp,
> channel int,
> data float,
> PRIMARY KEY ((siteid, channel),time)
> )
>
>
> I am finding that the CPU load on one of the servers stays at ~90% whilst
> the load on the other servers stays < 40%. All the servers are supposed to
> be identical.
>
> The client library I am using does load balancing between all nodes.
>
> I have also used the cassandra stress tool as follows
>
> cassandra-stress -d 192.168.21.7,192.168.21.9,192.168.21.12,192.168.21.14
> --replication-factor 3 -n 10000000 -t 100
>
> and have found that it behaves similarly.
>
> Can somebody explain why this happens?
>
>
>
>
> --
> Thanks
>
> A Jabbar Azam
>
--
Thanks
A Jabbar Azam