Cassandra Cluster issues

[Mehdi Bada <me...@dbi-services.com>]

Dear Cassandra Users, 

I have some issues since few days with the following cluster: 

- 5 nodes 
- Cassandra 3.7 
- 2 seed nodes 
- 1 keyspace with RF=2, 300Go / nodes, WRITE_LEVEL=ONE, READ_LEVEL=ONE 
- 1 enormous table (90% of the keyspace) 
- TTL for each line insered 

The cluster is write oriented. All machines are consuming between 5 - 10 % of the CPU and 45 % of RAM. 

The cluster is very slow since the last repair, not all writes have been done... I don't know how to start the debbuging of my cluster. 

Do you have any ideas ? 


Many thanks in advance 

Regards 
Mehdi Bada 

---- 

Mehdi Bada | Consultant 
Phone: +41 32 422 96 00 | Mobile: +41 79 928 75 48 | Fax: +41 32 422 96 15 
dbi services, Rue de la Jeunesse 2, CH-2800 Delémont 
mehdi.bada@dbi-services.com 
www.dbi-services.com 



⇒ dbi services is recruiting Oracle & SQL Server experts ! – Join the team 

Re: Cassandra Cluster issues

[benjamin roth <br...@gmail.com>]

Hm that question is like "My car does not start - whats the problem?".
You have to monitor, monitor, monitor, monitor. I'd strongly advice to
graph as many metrics as you can. Read them from the JMX interface and
write them to a TSDB, visualize them e.g. with Grafana.
Then read logs, trace your queries, check all the system metrics like CPU
consumption, Disk IO, Network IO, Memory usage, Java GC pauses.

Then you will be able to find the bottleneck.

2017-05-08 15:15 GMT+02:00 Mehdi Bada <me...@dbi-services.com>:

> Dear Cassandra Users,
>
> I have some issues since few days with the following cluster:
>
> - 5 nodes
> - Cassandra 3.7
> - 2 seed nodes
> - 1 keyspace with RF=2, 300Go / nodes, WRITE_LEVEL=ONE, READ_LEVEL=ONE
> - 1 enormous table (90% of the keyspace)
> - TTL for each line insered
>
> The cluster is write oriented. All machines are consuming between 5 - 10 %
> of the CPU and 45 % of RAM.
>
> The cluster is very slow since the last repair, not all writes have been
> done... I don't know how to start the debbuging of my cluster.
>
> Do you have any ideas ?
>
>
> Many thanks in advance
>
> Regards
> Mehdi Bada
>
> ----
>
> *Mehdi Bada* | Consultant
> Phone: +41 32 422 96 00 <+41%2032%20422%2096%2000> | Mobile: +41 79 928
> 75 48 <+41%2079%20928%2075%2048> | Fax: +41 32 422 96 15
> <+41%2032%20422%2096%2015>
> dbi services, Rue de la Jeunesse 2, CH-2800 Delémont
> mehdi.bada@dbi-services.com
> www.dbi-services.com
>
>
>
> *⇒ dbi services is recruiting Oracle & SQL Server experts ! – Join the
> team
> <http://www.dbi-services.com/fr/dbi-services-et-ses-collaborateurs/offres-emplois-opportunites-carrieres/>*
>
>