high context switches

[Jan Karlsson <ja...@ericsson.com>]

Hello,

We are running a 3 node cluster with RF=3 and 5 clients in a test environment. The C* settings are mostly default. We noticed quite high context switching during our tests. On 100 000 000 keys/partitions we averaged around 260 000 cs (with a max of 530 000).

We were running 12 000~ transactions per second. 10 000 reads and 2000 updates.

Nothing really wrong with that however I would like to understand why these numbers are so high. Have others noticed this behavior? How much context switching is expected and why? What are the variables that affect this?

/J

Re: high context switches

[Nikolai Grigoriev <ng...@gmail.com>]

How do the clients connect, which protocol is used and do they use
keep-alive connections? Is it possible that the clients use Thrift and the
server type is "sync"? It is just my guess, but in this scenario with high
number of clients connecting-disconnecting often there may be high number
of context switching.

On Fri, Nov 21, 2014 at 4:21 AM, Jan Karlsson <ja...@ericsson.com>
wrote:

>  Hello,
>
>
>
> We are running a 3 node cluster with RF=3 and 5 clients in a test
> environment. The C* settings are mostly default. We noticed quite high
> context switching during our tests. On 100 000 000 keys/partitions we
> averaged around 260 000 cs (with a max of 530 000).
>
>
>
> We were running 12 000~ transactions per second. 10 000 reads and 2000
> updates.
>
>
>
> Nothing really wrong with that however I would like to understand why
> these numbers are so high. Have others noticed this behavior? How much
> context switching is expected and why? What are the variables that affect
> this?
>
>
>
> /J
>



-- 
Nikolai Grigoriev
(514) 772-5178

RE: high context switches

[Jan Karlsson <ja...@ericsson.com>]

We use CQL with 1 session per client and default connection settings.

I do not think that we are using too many client threads. Number of native transport threads is set to default (max 128).


From: Robert Coli [mailto:rcoli@eventbrite.com]
Sent: den 21 november 2014 19:30
To: user@cassandra.apache.org
Subject: Re: high context switches

On Fri, Nov 21, 2014 at 1:21 AM, Jan Karlsson <ja...@ericsson.com>> wrote:
Nothing really wrong with that however I would like to understand why these numbers are so high. Have others noticed this behavior? How much context switching is expected and why? What are the variables that affect this?

I +1 Nikolai's conjecture that you are probably using a very high number of client threads.

However as a general statement Cassandra is highly multi-threaded. Threads are assigned within thread pools and these thread pools can be thought of as a type of processing pipeline, such that one is often the input to another. When pushing Cassandra near its maximum capacity, you will therefore spend a lot of time switching between threads.

=Rob
http://twitter.com/rcolidba

Re: high context switches

[Robert Coli <rc...@eventbrite.com>]

On Fri, Nov 21, 2014 at 1:21 AM, Jan Karlsson <ja...@ericsson.com>
wrote:

>  Nothing really wrong with that however I would like to understand why
> these numbers are so high. Have others noticed this behavior? How much
> context switching is expected and why? What are the variables that affect
> this?
>

I +1 Nikolai's conjecture that you are probably using a very high number of
client threads.

However as a general statement Cassandra is highly multi-threaded. Threads
are assigned within thread pools and these thread pools can be thought of
as a type of processing pipeline, such that one is often the input to
another. When pushing Cassandra near its maximum capacity, you will
therefore spend a lot of time switching between threads.

=Rob
http://twitter.com/rcolidba