You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Batranut Bogdan <ba...@yahoo.com> on 2014/04/24 21:50:35 UTC

Compaction throrheling

Hello all<br/>Can someone please explain to me what compaction throtheling does ? I am reffering to the yaml parameter. I have changed it from the default 16 to 160 but i see no improvement. I have a cluster with hdds. I might be missing something ...<br/><br/>Thanks<a href="https://overview.mail.yahoo.com?.src=iOS"><br/><br/>Sent from Yahoo Mail for iPhone</a>

Re: Compaction throrheling

Posted by Batranut Bogdan <ba...@yahoo.com>.
I have CPU to spare but reached hdd limit.
Well this might deserve it's own conversation thread....  but I did reach the limit of IO after after using a wide row and counter columnfamily ... def looks like this:

a int , b int, c string , timestamp int, d counter, e counter f counter, g counter PK ((a,b,c) ts) 

 After 200 mil PKs and 6 Timestamps for each of them my whole 8 core XEON, 32 Gb ram, 6 server  cluster went into medium to high load (according to opscenter) and the hdds load was ~ 100% according to iostat. 

But that is something else. 

I asked the question since my cluster is not under heavy load and I see compactions done with 4 MBs / sec (for big files according to my observations) and compactions done with 40 MBs/s for smaller ones. I need C* to do compactions as fast as possible if the system is not under load.
On Friday, April 25, 2014 12:06 AM, Robert Coli <rc...@eventbrite.com> wrote:
 
On Thu, Apr 24, 2014 at 1:54 PM, Batranut Bogdan <ba...@yahoo.com> wrote:

Is this a setting that will have an impact only on fast cpu + ssd ?

It's a setting that will only have impact if you have CPU or IO to spare. You don't need a fast CPU or SSD to meet those conditions.


=Rob

Re: Compaction throrheling

Posted by Robert Coli <rc...@eventbrite.com>.
On Thu, Apr 24, 2014 at 1:54 PM, Batranut Bogdan <ba...@yahoo.com> wrote:

> Is this a setting that will have an impact only on fast cpu + ssd ?
>

It's a setting that will only have impact if you have CPU or IO to spare.
You don't need a fast CPU or SSD to meet those conditions.

=Rob

Re: Compaction throrheling

Posted by Batranut Bogdan <ba...@yahoo.com>.
Is this a setting that will have an impact only on fast cpu + ssd ?
On Thursday, April 24, 2014 11:52 PM, Robert Coli <rc...@eventbrite.com> wrote:
 
On Thu, Apr 24, 2014 at 12:50 PM, Batranut Bogdan <ba...@yahoo.com> wrote:

Can someone please explain to me what compaction throtheling does ? I am reffering to the yaml parameter. I have changed it from the default 16 to 160 but i see no improvement. I have a cluster with hdds. I might be missing something ...
> 

It sets the maximum throughput used for compaction operations, across all compactions.

If your system is i/o or CPU bound on compaction, raising it or setting it to 0 (unlimited) will have no effect.

=Rob

Re: Compaction throrheling

Posted by Robert Coli <rc...@eventbrite.com>.
On Thu, Apr 24, 2014 at 12:50 PM, Batranut Bogdan <ba...@yahoo.com>wrote:

> Can someone please explain to me what compaction throtheling does ? I am
> reffering to the yaml parameter. I have changed it from the default 16 to
> 160 but i see no improvement. I have a cluster with hdds. I might be
> missing something ...
>

It sets the maximum throughput used for compaction operations, across all
compactions.

If your system is i/o or CPU bound on compaction, raising it or setting it
to 0 (unlimited) will have no effect.

=Rob