You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by James Golick <ja...@gmail.com> on 2010/05/29 21:55:07 UTC
Compaction bringing a node to its knees
I just experienced a compaction that brought a node to 100% of its IO
capacity and made its responses incredibly slow.
It wasn't enough to make the node actually appear as down, though, so it
slowed down the operation of the cluster considerably.
The CF being compacted contains a lot of relatively wide rows (hundreds of
thousands or millions of columns on the big end). Is that the problem?
Any suggestions on how to minimize impact here?
Re: Compaction bringing a node to its knees
Posted by Jonathan Ellis <jb...@gmail.com>.
I'm curious, did this help at all?
On Sat, May 29, 2010 at 3:03 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> You could try setting the compaction thread to a lower priority. You
> could add a thread priority to NamedThreadPool, and pass that up from
> CompactionExecutor constructor. According to
> http://www.javamex.com/tutorials/threads/priority_what.shtml you have
> to run as root and add a JVM option to get this to work.
>
> On Sat, May 29, 2010 at 2:55 PM, James Golick <ja...@gmail.com> wrote:
>> I just experienced a compaction that brought a node to 100% of its IO
>> capacity and made its responses incredibly slow.
>> It wasn't enough to make the node actually appear as down, though, so it
>> slowed down the operation of the cluster considerably.
>> The CF being compacted contains a lot of relatively wide rows (hundreds of
>> thousands or millions of columns on the big end). Is that the problem?
>> Any suggestions on how to minimize impact here?
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com
Re: Compaction bringing a node to its knees
Posted by Jonathan Ellis <jb...@gmail.com>.
You could try setting the compaction thread to a lower priority. You
could add a thread priority to NamedThreadPool, and pass that up from
CompactionExecutor constructor. According to
http://www.javamex.com/tutorials/threads/priority_what.shtml you have
to run as root and add a JVM option to get this to work.
On Sat, May 29, 2010 at 2:55 PM, James Golick <ja...@gmail.com> wrote:
> I just experienced a compaction that brought a node to 100% of its IO
> capacity and made its responses incredibly slow.
> It wasn't enough to make the node actually appear as down, though, so it
> slowed down the operation of the cluster considerably.
> The CF being compacted contains a lot of relatively wide rows (hundreds of
> thousands or millions of columns on the big end). Is that the problem?
> Any suggestions on how to minimize impact here?
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com