Can you query Cassandra while it's doing major compaction

[myreasoner <my...@gmail.com>]

If every node in the cluster is running major compaction, would it be able to
answer any read request?  And is it wise to write anything to a cluster
while it's doing major compaction?



--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Can-you-query-Cassandra-while-it-s-doing-major-compaction-tp7249985p7249985.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.

Re: Can you query Cassandra while it's doing major compaction

["R. Verlangen" <ro...@us2.nl>]

It will have a performance penalty, so it would be better to spread the
compactions over a period of time. But Cassandra will still take care of
any reads/writes (within the given timeout).

2012/2/3 myreasoner <my...@gmail.com>

> If every node in the cluster is running major compaction, would it be able
> to
> answer any read request?  And is it wise to write anything to a cluster
> while it's doing major compaction?
>
>
>
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Can-you-query-Cassandra-while-it-s-doing-major-compaction-tp7249985p7249985.html
> Sent from the cassandra-user@incubator.apache.org mailing list archive at
> Nabble.com.
>

Re: Can you query Cassandra while it's doing major compaction

[Adrian Cockcroft <ad...@gmail.com>]

At Netflix we rotate the major compactions around the cluster, don't
run them all at once. We also either take that node out of client
traffic so it doesn't get used as a coordinator or use the Astyanax
client that is latency and token aware to steer traffic to the other
replicas.

We are running on EC2 with lots of CPU and RAM but only two internal
disk spindles, so if you have lots of IOPS available in your config,
you may find that it doesn't affect read times much.

Adrian

On Thu, Feb 2, 2012 at 11:44 PM, Peter Schuller
<pe...@infidyne.com> wrote:
>> If every node in the cluster is running major compaction, would it be able to
>> answer any read request?  And is it wise to write anything to a cluster
>> while it's doing major compaction?
>
> Compaction is something that is supposed to be continuously running in
> the background. As noted, it will have a performance impact in that it
> (1) generates I/O, (2) leads to cache eviction, and (if you're CPU
> bound rather than disk bound) (3) adds CPU load.
>
> But there is no intention that clients should have to care about
> compaction; it's to be viewed as a background operation continuously
> happening. A good rule of thumb is that an individual node should be
> able to handle your traffic when doing compaction; you don't want to
> be in the position where you're just barely dealing with the traffic,
> and a node doing compaction not being able to handle it.
>
> --
> / Peter Schuller (@scode, http://worldmodscode.wordpress.com)

Re: Can you query Cassandra while it's doing major compaction

[Peter Schuller <pe...@infidyne.com>]

> If every node in the cluster is running major compaction, would it be able to
> answer any read request?  And is it wise to write anything to a cluster
> while it's doing major compaction?

Compaction is something that is supposed to be continuously running in
the background. As noted, it will have a performance impact in that it
(1) generates I/O, (2) leads to cache eviction, and (if you're CPU
bound rather than disk bound) (3) adds CPU load.

But there is no intention that clients should have to care about
compaction; it's to be viewed as a background operation continuously
happening. A good rule of thumb is that an individual node should be
able to handle your traffic when doing compaction; you don't want to
be in the position where you're just barely dealing with the traffic,
and a node doing compaction not being able to handle it.

-- 
/ Peter Schuller (@scode, http://worldmodscode.wordpress.com)