Never dropped droppable tombstoned data with LCS

[Safa Topal <sa...@gmail.com>]

Hello there,

We have a 3 node cluster with Cassandra 2.0.8 version. I am seeing data
that should be dropped already. In JMX, I can see that
DroppableTombstoneRatio is 0.68 for the column family and the
tombstone_threshold was left as default for the CF. We are using LCS on
related CF and replication factor of keyspace is set to 3.

We have experienced some downtimes because of "repair" and for that reason,
we are reluctant to run repair again.

How can we get rid of this tombstoned data without running repair? Or are
there any other way to run repair without exhausting the cluster? I have
seen a lot about "repair -pr", however I am not sure if it will be suitable
for our case.

Regards,
Safa T.

Re: Never dropped droppable tombstoned data with LCS

[Robert Coli <rc...@eventbrite.com>]

On Sun, Apr 26, 2015 at 1:50 PM, Safa Topal <sa...@gmail.com> wrote:

> We have a 3 node cluster with Cassandra 2.0.8 version. I am seeing data
> that should be dropped already. In JMX, I can see that
> DroppableTombstoneRatio is 0.68 for the column family and the
> tombstone_threshold was left as default for the CF. We are using LCS on
> related CF and replication factor of keyspace is set to 3.
>

2.0.8 contains significant bugs, I would upgrade to the HEAD of 2.0.x ASAP.

Regarding non-dropped data :

https://issues.apache.org/jira/browse/CASSANDRA-6654

?


> We have experienced some downtimes because of "repair" and for that
> reason, we are reluctant to run repair again.
>

Consider increasing your gc_grace_seconds to 34 days and running repair
once a month, on the first of the month, until you resolve the issue.
Not-running repair on a regular schedule will be fatal to consistency of
some data.


> How can we get rid of this tombstoned data without running repair? Or are
> there any other way to run repair without exhausting the cluster? I have
> seen a lot about "repair -pr", however I am not sure if it will be suitable
> for our case.
>

Repair -pr should always be used when you are repairing your entire
cluster; that's what it's for.

How is repair related to the non-purged data? Repair that kills you with
tombstones will probably also kill you without tombstones?

=Rob