Re: Minor Compactions Not Triggered

[Robert Coli <rc...@eventbrite.com>]

On Sun, May 31, 2015 at 11:37 AM, Anuj Wadehra <an...@yahoo.co.in>
wrote:

> 2. We thought that CQL compaction subproperty of *tombstone_threshold*
> will help us after major compactions. This property will ensure that even
> if we have one huge sstable, once tombstone threshold of 20% has reached,
> sstables will be compacted and tombstones will be dropped after
> gc_grace_periods (even if there no similar sized sstables as need by STCS).
> But in our initial testing, single huge sstable is not getting compacted
> even if we drop all rows in it and gc_grace_period has passed.  *Why
> tombstone_threshold is behaving like that?*
>

https://issues.apache.org/jira/browse/CASSANDRA-6654 ?

=Rob

Re: Minor Compactions Not Triggered

[Anuj Wadehra <an...@yahoo.co.in>]

Hi Robert,
I haven't yet asked that question in IRC channel. We are generating regular sstables in CF so that's not a concern. Important part is that we need to understand tombstone_threshold property because we think that it's very important once we have done major compaction. How the huge table will get rid of tombstones when there are no similar sized tables for long time in STCS? 

I think, I understand the behavior correctly now but the code issue in Cassandra 2.0.3 is preventing this property to work properly in our testing.
Below check is missing is 2.0.3 and available in 2.0.4 (https://issues.apache.org/jira/browse/CASSANDRA-6483). Thus, even if cold_reads_to_omit is 0, cold sstables are filtered out and never get compacted. tombstone_threshold property is applied after filtering so it's not working on single huge cold sstable generated after major compaction. I will again test this property by making some frequent reads and check how it works.

 static List<SSTableReader> filterColdSSTables(List<SSTableReader> sstables, double coldReadsToOmit)
    {
        if (coldReadsToOmit == 0.0)
            return sstables;

ThanksAnuj Wadehra


 


     On Tuesday, 2 June 2015 11:37 PM, Robert Coli <rc...@eventbrite.com> wrote:
   

 On Mon, Jun 1, 2015 at 11:25 AM, Anuj Wadehra <an...@yahoo.co.in> wrote:


| As per the algorithm shared in the CASSANDRA 6654, I understand that tombstone_threshold property only comes into picture if you have expirying columns and it wont have any effect if you have manually deleted rows in cf. Is my understanding correct?
According to you What would be the expected behavior of following steps??
I inserted x rowsI deleted x rowsRan major compaction to make sure that one big sstable contains all tombstonesWaited for gc grace period to see whether that big sstable formed after major compaction is compacted on its own without finding any other sstable |



That's a good question, and I don't actually know the answer. If you aren't generating new SSTables in the CF via writes and flushes, I would doubt any background process notices it's expired and re-compacts it. 
Have you considered asking this question in the #cassandra IRC channel on freenode?
=Rob 


  

Re: Minor Compactions Not Triggered

[Robert Coli <rc...@eventbrite.com>]

On Mon, Jun 1, 2015 at 11:25 AM, Anuj Wadehra <an...@yahoo.co.in>
wrote:

> As per the algorithm shared in the CASSANDRA 6654, I understand that
> tombstone_threshold property only comes into picture if you have expirying
> columns and it wont have any effect if you have manually deleted rows in
> cf. Is my understanding correct?
>
> According to you What would be the expected behavior of following steps??
>
> I inserted x rows
> I deleted x rows
> Ran major compaction to make sure that one big sstable contains all
> tombstones
> Waited for gc grace period to see whether that big sstable formed after
> major compaction is compacted on its own without finding any other sstable
>

That's a good question, and I don't actually know the answer. If you aren't
generating new SSTables in the CF via writes and flushes, I would doubt any
background process notices it's expired and re-compacts it.

Have you considered asking this question in the #cassandra IRC channel on
freenode?

=Rob

Re: Minor Compactions Not Triggered

[Anuj Wadehra <an...@yahoo.co.in>]

Thanks Robert !!!


As per the algorithm shared in the CASSANDRA 6654, I understand that tombstone_threshold property only comes into picture if you have expirying columns and it wont have any effect if you have manually deleted rows in cf. Is my understanding correct?


According to you What would be the expected behavior of following steps??


I inserted x rows

I deleted x rows

Ran major compaction to make sure that one big sstable contains all tombstones

Waited for gc grace period to see whether that big sstable formed after major compaction is compacted on its own without finding any other sstable


Thanks

Anuj



Sent from Yahoo Mail on Android

From:"Robert Coli" <rc...@eventbrite.com>
Date:Mon, 1 Jun, 2015 at 10:56 pm
Subject:Re: Minor Compactions Not Triggered

On Sun, May 31, 2015 at 11:37 AM, Anuj Wadehra <an...@yahoo.co.in> wrote:

2. We thought that CQL compaction subproperty of tombstone_threshold will help us after major compactions. This property will ensure that even if we have one huge sstable, once tombstone threshold of 20% has reached, sstables will be compacted and tombstones will be dropped after gc_grace_periods (even if there no similar sized sstables as need by STCS). But in our initial testing, single huge sstable is not getting compacted even if we drop all rows in it and gc_grace_period has passed.  Why tombstone_threshold is behaving like that?


https://issues.apache.org/jira/browse/CASSANDRA-6654 ?


=Rob