Re: Automatic tombstone compaction

[Keith Wright <kw...@nanigans.com>]

Hi all,

    I am running C* 1.2.13 with Vnodes at around 1 TB / node.  I just
noticed that one of my larger LCS CFs (300-400 GB/Node) is showing a
droppable tombstone ration of between 23 and 28% on my nodes.  I did not
indicate a value to be used in my table creation so I assume its using the
default 20%.  Note that I utilize TTLs somewhat heavily on this table with
most between 7 and 28 days against CQL3 collections (maps to be specific).
Couple of questions:

- Given compactions are not falling behind on my nodes, I would have
expected my ratio to be much closer to the default value of 20%.  Have
others seen this behavior?

- Again given that compactions are not falling behind, I am always up for
recovering data space and that are interested in lowering the default
value of 20%.  The documentation suggests that it will not trigger a
compaction for tombstones unless there are no compactions queued so I
assume this would not have a negative impact?

Thanks!

On 8/21/13, 11:23 AM, "Yuki Morishita" <mo...@gmail.com> wrote:

>Tamas,
>
>If there are rows with the same key in other SSTables, that rows won't
>be deleted.
>Tombstone compaction make guess if it can actually drop "safely" by
>scanning overlap with other SSTables.
>Do you have many rows in your large SSTable?
>If you don't, then chance to run tombstone compaction may be low.
>
>
>On Wed, Aug 21, 2013 at 9:33 AM,  <ta...@thomsonreuters.com>
>wrote:
>> Hi,
>>
>>
>>
>> I ran upgradesstables as part of the Cassandra upgrade, before issuing
>>the
>> CQL alter command.
>>
>> According to the docs, SizeTieredCompactionStrategy is fine (that is
>>what I
>> used, and plan on continue using), and automatic tombstone compaction is
>> available for it:
>>
>> 
>>http://www.datastax.com/documentation/cassandra/1.2/webhelp/index.html#ca
>>ssandra/operations/ops_about_config_compact_c.html
>>
>> I just had to include the Œclass¹ in the alter statement, otherwise it
>>would
>> not accept my command.
>>
>> Is that not right?
>>
>>
>>
>> Thanks,
>>
>> Tamas
>>
>>
>>
>> From: Haithem Jarraya [mailto:a-hjarraya@expedia.com]
>> Sent: 21. august 2013 16:24
>> To: user@cassandra.apache.org
>> Subject: Re: Automatic tombstone compaction
>>
>>
>>
>> Hi,
>>
>>
>>
>> do you mean LeveledCompactionStrategy?
>>
>>
>>
>> Also you will need to run nodetool upgradesstables  [keyspace][cf_name]
>> after changing the compaction strategy.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Haithem Jarraya
>>
>> On 21 Aug 2013, at 15:15, tamas.foldesi@thomsonreuters.com wrote:
>>
>>
>>
>> Hi,
>>
>>
>>
>> After upgrading from 1.0 to 1.2, I wanted to make use of the automatic
>> tombstone compaction feature, so using CQL3 I issued:
>>
>>
>>
>> ALTER TABLE versions WITH compaction = {'class' :
>> 'SizeTieredCompactionStrategy', 'min_threshold' : 4, 'max_threshold' :
>>32,
>> 'tombstone_compaction_interval' : 1, 'tombstone_threshold' : '0.1'};
>>
>>
>>
>> But I still see no trace that would suggest this works ­ we had 60G of
>>data
>> with TTL=1week pushed a while ago to the test cluster, the majority of
>>it
>> should be expired & compacted away by now. Not sure if it is relevant,
>>but
>> this old data is in one ~60G file + I have a few smaller files with
>>latest
>> data in them.
>>
>> Looking at JMX: DroppableTombstoneRatio = 0.892076544, which seems to
>>back
>> my theory.
>>
>> Am I doing something wrong, or am I expecting the wrong thing?
>>
>>
>>
>> Thanks,
>>
>> Tamas
>>
>>
>>
>>
>
>
>
>-- 
>Yuki Morishita
> t:yukim (http://twitter.com/yukim)