You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by ta...@thomsonreuters.com on 2013/08/21 16:15:12 UTC
Automatic tombstone compaction
Hi,
After upgrading from 1.0 to 1.2, I wanted to make use of the automatic tombstone compaction feature, so using CQL3 I issued:
ALTER TABLE versions WITH compaction = {'class' : 'SizeTieredCompactionStrategy', 'min_threshold' : 4, 'max_threshold' : 32, 'tombstone_compaction_interval' : 1, 'tombstone_threshold' : '0.1'};
But I still see no trace that would suggest this works - we had 60G of data with TTL=1week pushed a while ago to the test cluster, the majority of it should be expired & compacted away by now. Not sure if it is relevant, but this old data is in one ~60G file + I have a few smaller files with latest data in them.
Looking at JMX: DroppableTombstoneRatio = 0.892076544, which seems to back my theory.
Am I doing something wrong, or am I expecting the wrong thing?
Thanks,
Tamas
RE: Automatic tombstone compaction
Posted by ta...@thomsonreuters.com.
Actually each column in a row has the same TTL, so I guess that will not cause fragmentation...?
Anyway - any feedback on the questions in my previous mail?
________________________________
From: Nate McCall [nate@thelastpickle.com]
Sent: Wednesday, August 21, 2013 8:26 PM
To: Cassandra Users
Subject: Re: Automatic tombstone compaction
In the context of Yuki's response, if you are using the same key for the cache, then your rows will get increasingly fragmented.
On Wed, Aug 21, 2013 at 1:09 PM, <ta...@thomsonreuters.com>> wrote:
Well, these tables are somewhat similar to a 'cache' - we insert rows, then leave them for a week using TTL (usually untouched, read only), and then we need to compact them away. If I understand correctly, they should not be affected by the below issue...
The question is rather if the setup is correct - i.e the ALTER statement I quoted below?
Also, I the docs were not clear to me - is automatic tombstone compaction 'on' by default (even if the sstables have been migrated from an older version)? Or do I need to alter the columnfamily (like I did) to turn it on?
________________________________
From: Robert Coli [rcoli@eventbrite.com<ma...@eventbrite.com>]
Sent: Wednesday, August 21, 2013 7:45 PM
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: Re: Automatic tombstone compaction
On Wed, Aug 21, 2013 at 8:23 AM, Yuki Morishita <mo...@gmail.com>> wrote:
If there are rows with the same key in other SSTables, that rows won't
be deleted.
Tombstone compaction make guess if it can actually drop "safely" by
scanning overlap with other SSTables.
Background @ :
https://issues.apache.org/jira/browse/CASSANDRA-1074
=Rob
Re: Automatic tombstone compaction
Posted by Nate McCall <na...@thelastpickle.com>.
In the context of Yuki's response, if you are using the same key for the
cache, then your rows will get increasingly fragmented.
On Wed, Aug 21, 2013 at 1:09 PM, <ta...@thomsonreuters.com> wrote:
> Well, these tables are somewhat similar to a 'cache' - we insert rows,
> then leave them for a week using TTL (usually untouched, read only), and
> then we need to compact them away. If I understand correctly, they should
> not be affected by the below issue...
> The question is rather if the setup is correct - i.e the ALTER statement I
> quoted below?
>
> Also, I the docs were not clear to me - is automatic tombstone
> compaction 'on' by default (even if the sstables have been migrated from an
> older version)? Or do I need to alter the columnfamily (like I did) to turn
> it on?
>
> ------------------------------
> *From:* Robert Coli [rcoli@eventbrite.com]
> *Sent:* Wednesday, August 21, 2013 7:45 PM
>
> *To:* user@cassandra.apache.org
> *Subject:* Re: Automatic tombstone compaction
>
> On Wed, Aug 21, 2013 at 8:23 AM, Yuki Morishita <mo...@gmail.com>wrote:
>
>> If there are rows with the same key in other SSTables, that rows won't
>> be deleted.
>> Tombstone compaction make guess if it can actually drop "safely" by
>> scanning overlap with other SSTables.
>>
>
> Background @ :
>
> https://issues.apache.org/jira/browse/CASSANDRA-1074
>
> =Rob
>
>
RE: Automatic tombstone compaction
Posted by ta...@thomsonreuters.com.
Well, these tables are somewhat similar to a 'cache' - we insert rows, then leave them for a week using TTL (usually untouched, read only), and then we need to compact them away. If I understand correctly, they should not be affected by the below issue...
The question is rather if the setup is correct - i.e the ALTER statement I quoted below?
Also, I the docs were not clear to me - is automatic tombstone compaction 'on' by default (even if the sstables have been migrated from an older version)? Or do I need to alter the columnfamily (like I did) to turn it on?
________________________________
From: Robert Coli [rcoli@eventbrite.com]
Sent: Wednesday, August 21, 2013 7:45 PM
To: user@cassandra.apache.org
Subject: Re: Automatic tombstone compaction
On Wed, Aug 21, 2013 at 8:23 AM, Yuki Morishita <mo...@gmail.com>> wrote:
If there are rows with the same key in other SSTables, that rows won't
be deleted.
Tombstone compaction make guess if it can actually drop "safely" by
scanning overlap with other SSTables.
Background @ :
https://issues.apache.org/jira/browse/CASSANDRA-1074
=Rob
Re: Automatic tombstone compaction
Posted by Robert Coli <rc...@eventbrite.com>.
On Wed, Aug 21, 2013 at 8:23 AM, Yuki Morishita <mo...@gmail.com> wrote:
> If there are rows with the same key in other SSTables, that rows won't
> be deleted.
> Tombstone compaction make guess if it can actually drop "safely" by
> scanning overlap with other SSTables.
>
Background @ :
https://issues.apache.org/jira/browse/CASSANDRA-1074
=Rob
Re: Automatic tombstone compaction
Posted by Keith Wright <kw...@nanigans.com>.
Hi all,
I am running C* 1.2.13 with Vnodes at around 1 TB / node. I just
noticed that one of my larger LCS CFs (300-400 GB/Node) is showing a
droppable tombstone ration of between 23 and 28% on my nodes. I did not
indicate a value to be used in my table creation so I assume its using the
default 20%. Note that I utilize TTLs somewhat heavily on this table with
most between 7 and 28 days against CQL3 collections (maps to be specific).
Couple of questions:
- Given compactions are not falling behind on my nodes, I would have
expected my ratio to be much closer to the default value of 20%. Have
others seen this behavior?
- Again given that compactions are not falling behind, I am always up for
recovering data space and that are interested in lowering the default
value of 20%. The documentation suggests that it will not trigger a
compaction for tombstones unless there are no compactions queued so I
assume this would not have a negative impact?
Thanks!
On 8/21/13, 11:23 AM, "Yuki Morishita" <mo...@gmail.com> wrote:
>Tamas,
>
>If there are rows with the same key in other SSTables, that rows won't
>be deleted.
>Tombstone compaction make guess if it can actually drop "safely" by
>scanning overlap with other SSTables.
>Do you have many rows in your large SSTable?
>If you don't, then chance to run tombstone compaction may be low.
>
>
>On Wed, Aug 21, 2013 at 9:33 AM, <ta...@thomsonreuters.com>
>wrote:
>> Hi,
>>
>>
>>
>> I ran upgradesstables as part of the Cassandra upgrade, before issuing
>>the
>> CQL alter command.
>>
>> According to the docs, SizeTieredCompactionStrategy is fine (that is
>>what I
>> used, and plan on continue using), and automatic tombstone compaction is
>> available for it:
>>
>>
>>http://www.datastax.com/documentation/cassandra/1.2/webhelp/index.html#ca
>>ssandra/operations/ops_about_config_compact_c.html
>>
>> I just had to include the Œclass¹ in the alter statement, otherwise it
>>would
>> not accept my command.
>>
>> Is that not right?
>>
>>
>>
>> Thanks,
>>
>> Tamas
>>
>>
>>
>> From: Haithem Jarraya [mailto:a-hjarraya@expedia.com]
>> Sent: 21. august 2013 16:24
>> To: user@cassandra.apache.org
>> Subject: Re: Automatic tombstone compaction
>>
>>
>>
>> Hi,
>>
>>
>>
>> do you mean LeveledCompactionStrategy?
>>
>>
>>
>> Also you will need to run nodetool upgradesstables [keyspace][cf_name]
>> after changing the compaction strategy.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Haithem Jarraya
>>
>> On 21 Aug 2013, at 15:15, tamas.foldesi@thomsonreuters.com wrote:
>>
>>
>>
>> Hi,
>>
>>
>>
>> After upgrading from 1.0 to 1.2, I wanted to make use of the automatic
>> tombstone compaction feature, so using CQL3 I issued:
>>
>>
>>
>> ALTER TABLE versions WITH compaction = {'class' :
>> 'SizeTieredCompactionStrategy', 'min_threshold' : 4, 'max_threshold' :
>>32,
>> 'tombstone_compaction_interval' : 1, 'tombstone_threshold' : '0.1'};
>>
>>
>>
>> But I still see no trace that would suggest this works we had 60G of
>>data
>> with TTL=1week pushed a while ago to the test cluster, the majority of
>>it
>> should be expired & compacted away by now. Not sure if it is relevant,
>>but
>> this old data is in one ~60G file + I have a few smaller files with
>>latest
>> data in them.
>>
>> Looking at JMX: DroppableTombstoneRatio = 0.892076544, which seems to
>>back
>> my theory.
>>
>> Am I doing something wrong, or am I expecting the wrong thing?
>>
>>
>>
>> Thanks,
>>
>> Tamas
>>
>>
>>
>>
>
>
>
>--
>Yuki Morishita
> t:yukim (http://twitter.com/yukim)
Re: Automatic tombstone compaction
Posted by Yuki Morishita <mo...@gmail.com>.
Tamas,
If there are rows with the same key in other SSTables, that rows won't
be deleted.
Tombstone compaction make guess if it can actually drop "safely" by
scanning overlap with other SSTables.
Do you have many rows in your large SSTable?
If you don't, then chance to run tombstone compaction may be low.
On Wed, Aug 21, 2013 at 9:33 AM, <ta...@thomsonreuters.com> wrote:
> Hi,
>
>
>
> I ran upgradesstables as part of the Cassandra upgrade, before issuing the
> CQL alter command.
>
> According to the docs, SizeTieredCompactionStrategy is fine (that is what I
> used, and plan on continue using), and automatic tombstone compaction is
> available for it:
>
> http://www.datastax.com/documentation/cassandra/1.2/webhelp/index.html#cassandra/operations/ops_about_config_compact_c.html
>
> I just had to include the ‘class’ in the alter statement, otherwise it would
> not accept my command.
>
> Is that not right?
>
>
>
> Thanks,
>
> Tamas
>
>
>
> From: Haithem Jarraya [mailto:a-hjarraya@expedia.com]
> Sent: 21. august 2013 16:24
> To: user@cassandra.apache.org
> Subject: Re: Automatic tombstone compaction
>
>
>
> Hi,
>
>
>
> do you mean LeveledCompactionStrategy?
>
>
>
> Also you will need to run nodetool upgradesstables [keyspace][cf_name]
> after changing the compaction strategy.
>
>
>
> Thanks,
>
>
>
> Haithem Jarraya
>
> On 21 Aug 2013, at 15:15, tamas.foldesi@thomsonreuters.com wrote:
>
>
>
> Hi,
>
>
>
> After upgrading from 1.0 to 1.2, I wanted to make use of the automatic
> tombstone compaction feature, so using CQL3 I issued:
>
>
>
> ALTER TABLE versions WITH compaction = {'class' :
> 'SizeTieredCompactionStrategy', 'min_threshold' : 4, 'max_threshold' : 32,
> 'tombstone_compaction_interval' : 1, 'tombstone_threshold' : '0.1'};
>
>
>
> But I still see no trace that would suggest this works – we had 60G of data
> with TTL=1week pushed a while ago to the test cluster, the majority of it
> should be expired & compacted away by now. Not sure if it is relevant, but
> this old data is in one ~60G file + I have a few smaller files with latest
> data in them.
>
> Looking at JMX: DroppableTombstoneRatio = 0.892076544, which seems to back
> my theory.
>
> Am I doing something wrong, or am I expecting the wrong thing?
>
>
>
> Thanks,
>
> Tamas
>
>
>
>
--
Yuki Morishita
t:yukim (http://twitter.com/yukim)
Re: Automatic tombstone compaction
Posted by Haithem Jarraya <a-...@expedia.com>.
I guess I didn't understand your question then, I thought you changed the compaction strategy. If that what you did, you have to run upgradesstaable again.
On 21 Aug 2013, at 15:33, tamas.foldesi@thomsonreuters.com<ma...@thomsonreuters.com> wrote:
Hi,
I ran upgradesstables as part of the Cassandra upgrade, before issuing the CQL alter command.
According to the docs, SizeTieredCompactionStrategy is fine (that is what I used, and plan on continue using), and automatic tombstone compaction is available for it:
http://www.datastax.com/documentation/cassandra/1.2/webhelp/index.html#cassandra/operations/ops_about_config_compact_c.html
I just had to include the ‘class’ in the alter statement, otherwise it would not accept my command.
Is that not right?
Thanks,
Tamas
From: Haithem Jarraya [mailto:a-hjarraya@expedia.com<ma...@expedia.com>]
Sent: 21. august 2013 16:24
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Subject: Re: Automatic tombstone compaction
Hi,
do you mean LeveledCompactionStrategy?
Also you will need to run nodetool upgradesstables [keyspace][cf_name] after changing the compaction strategy.
Thanks,
Haithem Jarraya
On 21 Aug 2013, at 15:15, tamas.foldesi@thomsonreuters.com<ma...@thomsonreuters.com> wrote:
Hi,
After upgrading from 1.0 to 1.2, I wanted to make use of the automatic tombstone compaction feature, so using CQL3 I issued:
ALTER TABLE versions WITH compaction = {'class' : 'SizeTieredCompactionStrategy', 'min_threshold' : 4, 'max_threshold' : 32, 'tombstone_compaction_interval' : 1, 'tombstone_threshold' : '0.1'};
But I still see no trace that would suggest this works – we had 60G of data with TTL=1week pushed a while ago to the test cluster, the majority of it should be expired & compacted away by now. Not sure if it is relevant, but this old data is in one ~60G file + I have a few smaller files with latest data in them.
Looking at JMX: DroppableTombstoneRatio = 0.892076544, which seems to back my theory.
Am I doing something wrong, or am I expecting the wrong thing?
Thanks,
Tamas
RE: Automatic tombstone compaction
Posted by ta...@thomsonreuters.com.
Hi,
I ran upgradesstables as part of the Cassandra upgrade, before issuing the CQL alter command.
According to the docs, SizeTieredCompactionStrategy is fine (that is what I used, and plan on continue using), and automatic tombstone compaction is available for it:
http://www.datastax.com/documentation/cassandra/1.2/webhelp/index.html#cassandra/operations/ops_about_config_compact_c.html
I just had to include the 'class' in the alter statement, otherwise it would not accept my command.
Is that not right?
Thanks,
Tamas
From: Haithem Jarraya [mailto:a-hjarraya@expedia.com]
Sent: 21. august 2013 16:24
To: user@cassandra.apache.org
Subject: Re: Automatic tombstone compaction
Hi,
do you mean LeveledCompactionStrategy?
Also you will need to run nodetool upgradesstables [keyspace][cf_name] after changing the compaction strategy.
Thanks,
Haithem Jarraya
On 21 Aug 2013, at 15:15, tamas.foldesi@thomsonreuters.com<ma...@thomsonreuters.com> wrote:
Hi,
After upgrading from 1.0 to 1.2, I wanted to make use of the automatic tombstone compaction feature, so using CQL3 I issued:
ALTER TABLE versions WITH compaction = {'class' : 'SizeTieredCompactionStrategy', 'min_threshold' : 4, 'max_threshold' : 32, 'tombstone_compaction_interval' : 1, 'tombstone_threshold' : '0.1'};
But I still see no trace that would suggest this works - we had 60G of data with TTL=1week pushed a while ago to the test cluster, the majority of it should be expired & compacted away by now. Not sure if it is relevant, but this old data is in one ~60G file + I have a few smaller files with latest data in them.
Looking at JMX: DroppableTombstoneRatio = 0.892076544, which seems to back my theory.
Am I doing something wrong, or am I expecting the wrong thing?
Thanks,
Tamas
Re: Automatic tombstone compaction
Posted by Haithem Jarraya <a-...@expedia.com>.
Hi,
do you mean LeveledCompactionStrategy?
Also you will need to run nodetool upgradesstables [keyspace][cf_name] after changing the compaction strategy.
Thanks,
Haithem Jarraya
On 21 Aug 2013, at 15:15, tamas.foldesi@thomsonreuters.com<ma...@thomsonreuters.com> wrote:
Hi,
After upgrading from 1.0 to 1.2, I wanted to make use of the automatic tombstone compaction feature, so using CQL3 I issued:
ALTER TABLE versions WITH compaction = {'class' : 'SizeTieredCompactionStrategy', 'min_threshold' : 4, 'max_threshold' : 32, 'tombstone_compaction_interval' : 1, 'tombstone_threshold' : '0.1'};
But I still see no trace that would suggest this works – we had 60G of data with TTL=1week pushed a while ago to the test cluster, the majority of it should be expired & compacted away by now. Not sure if it is relevant, but this old data is in one ~60G file + I have a few smaller files with latest data in them.
Looking at JMX: DroppableTombstoneRatio = 0.892076544, which seems to back my theory.
Am I doing something wrong, or am I expecting the wrong thing?
Thanks,
Tamas