You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Anuj Wadehra <an...@yahoo.co.in> on 2015/04/14 03:23:29 UTC

Drawbacks of Major Compaction now that Automatic Tombstone Compaction Exists

Recently we faced an issue where every repair operation caused addition of hundreds of sstables (CASSANDRA-9146). In order to bring situation under control and make sure reads are not impacted, we were left with no option but to run major compaction to ensure that thousands of tiny sstables are compacted.


Queries:
Does major compaction has any drawback after automatic tombstone compaction got implemented in 1.2 via tombstone_threshold sub-property(CASSANDRA-3442)? 
I understand that the huge SSTable created after major compaction wont be compacted with new data any time soon but is that a problem if purged data is removed via automatic tombstone compaction? If we major compaction results in a huge file say 500GB, what are the drawbacks of it?

If one big sstable is a problem, is there any way of solving the problem? We tried running sstablesplit after major compaction to split the big sstable but as new sstables were of same size they are again compacted into single huge table once Cassandra was started after executing sstablesplit.



Thanks

Anuj Wadehra

Re: Drawbacks of Major Compaction now that Automatic Tombstone Compaction Exists

Posted by Paulo Motta <pa...@gmail.com>.
This list is for discussion about cassandra development, not some kind of
fallback when you don't get responses from the user list.

Please be patient and keep this discussion to the user list, or create a
JIRA ticket if you think you're facing a bug/issue.

2015-04-13 22:44 GMT-03:00 Anuj Wadehra <an...@yahoo.co.in>:

> I havent got much response regarding this on user list..so posting it on
> dev list too..
>
>
> Thanks
>
> Anuj Wadehra
>
> Sent from Yahoo Mail on Android
>
> From:"Anuj Wadehra" <an...@yahoo.co.in>
> Date:Tue, 14 Apr, 2015 at 7:05 am
> Subject:Drawbacks of Major Compaction now that Automatic Tombstone
> Compaction Exists
>
> Recently we faced an issue where every repair operation caused addition of
> hundreds of sstables (CASSANDRA-9146). In order to bring situation under
> control and make sure reads are not impacted, we were left with no option
> but to run major compaction to ensure that thousands of tiny sstables are
> compacted.
>
>
> Queries:
> Does major compaction has any drawback after automatic tombstone
> compaction got implemented in 1.2 via tombstone_threshold
> sub-property(CASSANDRA-3442)?
> I understand that the huge SSTable created after major compaction wont be
> compacted with new data any time soon but is that a problem if purged data
> is removed via automatic tombstone compaction? If we major compaction
> results in a huge file say 500GB, what are the drawbacks of it?
>
> If one big sstable is a problem, is there any way of solving the problem?
> We tried running sstablesplit after major compaction to split the big
> sstable but as new sstables were of same size they are again compacted into
> single huge table once Cassandra was started after executing sstablesplit.
>
>
>
> Thanks
>
> Anuj Wadehra
>
>

Re: Drawbacks of Major Compaction now that Automatic Tombstone Compaction Exists

Posted by Anuj Wadehra <an...@yahoo.co.in>.
I havent got much response regarding this on user list..so posting it on dev list too..


Thanks

Anuj Wadehra

Sent from Yahoo Mail on Android

From:"Anuj Wadehra" <an...@yahoo.co.in>
Date:Tue, 14 Apr, 2015 at 7:05 am
Subject:Drawbacks of Major Compaction now that Automatic Tombstone Compaction Exists

Recently we faced an issue where every repair operation caused addition of hundreds of sstables (CASSANDRA-9146). In order to bring situation under control and make sure reads are not impacted, we were left with no option but to run major compaction to ensure that thousands of tiny sstables are compacted.


Queries:
Does major compaction has any drawback after automatic tombstone compaction got implemented in 1.2 via tombstone_threshold sub-property(CASSANDRA-3442)? 
I understand that the huge SSTable created after major compaction wont be compacted with new data any time soon but is that a problem if purged data is removed via automatic tombstone compaction? If we major compaction results in a huge file say 500GB, what are the drawbacks of it?

If one big sstable is a problem, is there any way of solving the problem? We tried running sstablesplit after major compaction to split the big sstable but as new sstables were of same size they are again compacted into single huge table once Cassandra was started after executing sstablesplit.



Thanks

Anuj Wadehra