Evict Tombstones with STCS

[Anuj Wadehra <an...@yahoo.co.in>]

Hi,
We are using C* 2.0.x . What options are available if disk space is too full to do compaction on huge sstables formed by STCS (created around long ago but not getting compacted due to min_compaction_threshold being 4).
We suspect that huge space will be released when 2 largest sstables get compacted together such that tombstone eviction is possible. But there is not enough space for compacting them together assuming that compaction would need at least free disk=size of sstable1 + size of sstable 2 ??
I read STCS code and if no sstables are available for compactions, it should pick individual sstable for compaction. But somehow, huge sstables are not participating in individual compaction.. is it due to default 20% tombstone threshold?? And if it so, forceUserdefinedcompaction or setting unchecked_tombstone_compactions to true wont help either as tombstones are less than 20% and not much disk would be recovered.
It is not possible to add additional disks too.
We see huge difference in disk utilization of different nodes. May be some nodes were able to get away with tombstones while others didnt manage to evict tombstones.

Would be good to know more alternatives from community.

ThanksAnuj






Sent from Yahoo Mail on Android

Re: Evict Tombstones with STCS

[Alain RODRIGUEZ <ar...@gmail.com>]

Hi, I think you got it, this is probably the way to go:


> And if it so, forceUserdefinedcompaction or setting unchecked_tombstone_compactions
> to true wont help either as tombstones are less than 20% and not much disk
> would be recovered.


But if you have less than 20% tombstones in there I am not sure that
compacting those tables will help anyhow. It's probably time to add
capacity (nodes, as you can't add disks).

If you still want to give it a try, set the unchecked_tombstone_compactions
to true and the ratio to10% and see how it goes.

An other option to have better control on disk size and being able to use
up to 70% - 80% of the disk, then LCS might be a good option, if it fits
with your use case.

C*heers,

-----------------------
Alain Rodriguez - alain@thelastpickle.com
France

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com

2016-05-28 20:24 GMT+01:00 Anuj Wadehra <an...@yahoo.co.in>:

> Hi,
>
> We are using C* 2.0.x . What options are available if disk space is too
> full to do compaction on huge sstables formed by STCS (created around long
> ago but not getting compacted due to min_compaction_threshold being 4).
>
> We suspect that huge space will be released when 2 largest sstables get
> compacted together such that tombstone eviction is possible. But there is
> not enough space for compacting them together assuming that compaction
> would need at least free disk=size of sstable1 + size of sstable 2 ??
>
> I read STCS code and if no sstables are available for compactions, it
> should pick individual sstable for compaction. But somehow, huge sstables
> are not participating in individual compaction.. is it due to default 20%
> tombstone threshold?? And if it so, forceUserdefinedcompaction or setting
> unchecked_tombstone_compactions to true wont help either as tombstones are
> less than 20% and not much disk would be recovered.
>
> It is not possible to add additional disks too.
>
> We see huge difference in disk utilization of different nodes. May be some
> nodes were able to get away with tombstones while others didnt manage to
> evict tombstones.
>
>
> Would be good to know more alternatives from community.
>
>
> Thanks
> Anuj
>
>
>
>
>
>
>
> Sent from Yahoo Mail on Android
> <https://overview.mail.yahoo.com/mobile/?.src=Android>
>