TWCS on partitions spanning multiple time windows

[Hannu Kröger <hk...@gmail.com>]

Hi,

I have been reading a bit about TWCS to understand how it functions.

Current assumption: TWCS uses same tombstone checks as any other compaction
strategy to make sure that it doesn’t remove tombstones unless it is safe
to do so.

Scenario 1:

So let’s assume that I have a tables like this:

CREATE TABLE twcs.twcs (
    user_id int,
    id int,
    value int,
    text_value text,
    PRIMARY KEY (user_id, id, value)
)

I insert data for multiple users but also multiple events per user with TTL
of 10 days and have time window set to 1 day.

Basically when 10 days are up the first sstable contains just TTL'd data.
However TWCS cannot just drop sstables because same partition exists in
sstables of other windows. Otherwise it wouldn’t be safe, right?

Scenario 2:

Table is as follows

CREATE TABLE twcs.twcs2 (
    user_id int,
    day int,
    id int,
    value int,
    text_value text,
    PRIMARY KEY ((user_id, day), id, value)
)

I insert data with TTL of 10 days and have time window set to 2 days.

Basically when 10 days are up the first sstable contains just TTL:d data.
In this case TWCS can drop the whole sstable because the whole partition is
in the same time window and same sstable. Correct?

If we for some reason have time window and partition time buckets
misaligned, e.g. time window is 25 hours and time bucket is 24 hours, then
we end up in situation where we will actually never get rid of all
tombstones because same partition data will be across multiple time windows
which won’t be compacted together. So we would be in trouble, right?

Did I get it right?

Hannu

Re: TWCS on partitions spanning multiple time windows

[Jeff Jirsa <jj...@gmail.com>]

Inline

-- 
Jeff Jirsa


> On Dec 14, 2017, at 8:37 AM, Hannu Kröger <hk...@gmail.com> wrote:
> 
> Hi,
> 
> I have been reading a bit about TWCS to understand how it functions.
> 
> Current assumption: TWCS uses same tombstone checks as any other compaction strategy to make sure that it doesn’t remove tombstones unless it is safe to do so.
> 

Correct, though there’s a flag in 3.11.2/trunk to let you tell Cassandra you’re smart enough to ask it to do unsafe things

> Scenario 1:
> 
> So let’s assume that I have a tables like this:
> 
> CREATE TABLE twcs.twcs (
>     user_id int,
>     id int,
>     value int,
>     text_value text,
>     PRIMARY KEY (user_id, id, value)
> )
> 
> I insert data for multiple users but also multiple events per user with TTL of 10 days and have time window set to 1 day.
> 
> Basically when 10 days are up the first sstable contains just TTL'd data. However TWCS cannot just drop sstables because same partition exists in sstables of other windows. Otherwise it wouldn’t be safe, right?
> 

It won’t drop an sstable that overlaps with a live sstable, that doesn’t mean it can’t purge out most of the expired data.

Overlaps are almost always caused by read repair, not by partitions spanning windows, unless you’re writing some data with ttls and some without, writing with USING TIMESTAMP to write data that doesn’t match “now”


> Scenario 2:
> 
> Table is as follows
> 
> CREATE TABLE twcs.twcs2 (
>     user_id int,
>     day int,
>     id int,
>     value int,
>     text_value text,
>     PRIMARY KEY ((user_id, day), id, value)
> )
> 
> I insert data with TTL of 10 days and have time window set to 2 days.
> 
> Basically when 10 days are up the first sstable contains just TTL:d data. In this case TWCS can drop the whole sstable because the whole partition is in the same time window and same sstable. Correct?

This model is probably better not because of the drop behavior, but because reads won’t have to merge literally every sstable on the disk

Again, the full drop behavior looks for sstables with live data and overlapping time stamp at a full sstable level, it’s not per partition. But limiting partitions to one window will make the partitions easier to drop with the tombstone compaction.

> 
> If we for some reason have time window and partition time buckets misaligned, e.g. time window is 25 hours and time bucket is 24 hours, then we end up in situation where we will actually never get rid of all tombstones because same partition data will be across multiple time windows which won’t be compacted together. So we would be in trouble, right?
> 

You can still turn on tombstone compaction and purge most of the data, I think (without reading the code again) that it’ll  drop fully after 2 windows assuming there’s no read repair polluting the new window 

> Did I get it right?
> 
> Hannu