TWCS decommission and cleanups

[Benjamin Heiskell <bh...@datto.com>]

Hello all,

How does TWCS work when decommissioning a node? Does the data distribute
across the other nodes to the current time window's sstable (like
read-repairs)? Or will it compact into the sstables for the prior windows?
If that's how it works, how does it decide what sstable to compact with?
E.g.., if I ran a major compaction on one node (pushing everything into a
few sstables), then decommissioned it, what would you expect to see on
other nodes? Would the data distribute across all sstables based on the
row's timestamp?

I'm also a little unclear on how cleanups work. If I run a nodetool
cleanup, will that actually re-write the old, but not completely expired
sstables of prior time windows? Or are those sstables truly immutable?

I've read through the documentation (
http://cassandra.apache.org/doc/latest/operating/compaction.html#time-window-compactionstrategy)
and a few other resources (
http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html), but I still
don't have a clear understanding of how it works.

Any clarification would be greatly appreciated. Thanks!
Ben

Re: TWCS decommission and cleanups

[Jeff Jirsa <jj...@gmail.com>]

It does the right thing - sstables sent to other nodes maintain their min/max timestamps so they’ll go to the right buckets

The bucket is selected using the timestamp of the newest cell in the sstable

If you run a major compaction, you would undo the same bucketing 

Cleanup works by compacting an sstable with itself and excluding any partition no longer owned - it makes a new sstable, the original is deleted when the new one is finished


-- 
Jeff Jirsa
 

> On Nov 15, 2017, at 7:57 AM, Benjamin Heiskell <bh...@datto.com> wrote:
> 
> Hello all,
> 
> How does TWCS work when decommissioning a node? Does the data distribute across the other nodes to the current time window's sstable (like read-repairs)? Or will it compact into the sstables for the prior windows? If that's how it works, how does it decide what sstable to compact with? E.g.., if I ran a major compaction on one node (pushing everything into a few sstables), then decommissioned it, what would you expect to see on other nodes? Would the data distribute across all sstables based on the row's timestamp?
> 
> I'm also a little unclear on how cleanups work. If I run a nodetool cleanup, will that actually re-write the old, but not completely expired sstables of prior time windows? Or are those sstables truly immutable?
> 
> I've read through the documentation (http://cassandra.apache.org/doc/latest/operating/compaction.html#time-window-compactionstrategy) and a few other resources (http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html), but I still don't have a clear understanding of how it works.
> 
> Any clarification would be greatly appreciated. Thanks!
> Ben