Questions on Leveled Compaction sizing and compaction corner cases

[DuyHai Doan <do...@gmail.com>]

I was reading this
http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra and
need some confirmation:

A  Sizing


"*Each level is ten times as large as the previous*"

In the comments:

At October 14, 2011 at 12:33
am<http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra#comment-18817><http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra#comment-18817>Jonathan
said: "*L1
gets 50MB (~10 sstables of data), L2 gets 500MB/100 sstables, L3 gets 5GB*"


At January 22, 2013 at 7:51
pm<http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra#comment-196897>he
said "*Remember
that within a level, data is guaranteed not to overlap across sstables*, or
put another way, *a given row will be in at most one sstable*"

At February 11, 2013 at 7:32
am<http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra#comment-196901>he
said: "A compaction will run whenever there is more data in Level N >
0
than desired *(sstable_size_in_mb * 10**level)*"

At February 22, 2013 at 8:20
am<http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra#comment-196904>he
said: "Leveled compaction restricts sstable size to 5MB or a single
row, *whichever
is larger*"

 If I put all the info together:

1) sizeOf(Ln+1) = 10 * sizeOf(Ln) = sstable_size_in_mb * 10 ^ (n+1)
2) the size of a sstable is limited to *sstable_size_in_mb *by default* or
more if a partition is large enouth to exceed sstable_size_in_mb *
3) because of point 2), the equality ssTableCount(Ln+1) = ssTableCount(Ln)
does not always hold

Is it correct so far ?

B Compaction corner cases

Now, one of the biggest selling point of LCS is its frequent compaction and
that 90% of the read only touch 1 SSTable. Fine.

 Let's suppose we have data in 4 levels (taking the new default 160M
for *sstable_size_in_mb)
* : L0,
L1 (1.6Gb),
L2 (16Gb),
L3 (160Gb partially filled).

 For some reason there was a burst in write in the application so data gets
compacted up to L3. Now that the write/update workload is back to normal,
compaction never goes beyond L2.

 In this case, all my old/deleted/obsolete data in L3 will never be
compacted isn't it ? Or only at the next burst in write right ?

Regards

 Duy Hai DOAN