[GitHub] cassandra issue #147: [wip] CASSANDRA-10496

[michaelsembwever <gi...@git.apache.org>]

Github user michaelsembwever commented on the issue:

    https://github.com/apache/cassandra/pull/147
  
    @iksaif,
     continuing the conversation from [CASSANDRA-10496](https://issues.apache.org/jira/browse/CASSANDRA-10496).
    
    > What do you mean by "changing locations isn't supported" ?
    
    `switchCompactionLocation(..)` is a public method and can be called from other places, when rows need to now be written to a new sstable in a new location. That's why [here](https://github.com/thelastpickle/cassandra/commit/a34a72391bb2847b3bd6ed93b4306199ddf3a991#diff-19359d40a9c932efdebc62a067ed4390R40) i paired writers by their location, so if a location changes the writer can also change.
    
    > Currently it will create up to "minThreshold" sstables
    
    I'm not too sure I get that. The code you referenced is about which bucket is up for compaction, not how many writers to use during a bucket's compaction.
    Marcus' idea was to only create two sstables per bucket, one that contains all the rows that belong in the bucket, and another for old data that's been streamed in late. Therefore SplittingTimeWindowCompactionWriter. writersByBounds` should be a fixed array of size 2. The splitting approach as describe in  `SplittingTimeWindowCompactionWriter`'s class apidoc: splitting in half, then half, down to 50Mb; is quite different to the original idea.
    
    > getBuckets() currently use maxTimestamp, , which isn't available (currently) in the compaction task
    
    @krummas ?
    
    > Are you talking about sstables generated before this patch ?
    
    Yes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org