[jira] [Commented] (CASSANDRA-10490) DTCS historic compaction, possibly with major compaction

["Wei Deng (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094064#comment-15094064 ] 

Wei Deng commented on CASSANDRA-10490:
--------------------------------------

A related proposal was suggested by [~Bj0rn] in CASSANDRA-8361: "major compaction for DTCS should put data perfectly in windows rather than everything in one SSTable."

Since that JIRA got closed as duplicate, I'm pasting it here to continue the discussion.

> DTCS historic compaction, possibly with major compaction
> --------------------------------------------------------
>
>                 Key: CASSANDRA-10490
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10490
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jonathan Shook
>              Labels: compaction, triage
>             Fix For: 2.2.x, 3.x
>
>
> Presently, it's simply painful to run a major compaction with DTCS. It doesn't really serve a useful purpose. Instead, a DTCS major compaction should allow for a DTCS-style compaction to go back before max_sstable_age_days. We can call this a historic compaction, for lack of a better term.
> Such a compaction should not take precedence over normal compaction work, but should be considered a background task. By default there should be a cap on the number of these tasks running. It would be nice to have a separate "max_historic_compaction_tasks" and possibly a "max_historic_compaction_throughput" in the compaction settings to allow for separate throttles on this. I would set these at 1 and 20% of the usual compaction throughput if they aren't set explicitly.
> It may also be desirable to allow historic compaction to run apart from running a major compaction, and to simply disable major compaction altogether for DTCS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)