[jira] [Updated] (CASSANDRA-4341) Small SSTable Segments Can Hurt Leveling Process

["Jonathan Ellis (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CASSANDRA-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-4341:
--------------------------------------

    Attachment: 4341.txt

Patch attached with the following approach:

{noformat}
.           // We treat L0 compactions specially:
            // 1a. add sstables to the candidate set until we have at least maxSSTableSizeInMB
            // 1b. prefer choosing older sstables as candidates, to newer ones
            // 1c. any L0 sstables that overlap a candidate, will also become candidates
            // 2. At most MAX_COMPACTING_L0 sstables will be compacted at once
            // 3. If total candidate size is less than maxSSTableSizeInMB, we won't bother compacting with L1,
            //    and the result of the compaction will stay in L0 instead of being promoted (see promote())
{noformat}
                
> Small SSTable Segments Can Hurt Leveling Process
> ------------------------------------------------
>
>                 Key: CASSANDRA-4341
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4341
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: Benjamin Coverston
>              Labels: compaction
>             Fix For: 1.1.2
>
>         Attachments: 4341.txt
>
>
> This concerns:
> static int MAX_COMPACTING_L0 = 32;
> Repair can create very small SSTable segments. We should consider moving to a threshold that takes into account the size of the files brought into compaction rather than the number of files for this and similar situations. Bringing the small files from L0 to L1 magnifies the issue.
> If there are too many very small files in L0 perhaps even an intermediate compaction would even reduce the magnifying effect of a L0 to L1 compaction.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira