[jira] [Commented] (CASSANDRA-5137) Make sure SSTables left over from compaction get deleted and logged

["Jonathan Ellis (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13548749#comment-13548749 ] 

Jonathan Ellis commented on CASSANDRA-5137:
-------------------------------------------

Hmm.

This patch is correct as far as it goes but I think the existing assumption is broken: that if we have any sstable with ancestor X, then X is safe to delete.

Specifically, LCS will create multiple sstables from a given set of ancestors, so unless we know that we finished the compaction (and finished writing all the resulting descendant sstables), we could lose data if we delete the ancestors themselves.

One possible fix:

# Add a flag to SSTM for "this was the final sstable in the compaction"
# When we scan sstables, we can delete ancestors if we find that marker in any of the descendants
# Otherwise, we should delete the *descendants* and leave the ancestors alone (so we don't doublecount data for counters)
                
> Make sure SSTables left over from compaction get deleted and logged
> -------------------------------------------------------------------
>
>                 Key: CASSANDRA-5137
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5137
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.3
>            Reporter: Yuki Morishita
>            Assignee: Yuki Morishita
>            Priority: Minor
>             Fix For: 1.1.9, 1.2.1
>
>         Attachments: 5137-1.1.txt
>
>
> When opening ColumnFamily, cassandra checks SSTable files' ancestors and skips loading already compacted ones. Those files are expected to be deleted, but currently that never happens.
> Also, there is no indication of skipping loading file in the log, so it is confusing especially doing upgradesstables.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira