[jira] [Comment Edited] (HBASE-18128) compaction marker could be skipped

["Jingyun Tian (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-18128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036458#comment-16036458 ] 

Jingyun Tian edited comment on HBASE-18128 at 6/5/17 1:54 AM:
--------------------------------------------------------------

[~anoop.hbase] Thanks for comment, I think solve this in a generic way is a good idea, I will update the patch this week.
Only compaction marker need this treatment right now. Because when you replay a recovered edit, server will check if it is compaction marker, if it is, server will delete the old files before opening region. Currently there are no other meta cells have special treatment.




was (Author: tianjingyun):
[~anoop.hbase] Thanks for comment, I think solve this in a generic way is a good idea, I will update the patch this work.
Only compaction marker need this treatment right now. Because when you replay a recovered edit, server will check if it is compaction marker, if it is, server will delete the old files before opening region. Currently there are no other meta cells have special treatment.



> compaction marker could be skipped 
> -----------------------------------
>
>                 Key: HBASE-18128
>                 URL: https://issues.apache.org/jira/browse/HBASE-18128
>             Project: HBase
>          Issue Type: Improvement
>          Components: Compaction, regionserver
>            Reporter: Jingyun Tian
>            Assignee: Jingyun Tian
>         Attachments: HBASE-18128.patch
>
>
> The sequence for a compaction are as follows:
> 1. Compaction writes new files under region/.tmp directory (compaction output)
> 2. Compaction atomically moves the temporary file under region directory
> 3. Compaction appends a WAL edit containing the compaction input and output files. Forces sync on WAL.
> 4. Compaction deletes the input files from the region directory.
> But if a flush happened between 3 and 4, then the regionserver crushed. The compaction marker will be skipped when splitting log because the sequence id of compaction marker is smaller than lastFlushedSequenceId.
> {code}
>         if (lastFlushedSequenceId >= entry.getKey().getLogSeqNum()) {
>           editsSkipped++;
>           continue;
>         }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)