[jira] [Updated] (HBASE-15389) Write out multiple files when compaction

["Duo Zhang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Duo Zhang updated HBASE-15389:
------------------------------
    Attachment: HBASE-15389-uc.patch

I have to attend a training course today and it is Friday in China so I upload the unfinished patch here.

May still contain some bugs since the testcase has not been done yet.

I add a new StoreEngine and Compactor implementation. When the filesToCompact is ready, I calculate the boundaries and write out a file for each window.

The output files have same seqId so I add maxTimestamp to the sorting source just after seqId in StoreFile.Comparators.SEQ_ID. Two thought here:
1. Assign different seqIds for the output files. If we have less output files than input files, it could work. But this can not be satisfied every time so theoretically it is possible that we do not have enough different seqIds to assign. Correct me if I'm wrong.
2. Introduce a new comparator instead of modifing SEQ_ID directly. This could be done, no problem, but the SEQ_ID comparator is also used outside StoreFileManager so we need to find a place to store the comparator which should be used and remove the static reference. So here I choose the simple way first. Can change later if needed.

[~davelatham] [~claraxiong] PTAL. Thanks very much.

> Write out multiple files when compaction
> ----------------------------------------
>
>                 Key: HBASE-15389
>                 URL: https://issues.apache.org/jira/browse/HBASE-15389
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Compaction
>            Reporter: Duo Zhang
>         Attachments: HBASE-15389-uc.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)