[jira] [Commented] (HBASE-14141) HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backup tables

["Vladimir Rodionov (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730155#comment-15730155 ] 

Vladimir Rodionov commented on HBASE-14141:
-------------------------------------------

Overall high level design:

Backup:

# New MR job (WAL2HFile), which runs in a first phase of incremental backup.
## WAL2HFile first get list of tables registered in hbase:backup, then for every table from this list generates HFiles (one per region) 
# In a second phase DistCp moves these files to a backup destination

Restore (incremental)

# No need to WALPlayer job
# For every table from restore list, HFileSplitter job will collect all HFile(s), split them into new region boundaries.
# LoadHFileIncremental loads newly created files.

Security: no issues we run under hbase user.

*We no longer keep WAL files in backup destination - only HFiles*. This significantly reduce storage requirements for backup data.  


> HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backup tables
> --------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14141
>                 URL: https://issues.apache.org/jira/browse/HBASE-14141
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 2.0.0
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>              Labels: backup
>             Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)