[jira] [Commented] (HBASE-5547) Don't delete HFiles when in "backup mode"

["Jesse Yates (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238965#comment-13238965 ] 

Jesse Yates commented on HBASE-5547:
------------------------------------

Thoughts on how long we should keep around files? Indefinitely? The latter seems a bit excessive, especially if a 'backup mode' ensures you run every X minutes (and exports to another cluster, moves the files to another backup directory). 'Cleanup' in implies you want to remove the file when no one care about the hfiles anymore - thinking maybe a periodic chore on the rs?

With snapshots, I was expecting to add an file reference feature - essentially doing impl hardlinks for files we care about keeping around. Was thinking we could add a CP hook and impl that would let you add a checks (config based?) for if you want to keep a reference around for the file being cleaned up. In the backup situation, you would have a timer or (maybe check for a backup completed file/meta row) and see if you had elapsed that time or not; if not, you would add a reference, if so, do nothing and let the file get cleaned up.
                
> Don't delete HFiles when in "backup mode"
> -----------------------------------------
>
>                 Key: HBASE-5547
>                 URL: https://issues.apache.org/jira/browse/HBASE-5547
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Lars Hofhansl
>
> This came up in a discussion I had with Stack.
> It would be nice if HBase could be notified that a backup is in progress (via a znode for example) and in that case either:
> 1. rename HFiles to be delete to <file>.bck
> 2. rename the HFiles into a special directory
> 3. rename them to a general trash directory (which would not need to be tied to backup mode).
> That way it should be able to get a consistent backup based on HFiles (HDFS snapshots or hard links would be better options here, but we do not have those).
> #1 makes cleanup a bit harder.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira