[jira] [Commented] (HBASE-6233) [brainstorm] snapshots: hardlink alternatives

["Matteo Bertozzi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-6233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396133#comment-13396133 ] 

Matteo Bertozzi commented on HBASE-6233:
----------------------------------------

Taking a snapshot means keeping references to hfiles currently present in the table. 
Unfortunately during compaction and split these file can be removed from the original location.

One solution can be:
 * move hfiles to .snapshot/files directory, during the "take snapshot" operation
 * create symlinks to .snapshot/files in the table folder.
 * create the snapshot reference in .snapshot/name/files

This allows to restore snapshot easily by creating a symlink. 
The hbase code will not change since compaction can still call fs.delete() and ends up deleting just the symlink

One problem is that during the fs.rename() + fs.createSymlink() the filename is not available and if DFSInputStream.callGetBlockLocations() is called you end up with a FileNotFoundException.

_But this means that after a snapshot the files in the table folder are symlink, so you'll see both normal files + symlink during the table life._
                
> [brainstorm] snapshots: hardlink alternatives
> ---------------------------------------------
>
>                 Key: HBASE-6233
>                 URL: https://issues.apache.org/jira/browse/HBASE-6233
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: Matteo Bertozzi
>            Assignee: Matteo Bertozzi
>
> Discussion ticket around snapshots and hardlink alternatives.
> (See the HDFS-3370 discussion about hardlink and implementation problems)
> (taking for a moment WAL out of the discussion and focusing on hfiles)
> With hardlinks available taking snapshot will be fairly easy:
> * (hfiles are immutable)
> * hardlink to .snapshot/name to take snapshot
> * hardlink from .snapshot/name to restore the snapshot
> * No code change needed (on fs.delete() only one reference is deleted)
> but we don't have hardlinks, what are the alternatives?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira