Renaming ".snapshot" directory used by HBase snapshots

[Ted Yu <yu...@gmail.com>]

Hi,
In the first half of this email, let me summarize our findings:

Yesterday afternoon Huned ad I discovered an issue while playing with HBase
Snapshots on top of Hadoop's Snapshot branch (
http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-2802/).

HDFS (built from HDFS-2802 branch) doesn't allow paths with .snapshot as a
component while HBase tries to create paths with .snapshot as a component.
This leads to issues in HBase, and one of HDFS or HBase needs to give up
the .snapshot reserved keyword. HBase released Snapshots feature in 0.94.6
(quite recently) and it may not be too late to change HBase to use a
different path component in an upcoming new release.

In HBase these path names are not user visible. If there is a deployment of
0.94.6, one could provide a migration tool that renames .snapshot to
.hbase-snapshot or something to be able to move to the Snapshot release of
Hadoop. On the other hand, .snapshot in HDFS is a user visible name and is
a convention that is used by many file systems. It's a matter of
familiarity with such path names that would help users in using HDFS
snapshots.

You can find sample output from hdfs command here:
http://pastebin.com/bBqR4Fvr

You can find discussion on dev@ mailing list here:
http://search-hadoop.com/m/F6jwFcXMs51

---------

After the above discussion, we chose a relatively simple approach. Here is
excerpt from release notes of HBASE-8352 w.r.t. upgrading to 0.94.7 (and
newer) releases:

In order to avoid name collision with hdfs snapshot, ".snapshot" under
HBase root directory must be renamed as ".hbase-snapshot" before deployment
of 0.94.7 or later release is started.

Don't do rolling upgrade between 0.94.6.1 and 0.94.7 if there're on-going
snapshots activities.
Sequence of actions should be:

1. make sure all on-going snapshots activities have completed
2. shutdown all the masters
3. rename ".snapshot" under HBase root directory as ".hbase-snapshot"
4. upgrade the masters to 0.94.7+ release (You now can perform clone or
restore, but not taking snapshots)
5. upgrade the region servers
(You should not take snapshots until all the regions server are aligned to
0.94.7+ release)

If you take a snapshot during the RS upgrades nothing critical happens. The
".snapshot" folder would appear again and the snapshot will fail since half
of the data will be in ".hbase-snapshot" and the other half in ".snapshot"

--------

This email serves as notice while at the same time, feedback from community
is welcome.

Cheers