You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Aditya Kishore (JIRA)" <ji...@apache.org> on 2012/06/19 03:56:42 UTC
[jira] [Created] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Aditya Kishore created HBASE-6236:
-------------------------------------
Summary: Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
Key: HBASE-6236
URL: https://issues.apache.org/jira/browse/HBASE-6236
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Attachment: HBASE-6236_trunk.patch
@Ted:
I was of two minds about naming this new option. This new patch has the changed name.
Submitting only for trunk this time.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399543#comment-13399543 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
Patch integrated to trunk.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399690#comment-13399690 ]
Jonathan Hsieh commented on HBASE-6236:
---------------------------------------
@Aditya
I only looked at the 21/Jun/12 22:49 version -- assuming others have a similar problem.
There is a possible array out of bounds exception here. Can we check to make sure we are not out of bounds and provide an appropriate warning message and bail out if we are?
{code}
conf.set(HConstants.HBASE_DIR, path);
conf.set("fs.defaultFS", conf.get(HConstants.HBASE_DIR));
conf.set("fs.default.name", conf.get(HConstants.HBASE_DIR));
+ } else if (cmd.equals("-sidelineDir")) {
+ i++;
+ fsck.setSidelineDir(args[i]);
} else if (cmd.equals("-fixHoles")) {
fixHoles = true;
} else if (cmd.equals("-fix")) {
{code}
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Attachment: HBASE-6236_0.94.patch
HBASE-6236_trunk.patch
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Attachment: (was: HBASE-6236_0.94.patch)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399634#comment-13399634 ]
Jimmy Xiang commented on HBASE-6236:
------------------------------------
@Aditya, good catch. Please make sure they are the same.
Can you use /hbase/.hbck as the default?
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399095#comment-13399095 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
@Aditya:
Hadoop QA is not functioning. Please run test suite locally.
@Jonathan H, @Jimmy:
Can you take a look at the patch ?
Thanks
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399444#comment-13399444 ]
Hadoop QA commented on HBASE-6236:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12532994/HBASE-6236_trunk.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.security.access.TestAccessController
org.apache.hadoop.hbase.util.TestFSUtils
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2222//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2222//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2222//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2222//console
This message is automatically generated.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399659#comment-13399659 ]
Aditya Kishore commented on HBASE-6236:
---------------------------------------
And it might be a good idea to add ".hbck" to the list "org.apache.hadoop.hbase.HConstants.HBASE_NON_USER_TABLE_DIRS"
along with ".logs", ".oldlogs", ".corrupt", etc.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400035#comment-13400035 ]
Hudson commented on HBASE-6236:
-------------------------------
Integrated in HBase-0.94-security #37 (See [https://builds.apache.org/job/HBase-0.94-security/37/])
HBASE-6236 Addendum adds -sidelineDir option to hbck (Aditya) (Revision 1353066)
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353064)
HBASE-6236 revert upon Jimmy's request. (Revision 1353014)
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353012)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399201#comment-13399201 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
All Apache build servers are offline.
Please list the failing tests individually so that people can help verify.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396523#comment-13396523 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
{code}
+ return new Path(hbaseDir, ".hbcktmp" + "-" + startMillis);
{code}
Introduce some constant and make the above configurable ?
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396531#comment-13396531 ]
Aditya Kishore commented on HBASE-6236:
---------------------------------------
@[~zhihyu@ebaysf.com]
Agree that it would be a clean to move the prefix to a constant.
However, we would still want to enforce 2 criteria on the backup folder name/location:
1. The folder must be immediate child of HBase base folder to ensure that fs.rename() works, and
2. Prefix starts with a "." to ensure that it does not conflict with or presumed to be a table directory.
Instead of making it configurable, how about print a message at the end notifying the user the location of the backup folder?
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399920#comment-13399920 ]
Hudson commented on HBASE-6236:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #66 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/66/])
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or sim
ilar FS (Aditya) (Revision 1353065)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399197#comment-13399197 ]
Aditya Kishore commented on HBASE-6236:
---------------------------------------
Strangely, the test suit is randomly failing at different places during different runs on my test machine, even without applying the patch :(
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6236:
----------------------------------
Release Note:
-sidelineDir option has been added to both hbck and OfflineMetaRepair.
It allows specification of hdfs directory where offlined data is stored.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399703#comment-13399703 ]
Jonathan Hsieh commented on HBASE-6236:
---------------------------------------
@Aditya Go ahead and fix it in this instance. If there are other instances that have a similar problem, file and fix them in a follow-up jira?
I probably wrote the exemplar which has the bug in the first place. :)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399628#comment-13399628 ]
Aditya Kishore commented on HBASE-6236:
---------------------------------------
@Jimmy
Currently there are two different locations where hbck sidelines the backup data.
First one takes the form '/hbck/hbase-xxxxxx' and is used to sideline regions
{code:title=HBaseFsck.java}
857 private Path getSidelineDir() throws IOException {
858 Path hbaseDir = FSUtils.getRootDir(conf);
859 Path hbckDir = new Path(hbaseDir.getParent(), "hbck");
860 Path backupDir = new Path(hbckDir, hbaseDir.getName() + "-"
861 + startMillis);
862 return backupDir;
863 }
{code}
While the second one takes the form of '/hbase-xxxxx' and is used to sideline the current .META. and \-ROOT\- folders
{code:title=HBaseFsck.java}
960 Path backupDir = new Path(hbaseDir.getParent(), hbaseDir.getName() + "-"
961 + startMillis);
{code}
At lease we need to make them standard.
Secondly, I am not sure how pre-creating "/hbck" will solve this problem in a federated HDFS in all cases. Suppose the root of a cluster is mounted as "/hbase" in ViewFS then the pre created "/hbck" folder is going to be on a different namespace volume and renaming of a file from "/hbase/" to "/hbck/" is going to fail.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399652#comment-13399652 ]
Jimmy Xiang commented on HBASE-6236:
------------------------------------
That's right. I prefer /hbase/.hbck/hbase-xxxxxx as a default, with option to change it.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6236:
----------------------------------
Fix Version/s: 0.94.1
0.96.0
Release Note:
-sidelineDir has been added to offline meta repair tool.
User can use this option to specify HDFS path to backup existing -ROOT- and .META.
Hadoop Flags: Reviewed
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399734#comment-13399734 ]
Hudson commented on HBASE-6236:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #65 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/65/])
HBASE-6236 revert upon Jimmy's request. (Revision 1353013)
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353008)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
tedyu :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399623#comment-13399623 ]
Hudson commented on HBASE-6236:
-------------------------------
Integrated in HBase-TRUNK #3060 (See [https://builds.apache.org/job/HBase-TRUNK/3060/])
HBASE-6236 revert upon Jimmy's request. (Revision 1353013)
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353008)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
tedyu :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399841#comment-13399841 ]
Hudson commented on HBASE-6236:
-------------------------------
Integrated in HBase-0.94 #277 (See [https://builds.apache.org/job/HBase-0.94/277/])
HBASE-6236 Addendum adds -sidelineDir option to hbck (Aditya) (Revision 1353066)
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353064)
Result = FAILURE
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399450#comment-13399450 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
The failed tests are unrelated to the patch.
I ran them and they passed.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399693#comment-13399693 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
I think Aditya was following existing convention:
{code}
} else if (cmd.equals("-base")) {
// update hbase root dir to user-specified base
i++;
String path = args[i];
{code}
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399603#comment-13399603 ]
Hudson commented on HBASE-6236:
-------------------------------
Integrated in HBase-0.94 #272 (See [https://builds.apache.org/job/HBase-0.94/272/])
HBASE-6236 revert upon Jimmy's request. (Revision 1353014)
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353012)
Result = FAILURE
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399562#comment-13399562 ]
Jimmy Xiang commented on HBASE-6236:
------------------------------------
It is a good idea to have sideline dir option. I think we should keep the existing default folder name.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399135#comment-13399135 ]
Aditya Kishore commented on HBASE-6236:
---------------------------------------
I am running the test suit at this moment, will update here once it is done
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399557#comment-13399557 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
Saw Jimmy's comment and reverted the patch.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6236:
----------------------------------
Status: Open (was: Patch Available)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399727#comment-13399727 ]
Jimmy Xiang commented on HBASE-6236:
------------------------------------
Looks good to me. Should we also add this sideline dir option to HBaseFsck?
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396564#comment-13396564 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
Makes sense.
New configuration would increase the possibility of satisfying future use case.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399833#comment-13399833 ]
Hadoop QA commented on HBASE-6236:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533163/6236-94.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2232//console
This message is automatically generated.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6236:
----------------------------------
Attachment: 6236-94.txt
Patch for 0.94
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399551#comment-13399551 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
Integrated to 0.94 as well.
Thanks for the patch, Aditya.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Attachment: (was: HBASE-6236_trunk.patch)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6236:
----------------------------------
Status: Patch Available (was: Open)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399462#comment-13399462 ]
Aditya Kishore commented on HBASE-6236:
---------------------------------------
Even though I am running into some random test case failures, all of these HBCK/HBFsck tests have been passing regularly
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap
org.apache.hadoop.hbase.util.TestHBaseFsck
org.apache.hadoop.hbase.util.TestHBaseFsckComparator
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399099#comment-13399099 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
{code}
+ .println(" -backup <hdfs://> HDFS path to backup existing meta and root.");
{code}
How about giving the new option a name which is more consistent with its usage, such as -sidelineDir ?
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396526#comment-13396526 ]
Aditya Kishore commented on HBASE-6236:
---------------------------------------
@Hadoop QA
Build seems to have attempted to apply 0.94 patch to trunk
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu resolved HBASE-6236.
-----------------------------------
Resolution: Fixed
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6236:
----------------------------------
Attachment: 6236_trunk.patch
Re-attaching.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399003#comment-13399003 ]
Hadoop QA commented on HBASE-6236:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12532967/HBASE-6236_94.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2217//console
This message is automatically generated.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Attachment: HBASE-6236_trunk.patch
# "-sidelineDir" option to HBaseFsck too.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399646#comment-13399646 ]
Aditya Kishore commented on HBASE-6236:
---------------------------------------
@Jimmy
To be sure, you want the folders to take the form of "/hbase/.hbck/hbase-xxxxxx", right?
This way they won't clutter the "/hbase/" when many of them get created during.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399695#comment-13399695 ]
Aditya Kishore commented on HBASE-6236:
---------------------------------------
@Ted,
Yes, I was :)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399832#comment-13399832 ]
Zhihong Ted Yu commented on HBASE-6236:
---------------------------------------
Integrated to 0.94 and trunk.
Thanks for the patch, Aditya.
Thanks for the review, Jon and Jimmy.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6236:
----------------------------------
Attachment: 6236-94.addendum
Addendum adds -sidelineDir option to hbck
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Attachment: (was: HBASE-6236_trunk.patch)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Attachment: HBASE-6236_trunk.patch
Change Summary (consolidated):
-------------------------------
1. Changed the default sideline folder to "/hbase/.hbck/hbase-xxxxxxxxx" where xxxxxxxxx is System.currentTimeMillis().
2. Added a command line option "-sidelineDir" to override this default.
3. Added ".hbck" to the list of Non-user table dir so that it is not treated as a user table folder.
4. Fixed the potential NPE in argument parsing code.
Thank you all for your valuable time spent reviewing my submission :)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Description:
While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
was:
While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399782#comment-13399782 ]
Hadoop QA commented on HBASE-6236:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533129/HBASE-6236_trunk.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2227//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2227//console
This message is automatically generated.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399546#comment-13399546 ]
Jimmy Xiang commented on HBASE-6236:
------------------------------------
We should revert this change.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396522#comment-13396522 ]
Hadoop QA commented on HBASE-6236:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12532496/HBASE-6236_0.94.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2188//console
This message is automatically generated.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399544#comment-13399544 ]
Jimmy Xiang commented on HBASE-6236:
------------------------------------
hbck sideline the dir to some folder like /hbck/hbase-xxxxxx, not /hbase-xxxxx
In a federated HDFS, you can pre-create a folder like /hbck, which is for this kind of use cases.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Attachment: HBASE-6236_94.patch
HBASE-6236_trunk.patch
Here are new patches to address this issue.
Change summary:
1. By default, the sideline folder is "<hbase base dorectory>/.hbcktmp-xxxxxxxxxxxxx" where xxxxxxxxxxxxx is System.currentTimeMillis()
2. A new optional command line parameter "-backup <hdfs://...>" has been added which can override the the default sideline folder.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Attachments: HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399842#comment-13399842 ]
Hudson commented on HBASE-6236:
-------------------------------
Integrated in HBase-TRUNK #3063 (See [https://builds.apache.org/job/HBase-TRUNK/3063/])
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or sim
ilar FS (Aditya) (Revision 1353065)
Result = SUCCESS
tedyu :
Files :
* /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Attachment: (was: HBASE-6236_trunk.patch)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6236:
----------------------------------
Release Note: (was: -sidelineDir has been added to offline meta repair tool.
User can use this option to specify HDFS path to backup existing -ROOT- and .META.)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399694#comment-13399694 ]
Aditya Kishore commented on HBASE-6236:
---------------------------------------
@Jonathan:
I noticed that too but wasn't sure whether to combine that issue with the current one. I guess there is no harm in fixing that while I am touching this part of code. Will include it in the patch.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and -ROOT- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399810#comment-13399810 ]
Hadoop QA commented on HBASE-6236:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533142/HBASE-6236_trunk.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2229//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2229//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2229//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2229//console
This message is automatically generated.
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6236) Offline meta repair fails if the
HBase base mount point is on a different cluster/volume than its parent in
a ViewFS or similar FS
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6236:
----------------------------------
Attachment: (was: HBASE-6236_trunk.patch)
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6236) Offline meta repair fails if the HBase
base mount point is on a different cluster/volume than its parent in a
ViewFS or similar FS
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl closed HBASE-6236.
--------------------------------
> Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6236
> URL: https://issues.apache.org/jira/browse/HBASE-6236
> Project: HBase
> Issue Type: Bug
> Components: hbck
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 0.94.1, 0.96.0
>
> Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, HBASE-6236_trunk.patch
>
>
> While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck tries to move the existing .META. and \-ROOT\- directories to a backup folder.
> This backup folder is created at the same level as the base HBase folder (e.g. /hbase-xxxxxx if the base HBase folder is '/hbase').
> In a federated HDFS like ViewFS and other similar FS implementations, it is not possible to rename files/directories across namespace volumes (ViewFS guide section 3.5) and as a result hbck crashes.
> A solution to this problem is to create the backup directory under the folder where HBase base folder has been mounted. This ensures that source and destination of rename operation are on the same namespace volume.
> Patch for 0.94 and trunk is attached for review. The patch modifies the location of the backup directory from '/hbase-xxxxxxx' to '/hbase/.hbcktmp-xxxxxxx'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira