[jira] [Created] (HBASE-8644) "hbck -fixReferenceFiles" checks for write permissions in / even if sidelineDir is specified

["Clint Heath (JIRA)" <ji...@apache.org>]

Clint Heath created HBASE-8644:
----------------------------------

             Summary: "hbck -fixReferenceFiles" checks for write permissions in / even if sidelineDir is specified
                 Key: HBASE-8644
                 URL: https://issues.apache.org/jira/browse/HBASE-8644
             Project: HBase
          Issue Type: Bug
          Components: hbck
    Affects Versions: 0.94.2
            Reporter: Clint Heath
            Priority: Minor


We found a situation where a temporary reference file got left in META's recovered.edits directory and this was causing hbck to report inconsistency with HBase.  When trying to repair it with the "hbck -fixReferenceFiles" command, an access control exception was thrown, because hbck was trying to put the sidelined reference file in the root of HDFS and the hbase user did not have permissions.  But even after specifying a "sidelineDir" (hbck -fixReferenceFiles -sidelineDir hdfs://<namespace>/tmp), the same exception was thrown (logs obfuscated below):

INFO util.HBaseFsck: Trying to sildeline reference filehdfs://<namespace>/hbase/.META./1028685194/recovered.edits/0000000000000000016.temp to hdfs://<namespace>/0000000000000000016.temp 
Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode="/":hdfs:hadoop:drwxr-xr-x 

Since the operation requires moving data around in HDFS, we tried the same command as the "hdfs" user which should have no permission problems, but an exception was thrown nonetheless:
 
... 
WARN util.HBaseFsck: Got AccessControlException when preCheckPermission 
org.apache.hadoop.security.AccessControlException: Permission denied: action=WRITE path=hdfs://<namespace>/hbase/-ROOT- user=hdfs 

Ultimately in order to get this working, we ran this command to tell hbck to not run it's prechecks:

hbase hbck -fixReferenceFiles -sidelineDir hdfs://<namespace>/tmp -ignorePreCheckPermission


A few thoughts here:

1) it seems that / is a bad default location to attempt to sideline files

2) shouldn't hbck be run as the hbase user?  If so, it should sideline files to a directory it has write permissions to.

3) if we expect hbck to be run as the hdfs user, permissions should not be denied that user when HDFS itself will not deny them.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira