You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Eric Yang <ey...@hortonworks.com> on 2011/11/08 23:40:45 UTC

HBase user as different user from HDFS user for HBase 0.92 and Hadoop 0.20.205.0

Hi all,

HBase 0.92 branch is setting permission of / to 755 before HBase Master can start.  This triggers a permission denied error, when HBase user is different from HDFS user.  The stack trace of this exception:

hbase master log file:

2011-11-08 14:15:25,716 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
        at org.apache.hadoop.hdfs.DFSClient.setPermission(DFSClient.java:922)
        at org.apache.hadoop.hdfs.DistributedFileSystem.setPermission(DistributedFileSystem.java:515)
        at org.apache.hadoop.hbase.util.FSUtils.isInSafeMode(FSUtils.java:168)
        at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:456)
        at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:301)
        at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127)
        at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:112)
        at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:426)
        at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
        at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.security.AccessControlException: Permission denied
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkOwner(FSPermissionChecker.java:149)
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:140)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5164)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOwner(FSNamesystem.java:5123)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setPermission(FSNamesystem.java:845)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.setPermission(NameNode.java:653)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)

        at org.apache.hadoop.ipc.Client.call(Client.java:1066)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
        at $Proxy10.setPermission(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
        at $Proxy10.setPermission(Unknown Source)
        at org.apache.hadoop.hdfs.DFSClient.setPermission(DFSClient.java:920)
        ... 9 more

namenode log:

2011-11-08 14:15:25,713 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8020, call setPermission(/, rwxr-xr-x) from 127.0.0.1:55838: error: org.apache.hadoop.security.AccessControlException: Permission denied
org.apache.hadoop.security.AccessControlException: Permission denied
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkOwner(FSPermissionChecker.java:149)
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:140)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5164)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOwner(FSNamesystem.java:5123)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setPermission(FSNamesystem.java:845)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.setPermission(NameNode.java:653)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)

What is the reason of setting permission of "/" to 755?  In theory, HDFS has been setup with a /apps/hbase directory, and set proper permission for hbase to interact only in /apps/hbase directory.  Does this assumption work?

regards,
Eric

Re: HBase user as different user from HDFS user for HBase 0.92 and Hadoop 0.20.205.0

Posted by Eric Yang <ey...@hortonworks.com>.
I am using the default template in hbase/src/packages/templates/conf/hbase-site.xml.

regards,
Eric

On Nov 8, 2011, at 2:44 PM, Roman Shaposhnik wrote:

> On Tue, Nov 8, 2011 at 2:40 PM, Eric Yang <ey...@hortonworks.com> wrote:
>> Hi all,
>> 
>> HBase 0.92 branch is setting permission of / to 755 before HBase Master can start.
>> This triggers a permission denied error, when HBase user is different from HDFS user.  The stack trace of this exception:
> 
> What's an hbase-site.xml you're using? I don't see this issue with the
> one from Bigtop.
> 
> Thanks,
> Roman.

Re: HBase user as different user from HDFS user for HBase 0.92 and Hadoop 0.20.205.0

Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, Nov 8, 2011 at 2:40 PM, Eric Yang <ey...@hortonworks.com> wrote:
> Hi all,
>
> HBase 0.92 branch is setting permission of / to 755 before HBase Master can start.
> This triggers a permission denied error, when HBase user is different from HDFS user.  The stack trace of this exception:

What's an hbase-site.xml you're using? I don't see this issue with the
one from Bigtop.

Thanks,
Roman.

Re: HBase user as different user from HDFS user for HBase 0.92 and Hadoop 0.20.205.0

Posted by Eric Yang <ey...@hortonworks.com>.
Hi Harsh,

Thanks for the heads up.  My git repository was out of date.  HBASSE-4708 solved the problem.  Thanks again.

regards,
Eric

On Nov 8, 2011, at 6:49 PM, Harsh J wrote:

> This was https://issues.apache.org/jira/browse/HBASE-4510 but was reverted. Are you sure you are using the latest 0.92 HEAD?
> 
> Revert ticket was https://issues.apache.org/jira/browse/HBASE-4708
> 
> On 09-Nov-2011, at 4:10 AM, Eric Yang wrote:
> 
>> Hi all,
>> 
>> HBase 0.92 branch is setting permission of / to 755 before HBase Master can start.  This triggers a permission denied error, when HBase user is different from HDFS user.  The stack trace of this exception:
>> 
>> hbase master log file:
>> 
>> 2011-11-08 14:15:25,716 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
>> org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied
>>       at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>       at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>>       at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>>       at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>>       at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
>>       at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
>>       at org.apache.hadoop.hdfs.DFSClient.setPermission(DFSClient.java:922)
>>       at org.apache.hadoop.hdfs.DistributedFileSystem.setPermission(DistributedFileSystem.java:515)
>>       at org.apache.hadoop.hbase.util.FSUtils.isInSafeMode(FSUtils.java:168)
>>       at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:456)
>>       at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:301)
>>       at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127)
>>       at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:112)
>>       at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:426)
>>       at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
>>       at java.lang.Thread.run(Thread.java:662)
>> Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.security.AccessControlException: Permission denied
>>       at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkOwner(FSPermissionChecker.java:149)
>>       at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:140)
>>       at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5164)
>>       at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOwner(FSNamesystem.java:5123)
>>       at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setPermission(FSNamesystem.java:845)
>>       at org.apache.hadoop.hdfs.server.namenode.NameNode.setPermission(NameNode.java:653)
>>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>       at java.lang.reflect.Method.invoke(Method.java:597)
>>       at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>>       at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>>       at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>>       at java.security.AccessController.doPrivileged(Native Method)
>>       at javax.security.auth.Subject.doAs(Subject.java:396)
>>       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>>       at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>> 
>>       at org.apache.hadoop.ipc.Client.call(Client.java:1066)
>>       at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>       at $Proxy10.setPermission(Unknown Source)
>>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>       at java.lang.reflect.Method.invoke(Method.java:597)
>>       at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>       at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>       at $Proxy10.setPermission(Unknown Source)
>>       at org.apache.hadoop.hdfs.DFSClient.setPermission(DFSClient.java:920)
>>       ... 9 more
>> 
>> namenode log:
>> 
>> 2011-11-08 14:15:25,713 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8020, call setPermission(/, rwxr-xr-x) from 127.0.0.1:55838: error: org.apache.hadoop.security.AccessControlException: Permission denied
>> org.apache.hadoop.security.AccessControlException: Permission denied
>>       at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkOwner(FSPermissionChecker.java:149)
>>       at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:140)
>>       at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5164)
>>       at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOwner(FSNamesystem.java:5123)
>>       at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setPermission(FSNamesystem.java:845)
>>       at org.apache.hadoop.hdfs.server.namenode.NameNode.setPermission(NameNode.java:653)
>>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>       at java.lang.reflect.Method.invoke(Method.java:597)
>>       at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>>       at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>>       at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>>       at java.security.AccessController.doPrivileged(Native Method)
>>       at javax.security.auth.Subject.doAs(Subject.java:396)
>>       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>>       at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>> 
>> What is the reason of setting permission of "/" to 755?  In theory, HDFS has been setup with a /apps/hbase directory, and set proper permission for hbase to interact only in /apps/hbase directory.  Does this assumption work?
>> 
>> regards,
>> Eric
>

Re: HBase user as different user from HDFS user for HBase 0.92 and Hadoop 0.20.205.0

Posted by Harsh J <ha...@cloudera.com>.
This was https://issues.apache.org/jira/browse/HBASE-4510 but was reverted. Are you sure you are using the latest 0.92 HEAD?

Revert ticket was https://issues.apache.org/jira/browse/HBASE-4708

On 09-Nov-2011, at 4:10 AM, Eric Yang wrote:

> Hi all,
> 
> HBase 0.92 branch is setting permission of / to 755 before HBase Master can start.  This triggers a permission denied error, when HBase user is different from HDFS user.  The stack trace of this exception:
> 
> hbase master log file:
> 
> 2011-11-08 14:15:25,716 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
> org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
>        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
>        at org.apache.hadoop.hdfs.DFSClient.setPermission(DFSClient.java:922)
>        at org.apache.hadoop.hdfs.DistributedFileSystem.setPermission(DistributedFileSystem.java:515)
>        at org.apache.hadoop.hbase.util.FSUtils.isInSafeMode(FSUtils.java:168)
>        at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:456)
>        at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:301)
>        at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127)
>        at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:112)
>        at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:426)
>        at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
>        at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.security.AccessControlException: Permission denied
>        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkOwner(FSPermissionChecker.java:149)
>        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:140)
>        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5164)
>        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOwner(FSNamesystem.java:5123)
>        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setPermission(FSNamesystem.java:845)
>        at org.apache.hadoop.hdfs.server.namenode.NameNode.setPermission(NameNode.java:653)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at javax.security.auth.Subject.doAs(Subject.java:396)
>        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> 
>        at org.apache.hadoop.ipc.Client.call(Client.java:1066)
>        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>        at $Proxy10.setPermission(Unknown Source)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>        at $Proxy10.setPermission(Unknown Source)
>        at org.apache.hadoop.hdfs.DFSClient.setPermission(DFSClient.java:920)
>        ... 9 more
> 
> namenode log:
> 
> 2011-11-08 14:15:25,713 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8020, call setPermission(/, rwxr-xr-x) from 127.0.0.1:55838: error: org.apache.hadoop.security.AccessControlException: Permission denied
> org.apache.hadoop.security.AccessControlException: Permission denied
>        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkOwner(FSPermissionChecker.java:149)
>        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:140)
>        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5164)
>        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOwner(FSNamesystem.java:5123)
>        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setPermission(FSNamesystem.java:845)
>        at org.apache.hadoop.hdfs.server.namenode.NameNode.setPermission(NameNode.java:653)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at javax.security.auth.Subject.doAs(Subject.java:396)
>        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> 
> What is the reason of setting permission of "/" to 755?  In theory, HDFS has been setup with a /apps/hbase directory, and set proper permission for hbase to interact only in /apps/hbase directory.  Does this assumption work?
> 
> regards,
> Eric