Hbase on hadoop

[Rick Hangartner <ha...@strands.com>]

Hi,

We have an issue with hbase on hadoop and file system permissions we  
hope someone already knows the answer to.  Our apologies if we missed  
that this issue has already been addressed on this list.

We are running hbase-0.1.2 on top of hadoop-0.16.3, starting the hbase  
daemon from an "hbase" user account and the hadoop daemon and have  
observed this "feature".   We are running hbase in a separate "hadoop"  
user account and hadoop in it's own "hadoop" user account on a single  
machine.

When we try to start up hbase, we see this error message in the log:

2008-05-06 12:09:02,845 ERROR org.apache.hadoop.hbase.HMaster: Can not  
start master
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native  
Method)
	at  
sun 
.reflect 
.NativeConstructorAccessorImpl 
.newInstance(NativeConstructorAccessorImpl.java:39)
	at  
sun 
.reflect 
.DelegatingConstructorAccessorImpl 
.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
	at org.apache.hadoop.hbase.HMaster.doMain(HMaster.java:3329)
	at org.apache.hadoop.hbase.HMaster.main(HMaster.java:3363)
Caused by: org.apache.hadoop.ipc.RemoteException:  
org.apache.hadoop.fs.permission.AccessControlException: Superuser  
privilege is required
         ... (etc)

If we run hbase in the hadoop user account we don't have any problems.

We think we've narrowed the issue down a bit from the debug logs.

The method "FSNameSystem.checkPermission()" method is throwing the  
exception because the "PermissionChecker()" constructor is returning  
that the hbase user is not a superuser or in the same supergroup as  
hadoop.

   private void checkSuperuserPrivilege() throws  
AccessControlException {
     if (isPermissionEnabled) {
       PermissionChecker pc = new PermissionChecker(
           fsOwner.getUserName(), supergroup);
       if (!pc.isSuper) {
         throw new AccessControlException("Superuser privilege is  
required");
       }
     }
   }

If we look at at the "PermissionChecker()" constructor we see that it  
is comparing the hdfs owner name (which should be "hadoop") and the  
hdfs file system owner's group ("supergroup") to the current user and  
groups, which the log seems to indicate the user is "hbase" and the  
groups for user "hbase" only include "hbase" :
   PermissionChecker(String fsOwner, String supergroup
       ) throws AccessControlException{
     UserGroupInformation ugi = UserGroupInformation.getCurrentUGI();
     if (LOG.isDebugEnabled()) {
       LOG.debug("ugi=" + ugi);
     }

     if (ugi != null) {
       user = ugi.getUserName();
       groups.addAll(Arrays.asList(ugi.getGroupNames()));
       isSuper = user.equals(fsOwner) || groups.contains(supergroup);
     }
     else {
       throw new AccessControlException("ugi = null");
     }
   }

The current user and group is derived from the thread information:
   private static final ThreadLocal<UserGroupInformation> currentUGI
     = new ThreadLocal<UserGroupInformation>();

   /** @return the {@link UserGroupInformation} for the current thread  
*/
   public static UserGroupInformation getCurrentUGI() {
     return currentUGI.get();
   }

which we're hoping might be enough to illuminate the problem.

One question this raises is if the "hbase:hbase" user and group are  
being derived from the Linux file system user and group, or if they  
are the hdfs user and group?
Otherwise, how can we indicate that "hbase" user is in the hdfs group  
"supergroup"? Is there a parameter in a hadoop configuration file?   
Apparently setting the groups of the web server to include  
"supergroup" didn't have any effect, although perhaps that could be  
for some other reason?

Thanks.