You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Josh Elser (JIRA)" <ji...@apache.org> on 2019/03/02 21:44:00 UTC
[jira] [Updated] (HBASE-21982) HBase Kerberos with no Hadoop/HDFS
fails on startup
[ https://issues.apache.org/jira/browse/HBASE-21982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Elser updated HBASE-21982:
-------------------------------
Release Note: (was: as it needs core-site.xml with:
<configuration>
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value>
</property>
</configuration>)
> HBase Kerberos with no Hadoop/HDFS fails on startup
> ---------------------------------------------------
>
> Key: HBASE-21982
> URL: https://issues.apache.org/jira/browse/HBASE-21982
> Project: HBase
> Issue Type: Bug
> Components: master, regionserver, rpc
> Affects Versions: 1.4.2, 1.4.9
> Reporter: Greg Senia
> Priority: Major
>
> When attempting to Kerberize an HBase Instance that uses the localFS without Hadoop I noticed that instead of the HBase RegionServer Successfully checking in with the HBase Master it fails stating that it was using SIMPLE authentication vs Kerberos. So I think the real question here is does HBase support running without HDFS/Hadoop for the filesystem in Kerberos Mode or is HDFS required?
> Error on RegionServer:
> 3-02 13:09:46,314 DEBUG [regionserver/owlms.hdp.senia.org/10.69.68.21:16020] ipc.BlockingRpcConnection: Connecting to owlms.hdp.senia.org/10.69.68.21:16000
> 2019-03-02 13:09:46,315 DEBUG [regionserver/owlms.hdp.senia.org/10.69.68.21:16020] security.UserGroupInformation: PrivilegedAction as:hbase (auth:SIMPLE) from:org.apache.hadoop.hbase.ipc.BlockingRpcConnection.setupIOstreams(BlockingRpcConnection.java:452)
> 2019-03-02 13:09:46,315 DEBUG [regionserver/owlms.hdp.senia.org/10.69.68.21:16020] security.AbstractHBaseSaslRpcClient: Creating SASL GSSAPI client. Server's Kerberos principal name is hbase/owlms.hdp.senia.org@HDP.SENIA.ORG
> 2019-03-02 13:09:46,318 DEBUG [regionserver/owlms.hdp.senia.org/10.69.68.21:16020] security.HBaseSaslRpcClient: Have sent token of size 635 from initSASLContext.
> 2019-03-02 13:09:46,318 DEBUG [regionserver/owlms.hdp.senia.org/10.69.68.21:16020] security.UserGroupInformation: PrivilegedActionException as:hbase (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.security.AccessDeniedException): Kerberos principal name does NOT have the expected hostname part: hbase
> 2019-03-02 13:09:46,319 DEBUG [regionserver/owlms.hdp.senia.org/10.69.68.21:16020] security.UserGroupInformation: PrivilegedAction as:hbase (auth:SIMPLE) from:org.apache.hadoop.hbase.ipc.BlockingRpcConnection.handleSaslConnectionFailure(BlockingRpcConnection.java:374)
> 2019-03-02 13:09:46,319 WARN [regionserver/owlms.hdp.senia.org/10.69.68.21:16020] ipc.BlockingRpcConnection: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.security.AccessDeniedException): Kerberos principal name does NOT have the expected hostname part: hbase
> 2019-03-02 13:09:46,319 DEBUG [regionserver/owlms.hdp.senia.org/10.69.68.21:16020] security.UserGroupInformation: PrivilegedActionException as:hbase (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.security.AccessDeniedException): Kerberos principal name does NOT have the expected hostname part: hbase
> 2019-03-02 13:09:46,319 DEBUG [regionserver/owlms.hdp.senia.org/10.69.68.21:16020] ipc.FailedServers: Added failed server with address owlms.hdp.senia.org/10.69.68.21:16000 to list caused by org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.security.AccessDeniedException): Kerberos principal name does NOT have the expected hostname part: hbase
> 2019-03-02 13:09:46,319 WARN [regionserver/owlms.hdp.senia.org/10.69.68.21:16020] regionserver.HRegionServer: error telling master we are up
> com.google.protobuf.ServiceException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.security.AccessDeniedException): Kerberos principal name does NOT have the expected hostname part: hbase
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:335)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$200(AbstractRpcClient.java:94)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:571)
> at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2431)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:969)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.security.AccessDeniedException): Kerberos principal name does NOT have the expected hostname part: hbase
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:386)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:94)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:409)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:405)
> at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
> at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:422)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:327)
> ... 6 more
> Error on HBase Master:
> 2019-03-02 14:14:13,593 DEBUG [RpcServer.reader=3,bindAddress=owlms.hdp.senia.org,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: DISCONNECTING client 10.69.68.21:35620 because read count=-1. Number of active connections: 1
> 2019-03-02 14:14:14,615 INFO [owlms:16000.activeMasterManager] master.ServerManager: Waiting on RegionServer count=0 to settle; waited=10325215ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-10325215ms
> 2019-03-02 14:14:15,828 DEBUG [master/owlms.hdp.senia.org/10.69.68.21:16000-SendThread(owlms.hdp.senia.org:2181)] zookeeper.ClientCnxn: Got ping response for sessionid: 0x1693caa4d83009b after 0ms
> 2019-03-02 14:14:15,928 DEBUG [owlms:16000.activeMasterManager-SendThread(owlms.hdp.senia.org:2181)] zookeeper.ClientCnxn: Got ping response for sessionid: 0x1693caa4d83009c after 0ms
> 2019-03-02 14:14:16,119 INFO [owlms:16000.activeMasterManager] master.ServerManager: Waiting on RegionServer count=0 to settle; waited=10326719ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-10326719ms
> 2019-03-02 14:14:16,590 DEBUG [owlms:16000.activeMasterManager-SendThread(owlms.hdp.senia.org:2181)] zookeeper.ClientCnxn: Got ping response for sessionid: 0x1693caa4d83009e after 0ms
> 2019-03-02 14:14:16,595 DEBUG [RpcServer.listener,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: connection from 10.69.68.21:52423; # active connections: 1
> 2019-03-02 14:14:16,598 DEBUG [RpcServer.reader=4,bindAddress=owlms.hdp.senia.org,port=16000] ipc.RpcServer: Kerberos principal name is hbase
> 2019-03-02 14:14:16,598 DEBUG [RpcServer.reader=4,bindAddress=owlms.hdp.senia.org,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: Caught exception while reading:
> org.apache.hadoop.hbase.security.AccessDeniedException: Kerberos principal name does NOT have the expected hostname part: hbase
> at org.apache.hadoop.hbase.ipc.RpcServer$Connection.saslReadAndProcess(RpcServer.java:1468)
> at org.apache.hadoop.hbase.ipc.RpcServer$Connection.process(RpcServer.java:1788)
> at org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1769)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:955)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:725)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:701)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-03-02 14:14:16,598 DEBUG [RpcServer.reader=4,bindAddress=owlms.hdp.senia.org,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: DISCONNECTING client 10.69.68.21:52423 because read count=-1. Number of active connections: 1
> 2019-03-02 14:14:17,622 INFO [owlms:16000.activeMasterManager] master.ServerManager: Waiting on RegionServer count=0 to settle; waited=10328222ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-10328222ms
> 2019-03-02 14:14:19,125 INFO [owlms:16000.activeMasterManager] master.ServerManager: Waiting on RegionServer count=0 to settle; waited=10329725ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-10329725ms
> 2019-03-02 14:14:19,602 DEBUG [RpcServer.listener,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: connection from 10.69.68.21:58029; # active connections: 1
> 2019-03-02 14:14:19,608 DEBUG [RpcServer.reader=5,bindAddress=owlms.hdp.senia.org,port=16000] ipc.RpcServer: Kerberos principal name is hbase
> 2019-03-02 14:14:19,608 DEBUG [RpcServer.reader=5,bindAddress=owlms.hdp.senia.org,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: Caught exception while reading:
> org.apache.hadoop.hbase.security.AccessDeniedException: Kerberos principal name does NOT have the expected hostname part: hbase
> at org.apache.hadoop.hbase.ipc.RpcServer$Connection.saslReadAndProcess(RpcServer.java:1468)
> at org.apache.hadoop.hbase.ipc.RpcServer$Connection.process(RpcServer.java:1788)
> at org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1769)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:955)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:725)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:701)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)