You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Anton Koval (Jira)" <ji...@apache.org> on 2020/11/10 09:23:00 UTC
[jira] [Commented] (NIFI-7831) KeytabCredentialsService not working
with HBase Clients
[ https://issues.apache.org/jira/browse/NIFI-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229082#comment-17229082 ]
Anton Koval commented on NIFI-7831:
-----------------------------------
We have the same issue after upgrade NiFi from 1.11.4 to 1.12.1
> KeytabCredentialsService not working with HBase Clients
> -------------------------------------------------------
>
> Key: NIFI-7831
> URL: https://issues.apache.org/jira/browse/NIFI-7831
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.12.0
> Reporter: Manuel Navarro
> Priority: Major
>
> HBase Client (both 1.x and 2.x) is not able to renew ticket after expiration with KeytabCredentialsService configured (same behaviour with principal and password configured directly in the controller service). The same KeytabCredentialsService works ok with Hive and Hbase clients configured in the same NIFI cluster.
> Note that the same configuration works ok in version 1.11 (error start to appear after upgrade from 1.11 to 1.12).
> After 24hours (time renewal period in our case), the following error appears using HBase_2_ClientServices + HBase_2_ClientMapCacheService :
> {code:java}
> 2020-09-17 09:00:27,014 ERROR [Relogin service.Chore.1] org.apache.hadoop.hbase.AuthUtil Got exception while trying to refresh credentials: loginUserFromKeyTab must be done first java.io.IOException: loginUserFromKeyTab must be done first at org.apache.hadoop.security.UserGroupInformation.reloginFromKeytab(UserGroupInformation.java:1194) at org.apache.hadoop.security.UserGroupInformation.checkTGTAndReloginFromKeytab(UserGroupInformation.java:1125) at org.apache.hadoop.hbase.AuthUtil$1.chore(AuthUtil.java:206) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
> {code}
>
> With HBase_1_1_2_ClientServices + HBase_1_1_2_ClientMapCacheService the following error appears:
>
> {code:java}
> 2020-09-22 12:18:37,184 WARN [hconnection-0x55d9d8d1-shared--pool3-t769] o.a.hadoop.hbase.ipc.AbstractRpcClient Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 2020-09-22 12:18:37,197 ERROR [hconnection-0x55d9d8d1-shared--pool3-t769] o.a.hadoop.hbase.ipc.AbstractRpcClient SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'. javax.security.sasl.SaslException: GSS initiate failed at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:612) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:157) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:738) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:735) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:897) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:866) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1208) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:328) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.multi(ClientProtos.java:32879) at org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:128) at org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:53) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210) at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl$SingleServerRequestRunnable.run(AsyncProcess.java:723) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
> {code}
>
> Environment: Apache NIFI 1.12, RHEL 7.7, openjdk version "1.8.0_222-ea"
> Regards!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)