You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/03/03 08:30:50 UTC

[GitHub] [incubator-kyuubi] zhouyifan279 opened a new issue #2001: [Bug] Kyuubi SQL engine rejected hadoop kms token from server

zhouyifan279 opened a new issue #2001:
URL: https://github.com/apache/incubator-kyuubi/issues/2001


   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   
   
   ### Search before asking
   
   - [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Describe the bug
   
   Kyuubi SQL engine rejected hadoop kms token from kyuubi server. 
   This leads to authentication error after kms token expired.
   
   ### Affects Version(s)
   
   master/1.4.0/1.4.1
   
   ### Kyuubi Server Log Output
   
   _No response_
   
   ### Kyuubi Engine Log Output
   
   ```logtalk
   2022-02-28 09:33:13,404 [51629] - WARN  [SparkThriftBinaryFrontendServiceHandler-Pool: Thread-140:Logging@60] - No matching Hive token found for engine metastore uris thrift://hive1:9083,thrift://hive2:9083
   2022-02-28 09:33:13,444 [51669] - WARN  [SparkThriftBinaryFrontendServiceHandler-Pool: Thread-140:Logging@60] - Ignore token with earlier issue date: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:cluster01, Ident: 00 08 64 61 5f 6d 75 73 69 63 08 64 61 5f 6d 75 73 69 63 2e 68 69 76 65 2f 73 70 61 72 6b 31 31 2e 6a 64 2e 31 36 33 2e 6f 72 67 40 48 41 44 4f 4f 50 32 2e 48 5a 2e 4e 45 54 45 41 53 45 2e 43 4f 4d 8a 01 7f 3d dd 1a 50 8a 01 7f 61 e9 9e 50 8c 10 b4 bd 63 8e 08 cf
   2022-02-28 09:33:13,446 [51671] - INFO  [SparkThriftBinaryFrontendServiceHandler-Pool: Thread-140:Logging@54] - Ignore unknown token Kind: kms-dt, Service: kms://http@hadoop7874;hadoop7875:16000/kms, Ident: (owner=morty, renewer=morty, realUser=hive, issueDate=1646010374994, maxDate=1646615174994, sequenceNumber=90506870, masterKeyId=400)
   ```
   
   
   ### Kyuubi Server Configurations
   
   _No response_
   
   ### Kyuubi Engine Configurations
   
   _No response_
   
   ### Additional context
   
   Spark uses hadoop-client:2.9.2.
   
   Launch spark-shell the same way as SQL engine, we got the tokens in driver:
   
   ```
   scala> UserGroupInformation.getCurrentUser.getCredentials.getAllTokens
   res2: java.util.Collection[org.apache.hadoop.security.token.Token[_ <: org.apache.hadoop.security.token.TokenIdentifier]] = [Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:cluster01, Ident: 00 10 62 64 6d 73 5f 7a 68 6f 75 79 69 66 61 6e 30 33 04 79 61 72 6e 2e 68 69 76 65 2f 73 70 61 72 6b 31 30 2e 6a 64 2e 31 36 33 2e 6f 72 67 40 48 41 44 4f 4f 50 32 2e 48 5a 2e 4e 45 54 45 41 53 45 2e 43 4f 4d 8a 01 7f 44 e7 09 58 8a 01 7f 68 f3 8d 58 8c 10 ba cb f5 8e 08 e3, Kind: kms-dt, Service: 192.168.3.190:16000, Ident: (owner=morty, renewer=yarn, realUser=hive, issueDate=1646128466387, maxDate=1646733266387, sequenceNumber=91208684, masterKeyId=404), Kind: HIVE_DELEGATION_TOKEN, Service: , Ident: 00 10 62 64 6d 73 5f 7a 68 6f 75 79 69 66 61 6e 30 33 04 68...
   ```
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] zhouyifan279 commented on issue #2001: [Bug] Kyuubi SQL engine rejected hadoop kms token from server

Posted by GitBox <gi...@apache.org>.
zhouyifan279 commented on issue #2001:
URL: https://github.com/apache/incubator-kyuubi/issues/2001#issuecomment-1057802664


   Relates to hadoop client behavior change after https://issues.apache.org/jira/browse/HADOOP-14445


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org