[GitHub] [iceberg] jshmchenxi commented on issue #2753: Token expiration issued when using WORKER_POOL with Spark Thrift Server

[GitBox <gi...@apache.org>]

jshmchenxi commented on issue #2753:
URL: https://github.com/apache/iceberg/issues/2753#issuecomment-870683650


   It's a problem using wrong UGI. The UGI is correct outside the WORKER_POOL.
   We also ran into the following situration:
   
   1. Restart Kyuubi service
   2. User A submits several queries to Kyuubi
   3. After days of running, user B submits a query which triggers accessing HDFS via WORKER_POOL
   4. Exception thrown complaining about token of user A is expired
   ```
   Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): token (token for A: HDFS_DELEGATION_TOKEN owner=A, renewer=yarn, realUser=spark/server@HADOOP.COM, issueDate=1622108637132, maxDate=1622713437132, sequenceNumber=105708024, masterKeyId=708) can't be found in cache
   ``` 
   
   Obviously the UGI that WORKER_POOL is using is not correct.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org