You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@storm.apache.org by "Ethan Li (Jira)" <ji...@apache.org> on 2020/03/25 17:58:00 UTC
[jira] [Updated] (STORM-3606) AutoTGT shouldn't invoke TGT renewal
thread (from UserGroupInformation.loginUserFromSubject)
[ https://issues.apache.org/jira/browse/STORM-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Li updated STORM-3606:
----------------------------
Description:
When hadoop security is enabled,
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/security/auth/kerberos/AutoTGT.java#L199-L209
AutoTGT will invoke "loginUserFromSubject", and it will spawn a "TGT renewal" thread.
https://github.com/apache/hadoop/blob/branch-2.8.5/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L928-L957
which will eventually invoke system command "kinit -R", and then fail with the exception
{code:java}
org.apache.hadoop.util.Shell$ExitCodeException: kinit: Credentials cache file '/tmp/krb5cc_xxx' not found while renewing credentials
at org.apache.hadoop.util.Shell.runCommand(Shell.java:1004) ~[stormjar.jar:?]
at org.apache.hadoop.util.Shell.run(Shell.java:898) ~[stormjar.jar:?]
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213) ~[stormjar.jar:?]
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1307) ~[stormjar.jar:?]
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1289) ~[stormjar.jar:?]
at org.apache.hadoop.security.UserGroupInformation$1.run(UserGroupInformation.java:1011) [stormjar.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
{code}
"kinit" will never work from worker process since Storm don't keep TGT in local cache. Instead, TGT is saved in zookeeper and in memory of Worker process.
This exception is confusing but not harmful to topologies. And the TGT renewal thread will eventually abort.
It's better to find a real solution for it. But for now we can document what might happen in AutoTGT code.
To be clear, we still need
was:
When hadoop security is enabled,
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/security/auth/kerberos/AutoTGT.java#L199-L209
AutoTGT will invoke "loginUserFromSubject", and it will spawn a "TGT renewal" thread.
https://github.com/apache/hadoop/blob/branch-2.8.5/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L928-L957
which will eventually invoke system command "kinit -R", and then fail with the exception
{code:java}
org.apache.hadoop.util.Shell$ExitCodeException: kinit: Credentials cache file '/tmp/krb5cc_xxx' not found while renewing credentials
at org.apache.hadoop.util.Shell.runCommand(Shell.java:1004) ~[stormjar.jar:?]
at org.apache.hadoop.util.Shell.run(Shell.java:898) ~[stormjar.jar:?]
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213) ~[stormjar.jar:?]
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1307) ~[stormjar.jar:?]
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1289) ~[stormjar.jar:?]
at org.apache.hadoop.security.UserGroupInformation$1.run(UserGroupInformation.java:1011) [stormjar.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
{code}
"kinit" will never work from worker process since Storm don't keep TGT in local cache. Instead, TGT is saved in zookeeper and in memory of Worker process.
This exception is confusing but not harmful to topologies. And the TGT renewal thread will eventually abort.
It's better to find a real solution for it. But for now we can document what might happen in AutoTGT code.
> AutoTGT shouldn't invoke TGT renewal thread (from UserGroupInformation.loginUserFromSubject)
> --------------------------------------------------------------------------------------------
>
> Key: STORM-3606
> URL: https://issues.apache.org/jira/browse/STORM-3606
> Project: Apache Storm
> Issue Type: Bug
> Reporter: Ethan Li
> Priority: Minor
>
> When hadoop security is enabled,
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/security/auth/kerberos/AutoTGT.java#L199-L209
> AutoTGT will invoke "loginUserFromSubject", and it will spawn a "TGT renewal" thread.
> https://github.com/apache/hadoop/blob/branch-2.8.5/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L928-L957
> which will eventually invoke system command "kinit -R", and then fail with the exception
> {code:java}
> org.apache.hadoop.util.Shell$ExitCodeException: kinit: Credentials cache file '/tmp/krb5cc_xxx' not found while renewing credentials
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:1004) ~[stormjar.jar:?]
> at org.apache.hadoop.util.Shell.run(Shell.java:898) ~[stormjar.jar:?]
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213) ~[stormjar.jar:?]
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:1307) ~[stormjar.jar:?]
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:1289) ~[stormjar.jar:?]
> at org.apache.hadoop.security.UserGroupInformation$1.run(UserGroupInformation.java:1011) [stormjar.jar:?]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
> {code}
> "kinit" will never work from worker process since Storm don't keep TGT in local cache. Instead, TGT is saved in zookeeper and in memory of Worker process.
> This exception is confusing but not harmful to topologies. And the TGT renewal thread will eventually abort.
> It's better to find a real solution for it. But for now we can document what might happen in AutoTGT code.
> To be clear, we still need
--
This message was sent by Atlassian Jira
(v8.3.4#803005)