You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Tom White (JIRA)" <ji...@apache.org> on 2012/09/26 17:07:07 UTC
[jira] [Created] (HADOOP-8852) DelegationTokenRenewer thread is not
stopped when its filesystem is closed
Tom White created HADOOP-8852:
---------------------------------
Summary: DelegationTokenRenewer thread is not stopped when its filesystem is closed
Key: HADOOP-8852
URL: https://issues.apache.org/jira/browse/HADOOP-8852
Project: Hadoop Common
Issue Type: Bug
Reporter: Tom White
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer should be
Singleton
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469413#comment-13469413 ]
Daryn Sharp commented on HADOOP-8852:
-------------------------------------
You raise good points and I've been wary of this class for awhile. Before critiquing the patch, I now question the very existence of {{DelegationTokenRenewer}}. Here's why:
Like all hdfs filesystems, a valid tgt is required obtain a token from the NN. This calls into question the entire need for these filesystems to implicitly obtain a token. If you can get a token, you don't need a token!
The api of this class also cannot handle multi-token filesystems so its not generic enough to be of general utility. Furthermore, token renewal is correctly handled within jobs so there's no need for a filesystem to internally renew. One could make the case for a more generalized renewal outside of yarn, but that would be another jira.
My recommendation is delete the class and all the logic in the filesystems that use it.
> DelegationTokenRenewer should be Singleton
> ------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch, hadoop-8852-v1.patch
>
>
> Updated description:
> DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
> Original issue:
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-8852) DelegationTokenRenewer thread is
not stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla reassigned HADOOP-8852:
----------------------------------------
Assignee: Karthik Kambatla
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Tom White
> Assignee: Karthik Kambatla
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) DelegationTokenRenewer thread is not
stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated HADOOP-8852:
-------------------------------------
Status: Open (was: Patch Available)
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is
not stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464319#comment-13464319 ]
Karthik Kambatla commented on HADOOP-8852:
------------------------------------------
When I ran test-patch.sh locally, it didn't complain of any findbugs warnings.
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) DelegationTokenRenewer thread is not
stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated HADOOP-8852:
-------------------------------------
Attachment: hadoop-8852.patch
Uploading a patch that
# Modifies {{DelegationTokenRenewer}} fields to non-static.
# overrides {{close()}} to interrupt the Renewer thread
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is
not stopped when its filesystem is closed
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464423#comment-13464423 ]
Hadoop QA commented on HADOOP-8852:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12546790/hadoop-8852.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
org.apache.hadoop.hdfs.TestPersistBlocks
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1535//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1535//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1535//console
This message is automatically generated.
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is
not stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464846#comment-13464846 ]
Karthik Kambatla commented on HADOOP-8852:
------------------------------------------
Thanks Tom and Daryn for your comments.
Daryn, your suggestions make absolute sense. Will work on the patch. While working on this earlier, I thought of making it a Singleton, but thought there was a reason why it was not Singleton :)
I will add {{cancelToken()}} and {{shutdown()}} to {{DelegationTokenRenewer}}, and use them in the FileSystems.
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer should be
Singleton
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13468968#comment-13468968 ]
Karthik Kambatla commented on HADOOP-8852:
------------------------------------------
After spending some time on this, I noticed a couple of other related issues:
- WebHdfsFileSystem and HftpFileSystem have their own implemenations of TokenRenewer that are not used at all.
- In DelegationTokenRenewer#renew() (see snippet below), we fetch new tokens in case the renew fails. However, we use only the first token and ignore the rest. Later, if we can't renew this token, we re-fetch new tokens instead of using the previously fetched ones. Also, we might not be able to fetch new ones, in which case we give up. Is this valid behavior? Or, should we be first using the initially fetched tokens, and re-fetch only when we run out of them.
{code}
Token<?>[] tokens = fs.addDelegationTokens(null, null);
if (tokens.length == 0) {
throw new IOException("addDelegationTokens returned no tokens");
}
fs.setDelegationToken(tokens[0]);
{code}
Taking the above into consideration along with the goal of further decoupling DelegationTokenRenewer from actual FS-specific details of renewal/refetching/cancellation, I propose the following. Please advise appropriately.
# TokenRenewer should be an interface (it is currently a fully abstract class).
# Create a new abstract class TokenManager that holds a list (queue) of tokens, their kind, renewal period, a TokenRenewer implementation, and an abstract method #manage() to manage (renew/re-fetch/cancel) the tokens as appropriate.
# Each FileSystem (WebHdfs and Hftp) has its own implemenation (local extended class) of the TokenManager.
# DelegationTokenRenewer allows registering and de-registering TokenManagers. FileSystems call register() at init(), and de-register() at close().
# DelegationTokenRenewer stores the registered TokenManagers in its DelayQueue. On a TokenManager's turn, the manage() method is invoked. If the manage fails, the TokenManager is automatically de-registered.
> DelegationTokenRenewer should be Singleton
> ------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch, hadoop-8852-v1.patch
>
>
> Updated description:
> DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
> Original issue:
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) DelegationTokenRenewer thread is not
stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated HADOOP-8852:
-------------------------------------
Attachment: hadoop-8852.patch
Moved {{DelegationTokenRenewer#start()}} to constructor.
Ran test-patch.sh locally - findbugs doesn't show any warnings.
{color:green}+1 overall{color}.
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version ) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) Remove DelegationTokenRenewer
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated HADOOP-8852:
-------------------------------------
Description:
Update 2:
DelegationTokenRenewer is not required. The filesystems that are using it already have Krb tickets and do not need tokens. Remove DelegationTokenRenewer and all the related logic from WebHdfs and Hftp filesystems.
Update1:
DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
Original issue:
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
was:
Updated description:
DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
Original issue:
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
> Remove DelegationTokenRenewer
> -----------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch, hadoop-8852-v1.patch
>
>
> Update 2:
> DelegationTokenRenewer is not required. The filesystems that are using it already have Krb tickets and do not need tokens. Remove DelegationTokenRenewer and all the related logic from WebHdfs and Hftp filesystems.
> Update1:
> DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
> Original issue:
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer should be
Singleton
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469929#comment-13469929 ]
Karthik Kambatla commented on HADOOP-8852:
------------------------------------------
Offline discussion with ATM has helped me understand this better.
Gist for others in my position (without a clear understanding of Kerberos):
# Initial authentication is indeed expensive and involves communicating with KDC, one gets Krb ticket upon authentication.
# Subsequent authentications with this Krb ticket are quick and don't need KDC communication.
# Hadoop tokens will not make these subsequent authentications any more efficient, because functionally this is same as the above authentication. Tokens are useful when one doesn't have tickets - e.g., MR tasks that execute on different nodes.
As per Daryn's suggestion, I ll go ahead and remove DelegationTokenRenewer and the relevant logic from these filesystems.
Thanks Daryn and ATM.
> DelegationTokenRenewer should be Singleton
> ------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch, hadoop-8852-v1.patch
>
>
> Updated description:
> DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
> Original issue:
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) DelegationTokenRenewer thread is not
stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated HADOOP-8852:
-------------------------------------
Affects Version/s: 2.0.0-alpha
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is
not stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464120#comment-13464120 ]
Karthik Kambatla commented on HADOOP-8852:
------------------------------------------
{{DelegationTokenRenewer}} fields in both filesystems are static fields. As stopping the thread corresponding to static field in close() does not seem like good practice, we should make them non-static.
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Tom White
> Assignee: Karthik Kambatla
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer should be
Singleton
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469569#comment-13469569 ]
Karthik Kambatla commented on HADOOP-8852:
------------------------------------------
Isn't it more efficient to renew a token than to get a new token every time? I agree we can get the first token lazily, instead of implicitly obtaining a token during {{FileSystem#init()}}
For FileSystems, is there a penalty for not renewing the tokens within the renewal period? If there is no penalty, it might actually be detrimental to periodically renew even in the absence of any activity.
On the other hand, if there is a penalty, I see some merit to this class. In a world with several filesystems, MR jobs, and other external entities, this class can enable renewing tokens sequentially in the order the systems come up for renewal.
> DelegationTokenRenewer should be Singleton
> ------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch, hadoop-8852-v1.patch
>
>
> Updated description:
> DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
> Original issue:
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) DelegationTokenRenewer thread is not
stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated HADOOP-8852:
-------------------------------------
Status: Patch Available (was: Open)
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) Remove DelegationTokenRenewer
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated HADOOP-8852:
-------------------------------------
Summary: Remove DelegationTokenRenewer (was: DelegationTokenRenewer should be Singleton)
> Remove DelegationTokenRenewer
> -----------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch, hadoop-8852-v1.patch
>
>
> Updated description:
> DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
> Original issue:
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is
not stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464235#comment-13464235 ]
Karthik Kambatla commented on HADOOP-8852:
------------------------------------------
By the way, in the little time I worked on Hadoop so far, I noticed several cases like this where {{close()}} does not stop _all_ the threads it spawns. I feel we should add more structure to ensure we do not miss any of them. Thoughts?
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is
not stopped when its filesystem is closed
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464311#comment-13464311 ]
Hadoop QA commented on HADOOP-8852:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12546759/hadoop-8852.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1533//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1533//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1533//console
This message is automatically generated.
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is
not stopped when its filesystem is closed
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464652#comment-13464652 ]
Tom White commented on HADOOP-8852:
-----------------------------------
Generally looks good.
* Rather than always starting a DelegationTokenRenewer thread in the HftpFileSystem/WebHdfsFileSystem constructor (which is a bad practice, and is why findbugs is complaining), you could do what WebHdfsFileSystem does already and only start the thread if security is enabled. See the addRenewAction() method on WebHdfsFileSystem.
* The 3000ms join timeout could be made into a constant. Perhaps you could add a shutdown method to DelegationTokenRenewer which encapsulates the interrupt and join.
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) WebHdfsFileSystem and HftpFileSystem
don't need delegation tokens
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated HADOOP-8852:
-------------------------------------
Description:
Parent JIRA to track the work of removing delegation tokens from these filesystems.
This JIRA has evolved from the initial issue of these filesystems not stopping the DelegationTokenRenewer thread they were creating.
After further investigation, Daryn pointed out - "If you can get a token, you don't need a token"! Hence, these filesystems shouldn't use delegation tokens.
Evolution of the JIRA is listed below:
Update 2:
DelegationTokenRenewer is not required. The filesystems that are using it already have Krb tickets and do not need tokens. Remove DelegationTokenRenewer and all the related logic from WebHdfs and Hftp filesystems.
Update1:
DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
Initial issue:
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
was:
Update 2:
DelegationTokenRenewer is not required. The filesystems that are using it already have Krb tickets and do not need tokens. Remove DelegationTokenRenewer and all the related logic from WebHdfs and Hftp filesystems.
Update1:
DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
Original issue:
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
Issue Type: Improvement (was: Bug)
Summary: WebHdfsFileSystem and HftpFileSystem don't need delegation tokens (was: Remove DelegationTokenRenewer)
> WebHdfsFileSystem and HftpFileSystem don't need delegation tokens
> -----------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch, hadoop-8852-v1.patch
>
>
> Parent JIRA to track the work of removing delegation tokens from these filesystems.
> This JIRA has evolved from the initial issue of these filesystems not stopping the DelegationTokenRenewer thread they were creating.
> After further investigation, Daryn pointed out - "If you can get a token, you don't need a token"! Hence, these filesystems shouldn't use delegation tokens.
> Evolution of the JIRA is listed below:
> Update 2:
> DelegationTokenRenewer is not required. The filesystems that are using it already have Krb tickets and do not need tokens. Remove DelegationTokenRenewer and all the related logic from WebHdfs and Hftp filesystems.
> Update1:
> DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
> Initial issue:
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) DelegationTokenRenewer thread is not
stopped when its filesystem is closed
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated HADOOP-8852:
-------------------------------------
Attachment: hadoop-8852-v1.patch
Here is a preliminary patch that makes {{DelegationTokenRenewer}} a singleton:
# Dropped the generic part of it
# capture the requirement to extend {{FileSystem}} by augmenting the API - {{public <T extends FileSystem & Renewable> void addRenewAction(final T fs)}}
# {{WebHdfsFileSystem}} and {{HftpFileSystem}} de-register from {{DelegationTokenRenewer}} in {{close()}}
Removing a FileSystem from the queue currently is linear in queue-size. I thought it might be okay, given that the queue should be small. Alternatively, we can maintain a list of removed FileSystems; {{run()}} could remove {{RenewAction}} (from {{queue.take()}}) if it were present in the remove-list.
This patch doesn't shutdown {{DelegationTokenRenewer}} when it can't renew tokens. Shutting down the thread alone can lead to interesting scenarios when new {{RenewAction}}s are added. Nullifying the {{INSTANCE}} after interrupting the thread will lead to {{FileSystem}}'s trying to use the stale reference? Not sure what the best way to handle this would be. Thoughts?
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch, hadoop-8852-v1.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is
not stopped when its filesystem is closed
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464708#comment-13464708 ]
Daryn Sharp commented on HADOOP-8852:
-------------------------------------
Hmm. I think we may have regressed functionality because I thought I Nicholas and I added lazy start to the renewer when the first token is added. While a little bit larger in scope, and it's something I've always been meaning to do, I'd suggest the renewer should:
* be a singleton to avoid multiple threads
* lazy starts when a token is added
* filesystem won't explicitly start the renewer or join with it
* when hftp or webhdfs is closed, it should cancel the token if {{initDelegationToken}} had to create a token
* the cancel methods should tell the renewer to stop renewing the canceled token. perhaps the renewer should be the one to cancel the token to avoid race conditions (fs cancels just before the renewer tries to renew)
* the renewer should shutdown if it runs out of tokens
> DelegationTokenRenewer thread is not stopped when its filesystem is closed
> --------------------------------------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch
>
>
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) DelegationTokenRenewer should be
Singleton
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated HADOOP-8852:
-------------------------------------
Description:
Updated description:
DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
Original issue:
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
was:HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
Summary: DelegationTokenRenewer should be Singleton (was: DelegationTokenRenewer thread is not stopped when its filesystem is closed)
> DelegationTokenRenewer should be Singleton
> ------------------------------------------
>
> Key: HADOOP-8852
> URL: https://issues.apache.org/jira/browse/HADOOP-8852
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch, hadoop-8852-v1.patch
>
>
> Updated description:
> DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal.
> Original issue:
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira