You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Daryn Sharp (JIRA)" <ji...@apache.org> on 2013/05/07 19:11:17 UTC
[jira] [Updated] (HADOOP-9549) WebHdfsFileSystem hangs on close()
[ https://issues.apache.org/jira/browse/HADOOP-9549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HADOOP-9549:
--------------------------------
Attachment: HADOOP-9549.patch
The problem is the unnecessary synchronization on the thread-safe DelayQueue. The daemon thread synched while trying to take from the queue. This manifested as two problems:
# After creating a filesystem, the thread was blocked waiting for the token in the queue to be ready for renewal. Closing the fs and removing the token blocked on the queue waiting for the thread to renew the token. It would have "worked" when the token renewed after about 21 hours...
# Creating another webhdfs and/or hftp instance would block trying to add its token to the queue while the thread waited for the 1st fs token to be ready for renewal.
The synch on the queue was to allow removal of tokens by iterating the queue. The queue needed to be iterated, versus calling {{remove}}, because the {{RenewAction}} objects improperly implemented {{hashCode}} and {{equals}}.
I had a much cleaner overall implementation but it required changes to webhdfs and hftp. This patch should be good enough for now. I also added a bunch of missing tests.
> WebHdfsFileSystem hangs on close()
> ----------------------------------
>
> Key: HADOOP-9549
> URL: https://issues.apache.org/jira/browse/HADOOP-9549
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 2.0.4-alpha
> Reporter: Kihwal Lee
> Assignee: Daryn Sharp
> Priority: Blocker
> Attachments: HADOOP-9549.patch
>
>
> When close() is called via fs shoutdown hook, the synchronized method, removeRenewAction() hangs. This is because DelegationTokenRenewer calls DelayQueue.take() inside a synchronized block. Since this is a blocking call, it hangs forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira