You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "yikf (Jira)" <ji...@apache.org> on 2021/06/12 00:59:00 UTC
[jira] [Comment Edited] (HADOOP-17728) Fix issue of the
StatisticsDataReferenceCleaner cleanUp
[ https://issues.apache.org/jira/browse/HADOOP-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17362205#comment-17362205 ]
yikf edited comment on HADOOP-17728 at 6/12/21, 12:58 AM:
----------------------------------------------------------
[~liuml07] [~Jim_Brennan]
I'm thinking, the issue is remove() blocks indefinitely, so this would never let it check Thread.interrupted() again if nothing is in the queue.
[code|[https://github.com/apache/hadoop/blob/352949d07002a8435a8ff67eecf88e4aa8bd5935/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L4009]]
FYI, It is better that have a timeout?
was (Author: kaifeiyi):
[~liuml07] [~Jim_Brennan]
I'm thinking, the issue is remove() blocks indefinitely, so this would never let it check Thread.interrupted() again if nothing is in the queue.
[https://github.com/apache/hadoop/blob/352949d07002a8435a8ff67eecf88e4aa8bd5935/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L4009]
It is better that have a timeout?
> Fix issue of the StatisticsDataReferenceCleaner cleanUp
> -------------------------------------------------------
>
> Key: HADOOP-17728
> URL: https://issues.apache.org/jira/browse/HADOOP-17728
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 3.2.1
> Reporter: yikf
> Assignee: yikf
> Priority: Minor
> Labels: pull-request-available, reverted
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Cleaner thread will be blocked if we remove reference from ReferenceQueue unless the `queue.enqueue` called.
> ----
> As shown below, We call ReferenceQueue.remove() now while cleanUp, Call chain as follow:
> *StatisticsDataReferenceCleaner#queue.remove() -> ReferenceQueue.remove(0) -> lock.wait(0)*
> But, lock.notifyAll is called when queue.enqueue only, so Cleaner thread will be blocked.
>
> ThreadDump:
> {code:java}
> "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f7afc088800 nid=0x2119 in Object.wait() [0x00007f7b00230000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000c00c2f58> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:502)
> at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
> - locked <0x00000000c00c2f58> (a java.lang.ref.Reference$Lock)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org