You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Zhihua Deng (Jira)" <ji...@apache.org> on 2020/11/19 01:27:00 UTC
[jira] [Commented] (MAPREDUCE-7307) Potential thread leak in
LocatedFileStatusFetcher
[ https://issues.apache.org/jira/browse/MAPREDUCE-7307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235088#comment-17235088 ]
Zhihua Deng commented on MAPREDUCE-7307:
----------------------------------------
[~stevel@apache.org], [~jlowe] could you please take a look? thanks in advance!**
> Potential thread leak in LocatedFileStatusFetcher
> -------------------------------------------------
>
> Key: MAPREDUCE-7307
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7307
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
> Reporter: Zhihua Deng
> Assignee: Zhihua Deng
> Priority: Major
>
> We see that when using LocatedFileStatusFetcher to get file infos In parallel, if the listStatus thread is interrupted, the executor service in LocatedFileStatusFetcher is left unclosed, the thread stack will like this:
> {noformat}
> "GetFileInfo #63" #125 daemon prio=5 os_prio=0 tid=0x00007f6198106800 nid=0x881 waiting on condition [0x00007f60d9fde000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000082e810a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){noformat}
> This caused by if condition.await() throws InterruptedException, the method `shutDownNow` for the executor service would not be called as a result, should move such resource releasing call into the finally block.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org