You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ryan Blue (JIRA)" <ji...@apache.org> on 2016/02/19 18:35:18 UTC
[jira] [Updated] (MAPREDUCE-6639) Process hangs in
LocatedFileStatusFetcher if FileSystem.get throws
[ https://issues.apache.org/jira/browse/MAPREDUCE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan Blue updated MAPREDUCE-6639:
---------------------------------
Description:
ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable thread functions, [{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306], doesn't catch exceptions (the others do). When an exception is thrown, the thread exists and doesn't signal the error to the calling thread, which continues waiting to be signaled. This can happen when a FS implementation cannot be found.
The solution is to use a try-catch around the implementation of {{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when any exception is caught.
was:
ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable thread functions, [ProcessInitialInputPathCallable|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306], doesn't catch exceptions (the others do). When an exception is thrown, the thread exists and doesn't signal the error to the calling thread, which continues waiting to be signaled. This can happen when a FS implementation cannot be found.
The solution is to use a try-catch around the implementation of {{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when any exception is caught.
> Process hangs in LocatedFileStatusFetcher if FileSystem.get throws
> ------------------------------------------------------------------
>
> Key: MAPREDUCE-6639
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6639
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Reporter: Ryan Blue
>
> ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable thread functions, [{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306], doesn't catch exceptions (the others do). When an exception is thrown, the thread exists and doesn't signal the error to the calling thread, which continues waiting to be signaled. This can happen when a FS implementation cannot be found.
> The solution is to use a try-catch around the implementation of {{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when any exception is caught.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)