You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2009/04/27 05:40:30 UTC
[jira] Created: (HADOOP-5746) Errors encountered in MROutputThread
after the last map/reduce call can go undetected
Errors encountered in MROutputThread after the last map/reduce call can go undetected
-------------------------------------------------------------------------------------
Key: HADOOP-5746
URL: https://issues.apache.org/jira/browse/HADOOP-5746
Project: Hadoop Core
Issue Type: Bug
Components: contrib/streaming
Reporter: Devaraj Das
Fix For: 0.21.0
The framework map/reduce bridge methods make a check at the beginning of the respective methods whether _MROutputThread_ encountered an exception while writing keys/values that the streaming process emitted. However, if the exception happens in _MROutputThread_ after the last call to the map/reduce method, the exception goes undetected. An example of such an exception is an exception from the _DFSClient_ that fails to write to a file on the HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5746) Errors encountered in MROutputThread
after the last map/reduce call can go undetected
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-5746:
--------------------------------
Attachment: 5746.1.patch
Ok here is an early version of the patch (no testcase yet). The patch applies on 0.18 as well.
> Errors encountered in MROutputThread after the last map/reduce call can go undetected
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-5746
> URL: https://issues.apache.org/jira/browse/HADOOP-5746
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Devaraj Das
> Fix For: 0.21.0
>
> Attachments: 5746.1.patch
>
>
> The framework map/reduce bridge methods make a check at the beginning of the respective methods whether _MROutputThread_ encountered an exception while writing keys/values that the streaming process emitted. However, if the exception happens in _MROutputThread_ after the last call to the map/reduce method, the exception goes undetected. An example of such an exception is an exception from the _DFSClient_ that fails to write to a file on the HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5746) Errors encountered in MROutputThread
after the last map/reduce call can go undetected
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-5746:
-------------------------------
Release Note: If the child (streaming) process returns successfully and the MROutputThread throws an error, there was no way to detect that as all the IOExceptions was ignored. Such issues can occur when DFS clients were closed etc. Now a check for errors (in threads) is made before finishing off the task and an exception is thrown that fails he task.
> Errors encountered in MROutputThread after the last map/reduce call can go undetected
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-5746
> URL: https://issues.apache.org/jira/browse/HADOOP-5746
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Devaraj Das
> Assignee: Amar Kamat
> Fix For: 0.20.1
>
> Attachments: 5746-reproduce.1.patch, 5746-testcase.patch, 5746.1.patch, 5746.6.patch
>
>
> The framework map/reduce bridge methods make a check at the beginning of the respective methods whether _MROutputThread_ encountered an exception while writing keys/values that the streaming process emitted. However, if the exception happens in _MROutputThread_ after the last call to the map/reduce method, the exception goes undetected. An example of such an exception is an exception from the _DFSClient_ that fails to write to a file on the HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5746) Errors encountered in MROutputThread
after the last map/reduce call can go undetected
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-5746:
-------------------------------
Attachment: 5746-testcase.patch
5746-reproduce.1.patch
5746.6.patch
Attaching the patch [5746.6.patch]. It looks like its a very timing issue to reproduce/test this bug. The problem occurs when the pipe process finishes off and then the output thread cranks up with some exception (like fs errors).
Attaching a framework change [5746-reproduce.1.patch] and a testcase [/5746-testcase.patch] to verify the fix. The caller of {{PipeMapRed.waitOutputThreads()}} i.e {{PipeMapRed.mapRedFinished()}} simply ignores IOException hence changed the exception to {{RuntimeException}}. I dont know why PipeMapRed.mapRedFinished() ignores IOException. But for now I have kept it as it is.
> Errors encountered in MROutputThread after the last map/reduce call can go undetected
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-5746
> URL: https://issues.apache.org/jira/browse/HADOOP-5746
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Devaraj Das
> Fix For: 0.21.0
>
> Attachments: 5746-reproduce.1.patch, 5746-testcase.patch, 5746.1.patch, 5746.6.patch
>
>
> The framework map/reduce bridge methods make a check at the beginning of the respective methods whether _MROutputThread_ encountered an exception while writing keys/values that the streaming process emitted. However, if the exception happens in _MROutputThread_ after the last call to the map/reduce method, the exception goes undetected. An example of such an exception is an exception from the _DFSClient_ that fails to write to a file on the HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-5746) Errors encountered in MROutputThread
after the last map/reduce call can go undetected
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das resolved HADOOP-5746.
---------------------------------
Resolution: Fixed
Fix Version/s: (was: 0.21.0)
0.20.1
Assignee: Amar Kamat
Hadoop Flags: [Reviewed]
I just committed this. Thanks, Amar!
> Errors encountered in MROutputThread after the last map/reduce call can go undetected
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-5746
> URL: https://issues.apache.org/jira/browse/HADOOP-5746
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Devaraj Das
> Assignee: Amar Kamat
> Fix For: 0.20.1
>
> Attachments: 5746-reproduce.1.patch, 5746-testcase.patch, 5746.1.patch, 5746.6.patch
>
>
> The framework map/reduce bridge methods make a check at the beginning of the respective methods whether _MROutputThread_ encountered an exception while writing keys/values that the streaming process emitted. However, if the exception happens in _MROutputThread_ after the last call to the map/reduce method, the exception goes undetected. An example of such an exception is an exception from the _DFSClient_ that fails to write to a file on the HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5746) Errors encountered in
MROutputThread after the last map/reduce call can go undetected
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716259#action_12716259 ]
Amar Kamat commented on HADOOP-5746:
------------------------------------
Result of test-patch
{code}
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[code}
Streaming tests passed on my box except TestStreamingExitStatus which fails even on trunk.
> Errors encountered in MROutputThread after the last map/reduce call can go undetected
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-5746
> URL: https://issues.apache.org/jira/browse/HADOOP-5746
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Devaraj Das
> Fix For: 0.21.0
>
> Attachments: 5746-reproduce.1.patch, 5746-testcase.patch, 5746.1.patch, 5746.6.patch
>
>
> The framework map/reduce bridge methods make a check at the beginning of the respective methods whether _MROutputThread_ encountered an exception while writing keys/values that the streaming process emitted. However, if the exception happens in _MROutputThread_ after the last call to the map/reduce method, the exception goes undetected. An example of such an exception is an exception from the _DFSClient_ that fails to write to a file on the HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.