[jira] Updated: (HADOOP-1771) streaming hang when IOException in MROutputThread. (NPE)

["lohit vijayarenu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

lohit vijayarenu updated HADOOP-1771:
-------------------------------------

    Attachment: h-1771-3.patch

Thanks to Koji, while debugging a similar problem, we saw similar case where MROutputThread was done, but  the map task was still hung
Upon killing the streaming map executable, we saw there was a problem with the thread so it had terminated. 
Looks like the streaming map executable was still trying to write and was hung. 

The problem could be that clientIn_ and clientErr_ are not closed when their thread is done, which causes this problem.
Attached is an updated patch, which makes sure it will close the streams when their threads exit.

Please could anyone review/comment.
Thanks.

> streaming hang when IOException in MROutputThread. (NPE)
> --------------------------------------------------------
>
>                 Key: HADOOP-1771
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1771
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/streaming
>    Affects Versions: 0.13.1
>            Reporter: Koji Noguchi
>            Assignee: Koji Noguchi
>            Priority: Minor
>             Fix For: 0.15.0
>
>         Attachments: h-1771-2.patch, h-1771-3.patch, h-1771.patch
>
>
> One streaming task hang and had stderr userlog as follows.
> {code}
> Exception in thread "Thread-5" java.lang.NullPointerException
>          at java.lang.Throwable.printStackTrace(Throwable.java:460)
>          at org.apache.hadoop.streaming.PipeMapRed$MROutputThread.run(PipeMapRed.java:352)
> {code}
> In PipeMapRed.java
> {code}
> 351       } catch (IOException io) {
> 352         io.printStackTrace(log_);
> 353         outerrThreadsThrowable = io;
> {code}
> I guess log_ is  Null... Should call logStackTrace.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.