You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Koji Noguchi (JIRA)" <ji...@apache.org> on 2007/03/23 22:32:32 UTC
[jira] Created: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
streaming hang. (PipeMapRed$MROutputThread gone)
------------------------------------------------
Key: HADOOP-1154
URL: https://issues.apache.org/jira/browse/HADOOP-1154
Project: Hadoop
Issue Type: Bug
Components: contrib/streaming
Reporter: Koji Noguchi
One streaming reducer ('cat') hang,
'/bin/cat' reducer waiting in
0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
strace showed that it's waiting in write.
$ strace -p 24587
Process 24587 attached - interrupt to quit
write(1, "________ "..., 138 <unfinished ...>
Process 24587 detached
When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Koji Noguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Noguchi updated HADOOP-1154:
---------------------------------
Status: Patch Available (was: Open)
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Attachments: 1154-2.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Koji Noguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Noguchi updated HADOOP-1154:
---------------------------------
Attachment: 1154-2.patch
Sorry. My original patch didn't compile with jdk1.5.
Changed to use StringUtils.stringifyException.
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Attachments: 1154-2.patch, 1154.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-1154:
------------------------------
Resolution: Fixed
Fix Version/s: 0.13.0
Status: Resolved (was: Patch Available)
I've just committed this. Thanks Koji!
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Fix For: 0.13.0
>
> Attachments: 1154-2.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12488046 ]
Hadoop QA commented on HADOOP-1154:
-----------------------------------
-1
2 attempts failed to build and test the latest attachment http://issues.apache.org/jira/secure/attachment/12355173/1154.patch against trunk revision r527417.
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
Results are at http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/27/console
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Attachments: 1154.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Koji Noguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Noguchi reassigned HADOOP-1154:
------------------------------------
Assignee: Koji Noguchi
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Attachments: 1154.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Koji Noguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Noguchi updated HADOOP-1154:
---------------------------------
Attachment: PipeReducer.jstack
Attaching jstack of the PipeReducer process
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Attachments: PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489694 ]
Hadoop QA commented on HADOOP-1154:
-----------------------------------
Integrated in Hadoop-Nightly #61 (See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/61/)
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Fix For: 0.13.0
>
> Attachments: 1154-2.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Koji Noguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Noguchi updated HADOOP-1154:
---------------------------------
Status: Patch Available (was: Open)
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Attachments: 1154.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12488765 ]
Hadoop QA commented on HADOOP-1154:
-----------------------------------
-1, build or testing failed
2 attempts failed to build and test the latest attachment http://issues.apache.org/jira/secure/attachment/12355524/1154-2.patch against trunk revision r528230.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/45/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/45/console
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Attachments: 1154-2.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12487682 ]
Runping Qi commented on HADOOP-1154:
------------------------------------
+1
It is reasonable to fail a streaming task if its the threads readingfrom/writing to the streaming process fails.
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Attachments: 1154.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12488790 ]
Hadoop QA commented on HADOOP-1154:
-----------------------------------
-1, build or testing failed
2 attempts failed to build and test the latest attachment http://issues.apache.org/jira/secure/attachment/12355524/1154-2.patch against trunk revision r528230.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/47/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/47/console
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Attachments: 1154-2.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Koji Noguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Noguchi updated HADOOP-1154:
---------------------------------
Attachment: 1154.patch
This patch would fail the task when MROutput/ErrThread throws a Throwable.
Right now, I propagate the Throwable by including in the IOException.
This is probably not the right approach. I'd appreciate any suggestions.
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Attachments: 1154.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1154:
---------------------------------
Status: Open (was: Patch Available)
This patch fails to compile.
[javac] src/contrib/streaming/src/java/org/apache/hadoop/streaming/PipeReducer.java:78: cannot find symbol
[javac] symbol : constructor IOException(java.lang.String,java.lang.Throwable)
[javac] location: class java.io.IOException
[javac] throw new IOException ("MROutput/MRErrThread failed",
[javac] ^
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Attachments: 1154.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1154) streaming hang.
(PipeMapRed$MROutputThread gone)
Posted by "Koji Noguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Noguchi updated HADOOP-1154:
---------------------------------
Attachment: (was: 1154.patch)
> streaming hang. (PipeMapRed$MROutputThread gone)
> ------------------------------------------------
>
> Key: HADOOP-1154
> URL: https://issues.apache.org/jira/browse/HADOOP-1154
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Koji Noguchi
> Assigned To: Koji Noguchi
> Attachments: 1154-2.patch, PipeReducer.jstack
>
>
> One streaming reducer ('cat') hang,
> '/bin/cat' reducer waiting in
> 0 S _____ 24587 24557 0 75 0 - 631 pipe_w Mar22 ? 00:00:02 /bin/cat
> strace showed that it's waiting in write.
> $ strace -p 24587
> Process 24587 attached - interrupt to quit
> write(1, "________ "..., 138 <unfinished ...>
> Process 24587 detached
> When I looked at the jstack of ReduceTask process, I couldn't find the PipeMapRed$MROutputThread.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.