You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Harsh J (Resolved) (JIRA)" <ji...@apache.org> on 2012/01/16 11:20:39 UTC
[jira] [Resolved] (MAPREDUCE-568) Streaming causes a lot of broken
pipes which leads to job failure
[ https://issues.apache.org/jira/browse/MAPREDUCE-568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J resolved MAPREDUCE-568.
-------------------------------
Resolution: Invalid
This mostly happens with faults in the user scripts. Closing out as its not really a bug. We do not see this in regular streaming jobs all the time as the description seems to go.
> Streaming causes a lot of broken pipes which leads to job failure
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-568
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-568
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Reporter: Mahadev konar
>
> java.io.IOException: Broken pipe
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:260)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:124)
> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
> at org.apache.hadoop.streaming.PipeMapper.map(PipeMapper.java:77)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:215)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1247)
> at org.apache.hadoop.streaming.PipeMapper.map(PipeMapper.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:215)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1247)
> A lot of these errors occur in streaming. THis leads to job failure. I still am not sure what the reason might be of these errors, but these erros occur too often. I will try and invetigate more to see what the reason might be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira