You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Christian Kunz (JIRA)" <ji...@apache.org> on 2007/07/07 03:21:04 UTC
[jira] Created: (HADOOP-1573) Support for 0 reducers in PIPES
Support for 0 reducers in PIPES
-------------------------------
Key: HADOOP-1573
URL: https://issues.apache.org/jira/browse/HADOOP-1573
Project: Hadoop
Issue Type: Bug
Components: pipes
Affects Versions: 0.14.0
Reporter: Christian Kunz
Under pipes, when specifying number of reducers to zero, all map tasks fail:
java.io.IOException: pipe child exception
at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:124)
at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:82)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1763)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.net.SocketInputStream.read(SocketInputStream.java:182)
at java.io.DataInputStream.readByte(DataInputStream.java:248)
at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:313)
at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:335)
at org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:101)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1573) Support for 0 reducers in PIPES
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529840 ]
Hudson commented on HADOOP-1573:
--------------------------------
Integrated in Hadoop-Nightly #248 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/248/])
> Support for 0 reducers in PIPES
> -------------------------------
>
> Key: HADOOP-1573
> URL: https://issues.apache.org/jira/browse/HADOOP-1573
> Project: Hadoop
> Issue Type: Bug
> Components: pipes
> Affects Versions: 0.14.0
> Reporter: Christian Kunz
> Assignee: Owen O'Malley
> Fix For: 0.15.0
>
> Attachments: pipe-no-reduce.patch
>
>
> Under pipes, when specifying number of reducers to zero, all map tasks fail:
> java.io.IOException: pipe child exception
> at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:124)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:82)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1763)
> Caused by: java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:168)
> at java.net.SocketInputStream.read(SocketInputStream.java:182)
> at java.io.DataInputStream.readByte(DataInputStream.java:248)
> at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:313)
> at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:335)
> at org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:101)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1573) Support for 0 reducers in PIPES
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1573:
----------------------------------
Attachment: pipe-no-reduce.patch
This patch fixes the problem and adds a unit test to ensure that it works.
> Support for 0 reducers in PIPES
> -------------------------------
>
> Key: HADOOP-1573
> URL: https://issues.apache.org/jira/browse/HADOOP-1573
> Project: Hadoop
> Issue Type: Bug
> Components: pipes
> Affects Versions: 0.14.0
> Reporter: Christian Kunz
> Fix For: 0.15.0
>
> Attachments: pipe-no-reduce.patch
>
>
> Under pipes, when specifying number of reducers to zero, all map tasks fail:
> java.io.IOException: pipe child exception
> at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:124)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:82)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1763)
> Caused by: java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:168)
> at java.net.SocketInputStream.read(SocketInputStream.java:182)
> at java.io.DataInputStream.readByte(DataInputStream.java:248)
> at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:313)
> at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:335)
> at org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:101)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (HADOOP-1573) Support for 0 reducers in PIPES
Posted by Nigel Daley <nd...@yahoo-inc.com>.
Owen, the Solaris build machine doesn't compile the C++ code nor run
the C++ tests. Can you set up the necessary packages on the machine?
On Sep 19, 2007, at 1:43 AM, Hadoop QA (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/HADOOP-1573?
> page=com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel#action_12528694 ]
>
> Hadoop QA commented on HADOOP-1573:
> -----------------------------------
>
> +1 overall. Here are the results of testing the latest attachment
> http://issues.apache.org/jira/secure/attachment/12366154/pipe-no-
> reduce.patch
> against trunk revision r577171.
>
> @author +1. The patch does not contain any @author tags.
>
> javadoc +1. The javadoc tool did not generate any warning
> messages.
>
> javac +1. The applied patch does not generate any new compiler
> warnings.
>
> findbugs +1. The patch does not introduce any new Findbugs
> warnings.
>
> core tests +1. The patch passed core unit tests.
>
> contrib tests +1. The patch passed contrib unit tests.
>
> Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-
> Patch/789/testReport/
> Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/
> Hadoop-Patch/789/artifact/trunk/build/test/findbugs/
> newPatchFindbugsWarnings.html
> Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/
> Hadoop-Patch/789/artifact/trunk/build/test/checkstyle-errors.html
> Console output: http://lucene.zones.apache.org:8080/hudson/job/
> Hadoop-Patch/789/console
>
> This message is automatically generated.
>
>> Support for 0 reducers in PIPES
>> -------------------------------
>>
>> Key: HADOOP-1573
>> URL: https://issues.apache.org/jira/browse/
>> HADOOP-1573
>> Project: Hadoop
>> Issue Type: Bug
>> Components: pipes
>> Affects Versions: 0.14.0
>> Reporter: Christian Kunz
>> Assignee: Owen O'Malley
>> Fix For: 0.15.0
>>
>> Attachments: pipe-no-reduce.patch
>>
>>
>> Under pipes, when specifying number of reducers to zero, all map
>> tasks fail:
>> java.io.IOException: pipe child exception
>> at org.apache.hadoop.mapred.pipes.Application.abort
>> (Application.java:124)
>> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run
>> (PipesMapRunner.java:82)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main
>> (TaskTracker.java:1763)
>> Caused by: java.net.SocketException: Connection reset
>> at java.net.SocketInputStream.read(SocketInputStream.java:168)
>> at java.net.SocketInputStream.read(SocketInputStream.java:182)
>> at java.io.DataInputStream.readByte(DataInputStream.java:248)
>> at org.apache.hadoop.io.WritableUtils.readVLong
>> (WritableUtils.java:313)
>> at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:
>> 335)
>> at org.apache.hadoop.mapred.pipes.BinaryProtocol
>> $UplinkReaderThread.run(BinaryProtocol.java:101)
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
[jira] Commented: (HADOOP-1573) Support for 0 reducers in PIPES
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12528694 ]
Hadoop QA commented on HADOOP-1573:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12366154/pipe-no-reduce.patch
against trunk revision r577171.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/789/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/789/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/789/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/789/console
This message is automatically generated.
> Support for 0 reducers in PIPES
> -------------------------------
>
> Key: HADOOP-1573
> URL: https://issues.apache.org/jira/browse/HADOOP-1573
> Project: Hadoop
> Issue Type: Bug
> Components: pipes
> Affects Versions: 0.14.0
> Reporter: Christian Kunz
> Assignee: Owen O'Malley
> Fix For: 0.15.0
>
> Attachments: pipe-no-reduce.patch
>
>
> Under pipes, when specifying number of reducers to zero, all map tasks fail:
> java.io.IOException: pipe child exception
> at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:124)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:82)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1763)
> Caused by: java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:168)
> at java.net.SocketInputStream.read(SocketInputStream.java:182)
> at java.io.DataInputStream.readByte(DataInputStream.java:248)
> at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:313)
> at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:335)
> at org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:101)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1573) Support for 0 reducers in PIPES
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-1573:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Owen!
> Support for 0 reducers in PIPES
> -------------------------------
>
> Key: HADOOP-1573
> URL: https://issues.apache.org/jira/browse/HADOOP-1573
> Project: Hadoop
> Issue Type: Bug
> Components: pipes
> Affects Versions: 0.14.0
> Reporter: Christian Kunz
> Assignee: Owen O'Malley
> Fix For: 0.15.0
>
> Attachments: pipe-no-reduce.patch
>
>
> Under pipes, when specifying number of reducers to zero, all map tasks fail:
> java.io.IOException: pipe child exception
> at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:124)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:82)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1763)
> Caused by: java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:168)
> at java.net.SocketInputStream.read(SocketInputStream.java:182)
> at java.io.DataInputStream.readByte(DataInputStream.java:248)
> at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:313)
> at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:335)
> at org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:101)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1573) Support for 0 reducers in PIPES
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley reassigned HADOOP-1573:
-------------------------------------
Assignee: Owen O'Malley
> Support for 0 reducers in PIPES
> -------------------------------
>
> Key: HADOOP-1573
> URL: https://issues.apache.org/jira/browse/HADOOP-1573
> Project: Hadoop
> Issue Type: Bug
> Components: pipes
> Affects Versions: 0.14.0
> Reporter: Christian Kunz
> Assignee: Owen O'Malley
> Fix For: 0.15.0
>
> Attachments: pipe-no-reduce.patch
>
>
> Under pipes, when specifying number of reducers to zero, all map tasks fail:
> java.io.IOException: pipe child exception
> at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:124)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:82)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1763)
> Caused by: java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:168)
> at java.net.SocketInputStream.read(SocketInputStream.java:182)
> at java.io.DataInputStream.readByte(DataInputStream.java:248)
> at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:313)
> at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:335)
> at org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:101)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1573) Support for 0 reducers in PIPES
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1573:
----------------------------------
Fix Version/s: 0.15.0
Status: Patch Available (was: Open)
> Support for 0 reducers in PIPES
> -------------------------------
>
> Key: HADOOP-1573
> URL: https://issues.apache.org/jira/browse/HADOOP-1573
> Project: Hadoop
> Issue Type: Bug
> Components: pipes
> Affects Versions: 0.14.0
> Reporter: Christian Kunz
> Assignee: Owen O'Malley
> Fix For: 0.15.0
>
> Attachments: pipe-no-reduce.patch
>
>
> Under pipes, when specifying number of reducers to zero, all map tasks fail:
> java.io.IOException: pipe child exception
> at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:124)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:82)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1763)
> Caused by: java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:168)
> at java.net.SocketInputStream.read(SocketInputStream.java:182)
> at java.io.DataInputStream.readByte(DataInputStream.java:248)
> at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:313)
> at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:335)
> at org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:101)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.