You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Prashant Sharma (Created) (JIRA)" <ji...@apache.org> on 2011/10/19 11:50:11 UTC
[jira] [Created] (MAPREDUCE-3211) Fetch failures if used ephemeral
port for property "mapreduce.shuffle.port".
Fetch failures if used ephemeral port for property "mapreduce.shuffle.port".
-----------------------------------------------------------------------------
Key: MAPREDUCE-3211
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3211
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: job submission, mrv2
Affects Versions: 0.24.0
Environment: Linux
Reporter: Prashant Sharma
Assignee: Prashant Sharma
Fix For: 0.24.0
To reproduce the bug.
Use the following property in mapred-site.xml
<configuration>
<property>
<name> mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name> mapreduce.shuffle.port</name>
<value>0</value>
</property>
</configuration>
and following in yarn-site.xml
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce.shuffle</value>
<description>shuffle service that needs to be set for Map Reduce to run </description>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
Then try to start the yarn daemons.
And submit the job. (which would eventually fail).
Also observe NM logs. It says shuffle handler service started at port 0; :P
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3211) Fetch failures if used ephemeral
port for property "mapreduce.shuffle.port".
Posted by "Prashant Sharma (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma resolved MAPREDUCE-3211.
----------------------------------------
Resolution: Duplicate
Hadoop Flags: Reviewed
MAPREDUCE-2986
> Fetch failures if used ephemeral port for property "mapreduce.shuffle.port".
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3211
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3211
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, mrv2
> Affects Versions: 0.24.0
> Environment: Linux
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Labels: ShuffleHandler
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3211.patch, MAPREDUCE-3211.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> To reproduce the bug.
> Use the following property in mapred-site.xml
> <configuration>
> <property>
> <name> mapreduce.framework.name</name>
> <value>yarn</value>
> </property>
> <property>
> <name> mapreduce.shuffle.port</name>
> <value>0</value>
> </property>
> </configuration>
> and following in yarn-site.xml
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> <description>shuffle service that needs to be set for Map Reduce to run </description>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> Then try to start the yarn daemons.
> And submit the job. (which would eventually fail).
> Also observe NM logs. It says shuffle handler service started at port 0; :P
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira