You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Ja...@McAfee.com on 2013/11/13 13:49:25 UTC

FW: meaning of cryptic yarn nodemanager message: org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices

Good day - please could anyone explain the meaning of this message in yarn-${USER}-nodemanager-${HOST}.log:

"
2013-11-13 12:17:29,704 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
"
Love the creative spelling ! ('Auxilurary' should be 'Auxiliary' ? 'unless the refer' should be 'unless they refer' ?)

I guess this is because of the setting in yarn-site.xml :

"
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
    <description>shuffle service that needs to be set for Map Reduce to run </description>  </property> "
that I installed according to the instructions at:  
http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/SingleCluster.html

Why do these instructions specify this setting it may cause "issues" ? What sort of issues may be caused ? 
This sentence is nonsensical:
  " Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config."
What are the "tools" being referred to here ?
What "may have issues" ?
What are the "issues" ? 
What  must refer to the name in the config ?
Should I change the name to "httpshuffle"  or not ?

Any ideas / suggestions gratefully received.

Regards,
Jason