You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org> on 2015/05/09 00:08:01 UTC
[jira] [Resolved] (MAPREDUCE-5716) yarn framework occurs
Shell$ExitCodeException
[ https://issues.apache.org/jira/browse/MAPREDUCE-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli resolved MAPREDUCE-5716.
------------------------------------------------
Resolution: Invalid
Exit code 127 usually means that the command is not found / the file doesn't exist. May be your setup has issues w.r.t container-executor binary.
Let's take it to the user mailing-lists and investigate there before we declare there is a bug. Closing this as invalid for now.
> yarn framework occurs Shell$ExitCodeException
> ---------------------------------------------
>
> Key: MAPREDUCE-5716
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5716
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 2.2.0
> Environment: hadoop2.2.0,linux redhat 5.5
> Reporter: dimmacro
> Assignee: dimmacro
> Priority: Critical
>
> I use hadoop2.2.0 to run map-reduce task,at first I set the property "mapreduce.framework.name" with "local" in mapred-site.xml,everything goes fine ,and there is no exception,but when I run the mapreduce task on server cluster,setting the "mapreduce.framework.name" property value with "yarn",it shows exception belows:
> 2014-01-10 14:51:03,131 INFO ContainersLauncher #0 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer: [/home/hadoop/hadoop-2.2.0/bin/container-executor, hadoop, 1, application_1389336249740_0001, container_1389336249740_000
> _01_000013, /new/hadoop/data/tmp/nm-local-dir/usercache/hadoop/appcache/application_1389336249740_0001/container_1389336249740_0001_01_000013, /new/hadoop/data/tmp/nm-local-dir/nmPrivate/application_1389336249740_0001/container_1389336249740_0001_01_00001
> /launch_container.sh, /new/hadoop/data/tmp/nm-local-dir/nmPrivate/application_1389336249740_0001/container_1389336249740_0001_01_000013/container_1389336249740_0001_01_000013.tokens, /new/hadoop/data/tmp/nm-local-dir/nmPrivate/container_1389336249740_0001
> 01_000013.pid, /new/hadoop/data/tmp/nm-local-dir, /home/hadoop/hadoop-2.2.0/logs/userlogs, cgroups=none]
> 2014-01-10 14:51:03,134 INFO ContainersLauncher #2 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer: [/home/hadoop/hadoop-2.2.0/bin/container-executor, hadoop, 1, application_1389336249740_0001, container_1389336249740_000
> _01_000014, /new/hadoop/data/tmp/nm-local-dir/usercache/hadoop/appcache/application_1389336249740_0001/container_1389336249740_0001_01_000014, /new/hadoop/data/tmp/nm-local-dir/nmPrivate/application_1389336249740_0001/container_1389336249740_0001_01_00001
> /launch_container.sh, /new/hadoop/data/tmp/nm-local-dir/nmPrivate/application_1389336249740_0001/container_1389336249740_0001_01_000014/container_1389336249740_0001_01_000014.tokens, /new/hadoop/data/tmp/nm-local-dir/nmPrivate/container_1389336249740_0001
> 01_000014.pid, /new/hadoop/data/tmp/nm-local-dir, /home/hadoop/hadoop-2.2.0/logs/userlogs, cgroups=none]
> 2014-01-10 14:51:03,142 INFO AsyncDispatcher event handler org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1389336249740_0001_01_000013 transitioned from LOCALIZED to RUNNING
> 2014-01-10 14:51:03,144 INFO AsyncDispatcher event handler org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1389336249740_0001_01_000014 transitioned from LOCALIZED to RUNNING
> 2014-01-10 14:51:03,144 WARN ContainersLauncher #4 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch with container ID: container_1389336249740_0001_01_000016 and exit code: 127
> org.apache.hadoop.util.Shell$ExitCodeException:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
> at org.apache.hadoop.util.Shell.run(Shell.java:379)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
> at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:252)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> 2014-01-10 14:51:03,161 WARN ContainersLauncher #2 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container container_1389336249740_0001_01_000014 is : 127
> my mapred-site.xml content:
> <configuration>
> <property>
> <name>mapreduce.jobtracker.address</name>
> <value>local</value>
> <description>The host and port that the MapReduce job tracker runs
> at. If "local", then jobs are run in-process as a single map
> and reduce task.
> </description>
> </property>
> <property>
> <name>mapreduce.jobtracker.http.address</name>
> <value>121server:50030</value>
> <description>
> The job tracker http server address and port the server will listen on.
> If the port is 0 then the server will start on a free port.
> </description>
> </property>
> <property>
> <name>mapreduce.job.maps</name>
> <value>40</value>
> <description>The default number of map tasks per job.
> Ignored when mapreduce.jobtracker.address is "local".
> </description>
> </property>
> <property>
> <name>mapreduce.framework.name</name>
> <value>classic</value>
> </property>
> </configuration>
> my yarn-site.xml content:
> <configuration>
> <!-- Site specific YARN configuration properties -->
> <property>
> <name>yarn.resourcemanager.resource-tracker.address</name>
> <value>152server:8990</value>
> <description>host is the hostname of the resource manager and
> port is the port on which the NodeManagers contact the Resource Manager.
> </description>
> </property>
> <property>
> <name>yarn.resourcemanager.scheduler.address</name>
> <value>152server:8991</value>
> <description>host is the hostname of the resourcemanager and port is the port
> on which the Applications in the cluster talk to the Resource Manager.
> </description>
> </property>
> <property>
> <name>yarn.resourcemanager.scheduler.class</name> <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
> <description>In case you do not want to use the default scheduler</description>
> </property>
> <property>
> <name>yarn.resourcemanager.address</name>
> <value>152server:8993</value>
> <description>the host is the hostname of the ResourceManager and the port is the port on
> which the clients can talk to the Resource Manager. </description>
> </property>
> <property>
> <description>The address of the RM web application.</description>
> <name>yarn.resourcemanager.webapp.address</name>
> <value>152server:18088</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce_shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>yarn.nodemanager.resource.memory-mb</name>
> <value>5120</value>
> </property>
> </configuration>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)