You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by "Francis.Hu" <fr...@reachjunction.com> on 2013/07/11 04:35:15 UTC
答复: cannot submit a job via java client in hadoop- 2.0.5-alpha
Actually ,I have mapreduce.framework.name configured in mapred-site.xml, see
below:
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
<description>Execution framework set to Hadoop YARN.</description>
</property>
发件人: hadoop hive [mailto:hadoophive@gmail.com]
发送时间: Wednesday, July 10, 2013 18:39
收件人: user@hadoop.apache.org
主题: Re: cannot submit a job via java client in hadoop- 2.0.5-alpha
Here its showing like you are not using mapreduce.framework.name as yarn,
please resend it we are unable to see the configuration
On Wed, Jul 10, 2013 at 1:33 AM, Francis.Hu <fr...@reachjunction.com>
wrote:
Hi,All
I have a hadoop- 2.0.5-alpha cluster with 3 data nodes . I have Resource
Manager and all data nodes started and can access web ui of Resource
Manager.
I wrote a java client to submit a job as TestJob class below. But the job is
never submitted successfully. It throws out exception all the time.
My configurations are attached. Can anyone help me? Thanks.
---------my-java client
public class TestJob {
public void execute() {
Configuration conf1 = new Configuration();
conf1.addResource("resources/core-site.xml");
conf1.addResource("resources/hdfs-site.xml");
conf1.addResource("resources/yarn-site.xml");
conf1.addResource("resources/mapred-site.xml");
JobConf conf = new JobConf(conf1);
conf.setJar("/home/francis/hadoop-jobs/MapReduceJob.jar");
conf.setJobName("Test");
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);
conf.setMapperClass(DisplayRequestMapper.class);
conf.setReducerClass(DisplayRequestReducer.class);
FileInputFormat.setInputPaths(conf,new
Path("/home/francis/hadoop-jobs/2013070907.FNODE.2.txt"));
FileOutputFormat.setOutputPath(conf, new
Path("/home/francis/hadoop-jobs/result/"));
try {
JobClient client = new JobClient(conf);
RunningJob job = client.submitJob(conf);
job.waitForCompletion();
} catch (IOException e) {
e.printStackTrace();
}
}
}
----------Exception
jvm 1 | java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
jvm 1 | at
org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:119)
jvm 1 | at
org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:81)
jvm 1 | at
org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:74)
jvm 1 | at
org.apache.hadoop.mapred.JobClient.init(JobClient.java:482)
jvm 1 | at
org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:461)
jvm 1 | at
com.rh.elastic.hadoop.job.TestJob.execute(TestJob.java:59)
Thanks,
Francis.Hu