You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by rab ra <ra...@gmail.com> on 2014/09/01 12:48:40 UTC
toolrunner issue
Hello
I m having an issue in running one simple map reduce job.
The portion of the code is below. It gives a warning that Hadoop command
line parsing was not peformed.
This occurs despite the class implements Tool interface. Any clue?
public static void main(String[] args) throws Exception {
try{
int exitcode = ToolRunner.run(new Configuration(), new
MyClass(), args);
System.exit(exitcode);
}
catch(Exception e)
{
e.printStackTrace();
}
}
@Override
public int run(String[] args) throws Exception {
JobConf conf = new JobConf(MyClass.class);
System.out.println(args);
FileInputFormat.addInputPath(conf, new Path("/smallInput"));
conf.setInputFormat(CFNInputFormat.class);
conf.setMapperClass(MyMapper.class);
conf.setMapOutputKeyClass(Text.class);
conf.setMapOutputValueClass(Text.class);
FileOutputFormat.setOutputPath(conf, new Path("/TEST"));
JobClient.runJob(conf);
return 0;
Re: toolrunner issue
Posted by rab ra <ra...@gmail.com>.
Yes, its my bad.. U r right..
Thanks
On 1 Sep 2014 17:11, "unmesha sreeveni" <un...@gmail.com> wrote:
> public class MyClass extends Configured implements Tool{
> public static void main(String[] args) throws Exception {
> Configuration conf = new Configuration();
> int res = ToolRunner.run(conf, new MyClass(), args);
> System.exit(res);
> }
>
> @Override
> public int run(String[] args) throws Exception {
> // TODO Auto-generated method stub
> Job job = new Job(conf, "");
> job.setJarByClass(MyClass.class);
> job.setMapOutputKeyClass(IntWritable.class);
> job.setMapOutputValueClass(TwovalueWritable.class);
> job.setOutputKeyClass(IntWritable.class);
> job.setOutputValueClass(TwovalueWritable.class);
> job.setMapperClass(Mapper.class);
> job.setReducerClass(Reducer.class);
> job.setInputFormatClass(TextInputFormat.class);
> job.setOutputFormatClass(TextOutputFormat.class);
> .........................
> }
>
> I am able to work without any errors. Please make sure that you are doing
> the same code above.
>
>
> On Mon, Sep 1, 2014 at 4:18 PM, rab ra <ra...@gmail.com> wrote:
>
>> Hello
>>
>> I m having an issue in running one simple map reduce job.
>> The portion of the code is below. It gives a warning that Hadoop command
>> line parsing was not peformed.
>> This occurs despite the class implements Tool interface. Any clue?
>>
>> public static void main(String[] args) throws Exception {
>>
>> try{
>>
>> int exitcode = ToolRunner.run(new Configuration(), new
>> MyClass(), args);
>>
>> System.exit(exitcode);
>>
>> }
>>
>> catch(Exception e)
>>
>> {
>>
>> e.printStackTrace();
>>
>> }
>>
>> }
>>
>>
>>
>> @Override
>>
>> public int run(String[] args) throws Exception {
>>
>> JobConf conf = new JobConf(MyClass.class);
>>
>> System.out.println(args);
>>
>> FileInputFormat.addInputPath(conf, new Path("/smallInput"));
>>
>> conf.setInputFormat(CFNInputFormat.class);
>>
>> conf.setMapperClass(MyMapper.class);
>>
>> conf.setMapOutputKeyClass(Text.class);
>>
>> conf.setMapOutputValueClass(Text.class);
>>
>> FileOutputFormat.setOutputPath(conf, new Path("/TEST"));
>>
>> JobClient.runJob(conf);
>>
>> return 0;
>>
>
>
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: toolrunner issue
Posted by rab ra <ra...@gmail.com>.
Yes, its my bad.. U r right..
Thanks
On 1 Sep 2014 17:11, "unmesha sreeveni" <un...@gmail.com> wrote:
> public class MyClass extends Configured implements Tool{
> public static void main(String[] args) throws Exception {
> Configuration conf = new Configuration();
> int res = ToolRunner.run(conf, new MyClass(), args);
> System.exit(res);
> }
>
> @Override
> public int run(String[] args) throws Exception {
> // TODO Auto-generated method stub
> Job job = new Job(conf, "");
> job.setJarByClass(MyClass.class);
> job.setMapOutputKeyClass(IntWritable.class);
> job.setMapOutputValueClass(TwovalueWritable.class);
> job.setOutputKeyClass(IntWritable.class);
> job.setOutputValueClass(TwovalueWritable.class);
> job.setMapperClass(Mapper.class);
> job.setReducerClass(Reducer.class);
> job.setInputFormatClass(TextInputFormat.class);
> job.setOutputFormatClass(TextOutputFormat.class);
> .........................
> }
>
> I am able to work without any errors. Please make sure that you are doing
> the same code above.
>
>
> On Mon, Sep 1, 2014 at 4:18 PM, rab ra <ra...@gmail.com> wrote:
>
>> Hello
>>
>> I m having an issue in running one simple map reduce job.
>> The portion of the code is below. It gives a warning that Hadoop command
>> line parsing was not peformed.
>> This occurs despite the class implements Tool interface. Any clue?
>>
>> public static void main(String[] args) throws Exception {
>>
>> try{
>>
>> int exitcode = ToolRunner.run(new Configuration(), new
>> MyClass(), args);
>>
>> System.exit(exitcode);
>>
>> }
>>
>> catch(Exception e)
>>
>> {
>>
>> e.printStackTrace();
>>
>> }
>>
>> }
>>
>>
>>
>> @Override
>>
>> public int run(String[] args) throws Exception {
>>
>> JobConf conf = new JobConf(MyClass.class);
>>
>> System.out.println(args);
>>
>> FileInputFormat.addInputPath(conf, new Path("/smallInput"));
>>
>> conf.setInputFormat(CFNInputFormat.class);
>>
>> conf.setMapperClass(MyMapper.class);
>>
>> conf.setMapOutputKeyClass(Text.class);
>>
>> conf.setMapOutputValueClass(Text.class);
>>
>> FileOutputFormat.setOutputPath(conf, new Path("/TEST"));
>>
>> JobClient.runJob(conf);
>>
>> return 0;
>>
>
>
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: toolrunner issue
Posted by rab ra <ra...@gmail.com>.
Yes, its my bad.. U r right..
Thanks
On 1 Sep 2014 17:11, "unmesha sreeveni" <un...@gmail.com> wrote:
> public class MyClass extends Configured implements Tool{
> public static void main(String[] args) throws Exception {
> Configuration conf = new Configuration();
> int res = ToolRunner.run(conf, new MyClass(), args);
> System.exit(res);
> }
>
> @Override
> public int run(String[] args) throws Exception {
> // TODO Auto-generated method stub
> Job job = new Job(conf, "");
> job.setJarByClass(MyClass.class);
> job.setMapOutputKeyClass(IntWritable.class);
> job.setMapOutputValueClass(TwovalueWritable.class);
> job.setOutputKeyClass(IntWritable.class);
> job.setOutputValueClass(TwovalueWritable.class);
> job.setMapperClass(Mapper.class);
> job.setReducerClass(Reducer.class);
> job.setInputFormatClass(TextInputFormat.class);
> job.setOutputFormatClass(TextOutputFormat.class);
> .........................
> }
>
> I am able to work without any errors. Please make sure that you are doing
> the same code above.
>
>
> On Mon, Sep 1, 2014 at 4:18 PM, rab ra <ra...@gmail.com> wrote:
>
>> Hello
>>
>> I m having an issue in running one simple map reduce job.
>> The portion of the code is below. It gives a warning that Hadoop command
>> line parsing was not peformed.
>> This occurs despite the class implements Tool interface. Any clue?
>>
>> public static void main(String[] args) throws Exception {
>>
>> try{
>>
>> int exitcode = ToolRunner.run(new Configuration(), new
>> MyClass(), args);
>>
>> System.exit(exitcode);
>>
>> }
>>
>> catch(Exception e)
>>
>> {
>>
>> e.printStackTrace();
>>
>> }
>>
>> }
>>
>>
>>
>> @Override
>>
>> public int run(String[] args) throws Exception {
>>
>> JobConf conf = new JobConf(MyClass.class);
>>
>> System.out.println(args);
>>
>> FileInputFormat.addInputPath(conf, new Path("/smallInput"));
>>
>> conf.setInputFormat(CFNInputFormat.class);
>>
>> conf.setMapperClass(MyMapper.class);
>>
>> conf.setMapOutputKeyClass(Text.class);
>>
>> conf.setMapOutputValueClass(Text.class);
>>
>> FileOutputFormat.setOutputPath(conf, new Path("/TEST"));
>>
>> JobClient.runJob(conf);
>>
>> return 0;
>>
>
>
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: toolrunner issue
Posted by rab ra <ra...@gmail.com>.
Yes, its my bad.. U r right..
Thanks
On 1 Sep 2014 17:11, "unmesha sreeveni" <un...@gmail.com> wrote:
> public class MyClass extends Configured implements Tool{
> public static void main(String[] args) throws Exception {
> Configuration conf = new Configuration();
> int res = ToolRunner.run(conf, new MyClass(), args);
> System.exit(res);
> }
>
> @Override
> public int run(String[] args) throws Exception {
> // TODO Auto-generated method stub
> Job job = new Job(conf, "");
> job.setJarByClass(MyClass.class);
> job.setMapOutputKeyClass(IntWritable.class);
> job.setMapOutputValueClass(TwovalueWritable.class);
> job.setOutputKeyClass(IntWritable.class);
> job.setOutputValueClass(TwovalueWritable.class);
> job.setMapperClass(Mapper.class);
> job.setReducerClass(Reducer.class);
> job.setInputFormatClass(TextInputFormat.class);
> job.setOutputFormatClass(TextOutputFormat.class);
> .........................
> }
>
> I am able to work without any errors. Please make sure that you are doing
> the same code above.
>
>
> On Mon, Sep 1, 2014 at 4:18 PM, rab ra <ra...@gmail.com> wrote:
>
>> Hello
>>
>> I m having an issue in running one simple map reduce job.
>> The portion of the code is below. It gives a warning that Hadoop command
>> line parsing was not peformed.
>> This occurs despite the class implements Tool interface. Any clue?
>>
>> public static void main(String[] args) throws Exception {
>>
>> try{
>>
>> int exitcode = ToolRunner.run(new Configuration(), new
>> MyClass(), args);
>>
>> System.exit(exitcode);
>>
>> }
>>
>> catch(Exception e)
>>
>> {
>>
>> e.printStackTrace();
>>
>> }
>>
>> }
>>
>>
>>
>> @Override
>>
>> public int run(String[] args) throws Exception {
>>
>> JobConf conf = new JobConf(MyClass.class);
>>
>> System.out.println(args);
>>
>> FileInputFormat.addInputPath(conf, new Path("/smallInput"));
>>
>> conf.setInputFormat(CFNInputFormat.class);
>>
>> conf.setMapperClass(MyMapper.class);
>>
>> conf.setMapOutputKeyClass(Text.class);
>>
>> conf.setMapOutputValueClass(Text.class);
>>
>> FileOutputFormat.setOutputPath(conf, new Path("/TEST"));
>>
>> JobClient.runJob(conf);
>>
>> return 0;
>>
>
>
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Center for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: toolrunner issue
Posted by unmesha sreeveni <un...@gmail.com>.
public class MyClass extends Configured implements Tool{
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
int res = ToolRunner.run(conf, new MyClass(), args);
System.exit(res);
}
@Override
public int run(String[] args) throws Exception {
// TODO Auto-generated method stub
Job job = new Job(conf, "");
job.setJarByClass(MyClass.class);
job.setMapOutputKeyClass(IntWritable.class);
job.setMapOutputValueClass(TwovalueWritable.class);
job.setOutputKeyClass(IntWritable.class);
job.setOutputValueClass(TwovalueWritable.class);
job.setMapperClass(Mapper.class);
job.setReducerClass(Reducer.class);
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
.........................
}
I am able to work without any errors. Please make sure that you are doing
the same code above.
On Mon, Sep 1, 2014 at 4:18 PM, rab ra <ra...@gmail.com> wrote:
> Hello
>
> I m having an issue in running one simple map reduce job.
> The portion of the code is below. It gives a warning that Hadoop command
> line parsing was not peformed.
> This occurs despite the class implements Tool interface. Any clue?
>
> public static void main(String[] args) throws Exception {
>
> try{
>
> int exitcode = ToolRunner.run(new Configuration(), new
> MyClass(), args);
>
> System.exit(exitcode);
>
> }
>
> catch(Exception e)
>
> {
>
> e.printStackTrace();
>
> }
>
> }
>
>
>
> @Override
>
> public int run(String[] args) throws Exception {
>
> JobConf conf = new JobConf(MyClass.class);
>
> System.out.println(args);
>
> FileInputFormat.addInputPath(conf, new Path("/smallInput"));
>
> conf.setInputFormat(CFNInputFormat.class);
>
> conf.setMapperClass(MyMapper.class);
>
> conf.setMapOutputKeyClass(Text.class);
>
> conf.setMapOutputValueClass(Text.class);
>
> FileOutputFormat.setOutputPath(conf, new Path("/TEST"));
>
> JobClient.runJob(conf);
>
> return 0;
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Center for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: toolrunner issue
Posted by unmesha sreeveni <un...@gmail.com>.
public class MyClass extends Configured implements Tool{
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
int res = ToolRunner.run(conf, new MyClass(), args);
System.exit(res);
}
@Override
public int run(String[] args) throws Exception {
// TODO Auto-generated method stub
Job job = new Job(conf, "");
job.setJarByClass(MyClass.class);
job.setMapOutputKeyClass(IntWritable.class);
job.setMapOutputValueClass(TwovalueWritable.class);
job.setOutputKeyClass(IntWritable.class);
job.setOutputValueClass(TwovalueWritable.class);
job.setMapperClass(Mapper.class);
job.setReducerClass(Reducer.class);
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
.........................
}
I am able to work without any errors. Please make sure that you are doing
the same code above.
On Mon, Sep 1, 2014 at 4:18 PM, rab ra <ra...@gmail.com> wrote:
> Hello
>
> I m having an issue in running one simple map reduce job.
> The portion of the code is below. It gives a warning that Hadoop command
> line parsing was not peformed.
> This occurs despite the class implements Tool interface. Any clue?
>
> public static void main(String[] args) throws Exception {
>
> try{
>
> int exitcode = ToolRunner.run(new Configuration(), new
> MyClass(), args);
>
> System.exit(exitcode);
>
> }
>
> catch(Exception e)
>
> {
>
> e.printStackTrace();
>
> }
>
> }
>
>
>
> @Override
>
> public int run(String[] args) throws Exception {
>
> JobConf conf = new JobConf(MyClass.class);
>
> System.out.println(args);
>
> FileInputFormat.addInputPath(conf, new Path("/smallInput"));
>
> conf.setInputFormat(CFNInputFormat.class);
>
> conf.setMapperClass(MyMapper.class);
>
> conf.setMapOutputKeyClass(Text.class);
>
> conf.setMapOutputValueClass(Text.class);
>
> FileOutputFormat.setOutputPath(conf, new Path("/TEST"));
>
> JobClient.runJob(conf);
>
> return 0;
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Center for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: toolrunner issue
Posted by unmesha sreeveni <un...@gmail.com>.
public class MyClass extends Configured implements Tool{
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
int res = ToolRunner.run(conf, new MyClass(), args);
System.exit(res);
}
@Override
public int run(String[] args) throws Exception {
// TODO Auto-generated method stub
Job job = new Job(conf, "");
job.setJarByClass(MyClass.class);
job.setMapOutputKeyClass(IntWritable.class);
job.setMapOutputValueClass(TwovalueWritable.class);
job.setOutputKeyClass(IntWritable.class);
job.setOutputValueClass(TwovalueWritable.class);
job.setMapperClass(Mapper.class);
job.setReducerClass(Reducer.class);
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
.........................
}
I am able to work without any errors. Please make sure that you are doing
the same code above.
On Mon, Sep 1, 2014 at 4:18 PM, rab ra <ra...@gmail.com> wrote:
> Hello
>
> I m having an issue in running one simple map reduce job.
> The portion of the code is below. It gives a warning that Hadoop command
> line parsing was not peformed.
> This occurs despite the class implements Tool interface. Any clue?
>
> public static void main(String[] args) throws Exception {
>
> try{
>
> int exitcode = ToolRunner.run(new Configuration(), new
> MyClass(), args);
>
> System.exit(exitcode);
>
> }
>
> catch(Exception e)
>
> {
>
> e.printStackTrace();
>
> }
>
> }
>
>
>
> @Override
>
> public int run(String[] args) throws Exception {
>
> JobConf conf = new JobConf(MyClass.class);
>
> System.out.println(args);
>
> FileInputFormat.addInputPath(conf, new Path("/smallInput"));
>
> conf.setInputFormat(CFNInputFormat.class);
>
> conf.setMapperClass(MyMapper.class);
>
> conf.setMapOutputKeyClass(Text.class);
>
> conf.setMapOutputValueClass(Text.class);
>
> FileOutputFormat.setOutputPath(conf, new Path("/TEST"));
>
> JobClient.runJob(conf);
>
> return 0;
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Center for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: toolrunner issue
Posted by unmesha sreeveni <un...@gmail.com>.
public class MyClass extends Configured implements Tool{
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
int res = ToolRunner.run(conf, new MyClass(), args);
System.exit(res);
}
@Override
public int run(String[] args) throws Exception {
// TODO Auto-generated method stub
Job job = new Job(conf, "");
job.setJarByClass(MyClass.class);
job.setMapOutputKeyClass(IntWritable.class);
job.setMapOutputValueClass(TwovalueWritable.class);
job.setOutputKeyClass(IntWritable.class);
job.setOutputValueClass(TwovalueWritable.class);
job.setMapperClass(Mapper.class);
job.setReducerClass(Reducer.class);
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
.........................
}
I am able to work without any errors. Please make sure that you are doing
the same code above.
On Mon, Sep 1, 2014 at 4:18 PM, rab ra <ra...@gmail.com> wrote:
> Hello
>
> I m having an issue in running one simple map reduce job.
> The portion of the code is below. It gives a warning that Hadoop command
> line parsing was not peformed.
> This occurs despite the class implements Tool interface. Any clue?
>
> public static void main(String[] args) throws Exception {
>
> try{
>
> int exitcode = ToolRunner.run(new Configuration(), new
> MyClass(), args);
>
> System.exit(exitcode);
>
> }
>
> catch(Exception e)
>
> {
>
> e.printStackTrace();
>
> }
>
> }
>
>
>
> @Override
>
> public int run(String[] args) throws Exception {
>
> JobConf conf = new JobConf(MyClass.class);
>
> System.out.println(args);
>
> FileInputFormat.addInputPath(conf, new Path("/smallInput"));
>
> conf.setInputFormat(CFNInputFormat.class);
>
> conf.setMapperClass(MyMapper.class);
>
> conf.setMapOutputKeyClass(Text.class);
>
> conf.setMapOutputValueClass(Text.class);
>
> FileOutputFormat.setOutputPath(conf, new Path("/TEST"));
>
> JobClient.runJob(conf);
>
> return 0;
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Center for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/