You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Kadu canGica Eduardo <ka...@gmail.com> on 2011/09/01 02:26:47 UTC

FileOutputFormat (getWorkOutputPath)

Hi,

what is the right way to use the method getWorkOutputPath from the
FileOutputFormat?

Here is a sample of my code:

public void map(LongWritable key, Text value, OutputCollector<Text, Text>
output, Reporter reporter) throws IOException {
...
     JobConf conf = new JobConf(new Configuration());
     dir = FileOutputFormat.getWorkOutputPath(conf).toString();
...

i'm getting the "java.lang.NullPointerException" error.

what is the solution for this?

Thanks in advance.

Carlos.

Re: FileOutputFormat (getWorkOutputPath)

Posted by Sudharsan Sampath <su...@gmail.com>.

Hi,

Also move the line 'dir = FileOutputFormat.getWorkOutputPath(conf).toString();'
to the configure method as the map() is called for every input line.

Thanks
Sudhan S

On Thu, Sep 1, 2011 at 9:11 AM, Kadu canGica Eduardo
<ka...@gmail.com>wrote:

> Thanks a lot Harsh J.
> Now it is working fine!
>
>
> 2011/8/31 Harsh J <ha...@cloudera.com>
>
>> Override the "configure" method in your Mapper class to get a handle
>> to the proper configuration object. Do not instantiate a new
>> Configuration object, as you're doing below:
>>
>> Wrong:
>>
>> JobConf conf = new JobConf(new Configuration());
>>
>> Right:
>>
>> @Override
>> void configure(Configuration conf) {
>>  this.conf = conf;
>> }
>> …
>> …
>> JobConf conf = new JobConf(conf);
>>
>> On Thu, Sep 1, 2011 at 5:56 AM, Kadu canGica Eduardo
>> <ka...@gmail.com> wrote:
>> > Hi,
>> >
>> > what is the right way to use the method getWorkOutputPath from the
>> > FileOutputFormat?
>> >
>> > Here is a sample of my code:
>> >
>> > public void map(LongWritable key, Text value, OutputCollector<Text,
>> Text>
>> > output, Reporter reporter) throws IOException {
>> > ...
>> >      JobConf conf = new JobConf(new Configuration());
>> >      dir = FileOutputFormat.getWorkOutputPath(conf).toString();
>> > ...
>> >
>> > i'm getting the "java.lang.NullPointerException" error.
>> >
>> > what is the solution for this?
>> >
>> > Thanks in advance.
>> >
>> > Carlos.
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>

Re: FileOutputFormat (getWorkOutputPath)

Posted by Kadu canGica Eduardo <ka...@gmail.com>.

Thanks a lot Harsh J.
Now it is working fine!

2011/8/31 Harsh J <ha...@cloudera.com>

> Override the "configure" method in your Mapper class to get a handle
> to the proper configuration object. Do not instantiate a new
> Configuration object, as you're doing below:
>
> Wrong:
>
> JobConf conf = new JobConf(new Configuration());
>
> Right:
>
> @Override
> void configure(Configuration conf) {
>  this.conf = conf;
> }
> …
> …
> JobConf conf = new JobConf(conf);
>
> On Thu, Sep 1, 2011 at 5:56 AM, Kadu canGica Eduardo
> <ka...@gmail.com> wrote:
> > Hi,
> >
> > what is the right way to use the method getWorkOutputPath from the
> > FileOutputFormat?
> >
> > Here is a sample of my code:
> >
> > public void map(LongWritable key, Text value, OutputCollector<Text, Text>
> > output, Reporter reporter) throws IOException {
> > ...
> >      JobConf conf = new JobConf(new Configuration());
> >      dir = FileOutputFormat.getWorkOutputPath(conf).toString();
> > ...
> >
> > i'm getting the "java.lang.NullPointerException" error.
> >
> > what is the solution for this?
> >
> > Thanks in advance.
> >
> > Carlos.
> >
>
>
>
> --
> Harsh J
>

Re: FileOutputFormat (getWorkOutputPath)

Posted by Harsh J <ha...@cloudera.com>.

Override the "configure" method in your Mapper class to get a handle
to the proper configuration object. Do not instantiate a new
Configuration object, as you're doing below:

Wrong:

JobConf conf = new JobConf(new Configuration());

Right:

@Override
void configure(Configuration conf) {
 this.conf = conf;
}
…
…
JobConf conf = new JobConf(conf);

On Thu, Sep 1, 2011 at 5:56 AM, Kadu canGica Eduardo
<ka...@gmail.com> wrote:
> Hi,
>
> what is the right way to use the method getWorkOutputPath from the
> FileOutputFormat?
>
> Here is a sample of my code:
>
> public void map(LongWritable key, Text value, OutputCollector<Text, Text>
> output, Reporter reporter) throws IOException {
> ...
>      JobConf conf = new JobConf(new Configuration());
>      dir = FileOutputFormat.getWorkOutputPath(conf).toString();
> ...
>
> i'm getting the "java.lang.NullPointerException" error.
>
> what is the solution for this?
>
> Thanks in advance.
>
> Carlos.
>



-- 
Harsh J