You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by abhinit <ab...@gmail.com> on 2008/12/05 06:19:15 UTC

getting Configuration object in mapper

I have set some variable using the JobConf object.

jobConf.set("Operator", operator) etc.

How can I get an instance of Configuration object/ JobConf object inside
a map method so that I can retrieve these variables.

Thanks
-Abhinit

Re: getting Configuration object in mapper

Posted by Sagar Naik <sn...@attributor.com>.
check : mapred.task.is.map

Craig Macdonald wrote:
> I have a related question - I have a class which is both mapper and 
> reducer. How can I tell in configure() if the current task is map or a 
> reduce task? Parse the taskid?
>
> C
>
> Owen O'Malley wrote:
>>
>> On Dec 4, 2008, at 9:19 PM, abhinit wrote:
>>
>>> I have set some variable using the JobConf object.
>>>
>>> jobConf.set("Operator", operator) etc.
>>>
>>> How can I get an instance of Configuration object/ JobConf object 
>>> inside
>>> a map method so that I can retrieve these variables.
>>
>> In your Mapper class, implement a method like:
>>  public void configure(JobConf job) { ... }
>>
>> This will be called when the object is created with the job conf.
>>
>> -- Owen
>

Re: getting Configuration object in mapper

Posted by Arun C Murthy <ac...@yahoo-inc.com>.
On Dec 5, 2008, at 12:32 PM, Craig Macdonald wrote:

> I have a related question - I have a class which is both mapper and  
> reducer. How can I tell in configure() if the current task is map or  
> a reduce task? Parse the taskid?
>

Get the taskid, then use
http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/TaskAttemptID.html#forName(java.lang.String)
followed by
http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/TaskAttemptID.html#isMap()

Arun

Re: getting Configuration object in mapper

Posted by Craig Macdonald <cr...@dcs.gla.ac.uk>.
I have a related question - I have a class which is both mapper and 
reducer. How can I tell in configure() if the current task is map or a 
reduce task? Parse the taskid?

C

Owen O'Malley wrote:
>
> On Dec 4, 2008, at 9:19 PM, abhinit wrote:
>
>> I have set some variable using the JobConf object.
>>
>> jobConf.set("Operator", operator) etc.
>>
>> How can I get an instance of Configuration object/ JobConf object inside
>> a map method so that I can retrieve these variables.
>
> In your Mapper class, implement a method like:
>  public void configure(JobConf job) { ... }
>
> This will be called when the object is created with the job conf.
>
> -- Owen

Re: getting Configuration object in mapper

Posted by Owen O'Malley <om...@apache.org>.
On Dec 4, 2008, at 9:19 PM, abhinit wrote:

> I have set some variable using the JobConf object.
>
> jobConf.set("Operator", operator) etc.
>
> How can I get an instance of Configuration object/ JobConf object  
> inside
> a map method so that I can retrieve these variables.

In your Mapper class, implement a method like:
  public void configure(JobConf job) { ... }

This will be called when the object is created with the job conf.

-- Owen