You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Tanniru Govardhan <go...@gmail.com> on 2013/07/26 15:37:17 UTC

Context Object in Map Reduce

Hi everyone,
I am trying to pass a string variable from Driver class to Mapper class.
I came to know that i need to use context variable for this purpose.
But the Mapper method i have used has a signature without context object.
It is not allowing to change the method signature.(reporting syntax error)
Can anyone please suggest me what can I do regarding this?
I am very new to Map Reduce Programming.

Here is the code:

class Map<V> extends MapReduceBase implements Mapper<LongWritable, Text,
Text, IntWritable>
{
    String myfilename;
  private Text word = new Text();
  private IntWritable var = new IntWritable();

  public void map(LongWritable key, Text value, *Context
context,*OutputCollector<Text, IntWritable> output, Reporter reporter)
*//here is the error *
 throws IOException
{
         Configuration conf = context.getConfiguration();
             myfilename = conf.get("filename");
             -- - - -- - -- -
        }
 public  void compute() throws Exception
  {
  Configuration config = new Configuration();
      config.set("filename", "Syn-100n-8t-2l-2k.vars");

    JobConf conf = new JobConf(Hadoop_GNTS.class);
    -- - - - -- - -
}

Thanks

RE: Context Object in Map Reduce

Posted by Devaraj k <de...@huawei.com>.

If you are using new Mapper API (org.apache.hadoop.mapreduce.Mapper) for your Job, then you can get the configuration object from Context like "context.getConfiguration() ". If you are using old Mapper API (org.apache.hadoop.mapred.Mapper) you can get the JobConf object as an argument to configure() method which you implement in Mapper as you mentioned below.

Thanks
Devaraj k

From: Tanniru Govardhan [mailto:govardhan5126@gmail.com]
Sent: 26 July 2013 21:06
To: user@hadoop.apache.org; dunanimanish@gmail.com
Subject: Re: Context Object in Map Reduce

Thanks Manish. you are correct. I figured it out somehow.
This is what i had to change.

In the main class
Configuration config = new Configuration();
config.set("filename", "Syn-100n-8t-2l-2k.vars");
JobConf conf = new JobConf(config,Hadoop_GNTS.class);

In the Map class
public void configure(JobConf job)
  {
   myfilename = job.get("filename");
  }

Thanks

On Fri, Jul 26, 2013 at 10:56 AM, manish dunani <ma...@gmail.com>> wrote:
If u r using OutputCollector,Reporter then no need to use Context..Context works same as OutputCollector and Reporter.

That is used to collect mapper's <key ,value> pair.

On Fri, Jul 26, 2013 at 8:03 PM, manish dunani <ma...@gmail.com>> wrote:
Can u please elaborate what is the code and what is error.??

Then it will be much better to give the answer.

Regards Manish Dunani..

On Fri, Jul 26, 2013 at 7:07 PM, Tanniru Govardhan <go...@gmail.com>> wrote:
Hi everyone,
I am trying to pass a string variable from Driver class to Mapper class.
I came to know that i need to use context variable for this purpose.
But the Mapper method i have used has a signature without context object.
It is not allowing to change the method signature.(reporting syntax error)
Can anyone please suggest me what can I do regarding this?
I am very new to Map Reduce Programming.

Here is the code:

class Map<V> extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable>
{
   String myfilename;
  private Text word = new Text();
  private IntWritable var = new IntWritable();

  public void map(LongWritable key, Text value, Context context, OutputCollector<Text, IntWritable> output, Reporter reporter) //here is the error
throws IOException
{
         Configuration conf = context.getConfiguration();
             myfilename = conf.get("filename");
             -- - - -- - -- -
        }
 public  void compute() throws Exception
  {
 Configuration config = new Configuration();
      config.set("filename", "Syn-100n-8t-2l-2k.vars");

    JobConf conf = new JobConf(Hadoop_GNTS.class);
    -- - - - -- - -
}

Thanks

--
MANISH DUNANI
-THANX
+91 9426881954<tel:%2B91%209426881954>,+91 8460656443<tel:%2B91%208460656443>
manishd207@gmail.com<ma...@gmail.com>

--
MANISH DUNANI
-THANX
+91 9426881954<tel:%2B91%209426881954>,+91 8460656443<tel:%2B91%208460656443>
manishd207@gmail.com<ma...@gmail.com>

RE: Context Object in Map Reduce

Posted by Devaraj k <de...@huawei.com>.

If you are using new Mapper API (org.apache.hadoop.mapreduce.Mapper) for your Job, then you can get the configuration object from Context like "context.getConfiguration() ". If you are using old Mapper API (org.apache.hadoop.mapred.Mapper) you can get the JobConf object as an argument to configure() method which you implement in Mapper as you mentioned below.

Thanks
Devaraj k

From: Tanniru Govardhan [mailto:govardhan5126@gmail.com]
Sent: 26 July 2013 21:06
To: user@hadoop.apache.org; dunanimanish@gmail.com
Subject: Re: Context Object in Map Reduce

Thanks Manish. you are correct. I figured it out somehow.
This is what i had to change.

In the main class
Configuration config = new Configuration();
config.set("filename", "Syn-100n-8t-2l-2k.vars");
JobConf conf = new JobConf(config,Hadoop_GNTS.class);

In the Map class
public void configure(JobConf job)
  {
   myfilename = job.get("filename");
  }

Thanks

On Fri, Jul 26, 2013 at 10:56 AM, manish dunani <ma...@gmail.com>> wrote:
If u r using OutputCollector,Reporter then no need to use Context..Context works same as OutputCollector and Reporter.

That is used to collect mapper's <key ,value> pair.

On Fri, Jul 26, 2013 at 8:03 PM, manish dunani <ma...@gmail.com>> wrote:
Can u please elaborate what is the code and what is error.??

Then it will be much better to give the answer.

Regards Manish Dunani..

On Fri, Jul 26, 2013 at 7:07 PM, Tanniru Govardhan <go...@gmail.com>> wrote:
Hi everyone,
I am trying to pass a string variable from Driver class to Mapper class.
I came to know that i need to use context variable for this purpose.
But the Mapper method i have used has a signature without context object.
It is not allowing to change the method signature.(reporting syntax error)
Can anyone please suggest me what can I do regarding this?
I am very new to Map Reduce Programming.

Here is the code:

class Map<V> extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable>
{
   String myfilename;
  private Text word = new Text();
  private IntWritable var = new IntWritable();

  public void map(LongWritable key, Text value, Context context, OutputCollector<Text, IntWritable> output, Reporter reporter) //here is the error
throws IOException
{
         Configuration conf = context.getConfiguration();
             myfilename = conf.get("filename");
             -- - - -- - -- -
        }
 public  void compute() throws Exception
  {
 Configuration config = new Configuration();
      config.set("filename", "Syn-100n-8t-2l-2k.vars");

    JobConf conf = new JobConf(Hadoop_GNTS.class);
    -- - - - -- - -
}

Thanks

--
MANISH DUNANI
-THANX
+91 9426881954<tel:%2B91%209426881954>,+91 8460656443<tel:%2B91%208460656443>
manishd207@gmail.com<ma...@gmail.com>

--
MANISH DUNANI
-THANX
+91 9426881954<tel:%2B91%209426881954>,+91 8460656443<tel:%2B91%208460656443>
manishd207@gmail.com<ma...@gmail.com>

RE: Context Object in Map Reduce

Posted by Devaraj k <de...@huawei.com>.

If you are using new Mapper API (org.apache.hadoop.mapreduce.Mapper) for your Job, then you can get the configuration object from Context like "context.getConfiguration() ". If you are using old Mapper API (org.apache.hadoop.mapred.Mapper) you can get the JobConf object as an argument to configure() method which you implement in Mapper as you mentioned below.

Thanks
Devaraj k

From: Tanniru Govardhan [mailto:govardhan5126@gmail.com]
Sent: 26 July 2013 21:06
To: user@hadoop.apache.org; dunanimanish@gmail.com
Subject: Re: Context Object in Map Reduce

Thanks Manish. you are correct. I figured it out somehow.
This is what i had to change.

In the main class
Configuration config = new Configuration();
config.set("filename", "Syn-100n-8t-2l-2k.vars");
JobConf conf = new JobConf(config,Hadoop_GNTS.class);

In the Map class
public void configure(JobConf job)
  {
   myfilename = job.get("filename");
  }

Thanks

On Fri, Jul 26, 2013 at 10:56 AM, manish dunani <ma...@gmail.com>> wrote:
If u r using OutputCollector,Reporter then no need to use Context..Context works same as OutputCollector and Reporter.

That is used to collect mapper's <key ,value> pair.

On Fri, Jul 26, 2013 at 8:03 PM, manish dunani <ma...@gmail.com>> wrote:
Can u please elaborate what is the code and what is error.??

Then it will be much better to give the answer.

Regards Manish Dunani..

On Fri, Jul 26, 2013 at 7:07 PM, Tanniru Govardhan <go...@gmail.com>> wrote:
Hi everyone,
I am trying to pass a string variable from Driver class to Mapper class.
I came to know that i need to use context variable for this purpose.
But the Mapper method i have used has a signature without context object.
It is not allowing to change the method signature.(reporting syntax error)
Can anyone please suggest me what can I do regarding this?
I am very new to Map Reduce Programming.

Here is the code:

class Map<V> extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable>
{
   String myfilename;
  private Text word = new Text();
  private IntWritable var = new IntWritable();

  public void map(LongWritable key, Text value, Context context, OutputCollector<Text, IntWritable> output, Reporter reporter) //here is the error
throws IOException
{
         Configuration conf = context.getConfiguration();
             myfilename = conf.get("filename");
             -- - - -- - -- -
        }
 public  void compute() throws Exception
  {
 Configuration config = new Configuration();
      config.set("filename", "Syn-100n-8t-2l-2k.vars");

    JobConf conf = new JobConf(Hadoop_GNTS.class);
    -- - - - -- - -
}

Thanks

--
MANISH DUNANI
-THANX
+91 9426881954<tel:%2B91%209426881954>,+91 8460656443<tel:%2B91%208460656443>
manishd207@gmail.com<ma...@gmail.com>

--
MANISH DUNANI
-THANX
+91 9426881954<tel:%2B91%209426881954>,+91 8460656443<tel:%2B91%208460656443>
manishd207@gmail.com<ma...@gmail.com>

RE: Context Object in Map Reduce

Posted by Devaraj k <de...@huawei.com>.

If you are using new Mapper API (org.apache.hadoop.mapreduce.Mapper) for your Job, then you can get the configuration object from Context like "context.getConfiguration() ". If you are using old Mapper API (org.apache.hadoop.mapred.Mapper) you can get the JobConf object as an argument to configure() method which you implement in Mapper as you mentioned below.

Thanks
Devaraj k

From: Tanniru Govardhan [mailto:govardhan5126@gmail.com]
Sent: 26 July 2013 21:06
To: user@hadoop.apache.org; dunanimanish@gmail.com
Subject: Re: Context Object in Map Reduce

Thanks Manish. you are correct. I figured it out somehow.
This is what i had to change.

In the main class
Configuration config = new Configuration();
config.set("filename", "Syn-100n-8t-2l-2k.vars");
JobConf conf = new JobConf(config,Hadoop_GNTS.class);

In the Map class
public void configure(JobConf job)
  {
   myfilename = job.get("filename");
  }

Thanks

On Fri, Jul 26, 2013 at 10:56 AM, manish dunani <ma...@gmail.com>> wrote:
If u r using OutputCollector,Reporter then no need to use Context..Context works same as OutputCollector and Reporter.

That is used to collect mapper's <key ,value> pair.

On Fri, Jul 26, 2013 at 8:03 PM, manish dunani <ma...@gmail.com>> wrote:
Can u please elaborate what is the code and what is error.??

Then it will be much better to give the answer.

Regards Manish Dunani..

On Fri, Jul 26, 2013 at 7:07 PM, Tanniru Govardhan <go...@gmail.com>> wrote:
Hi everyone,
I am trying to pass a string variable from Driver class to Mapper class.
I came to know that i need to use context variable for this purpose.
But the Mapper method i have used has a signature without context object.
It is not allowing to change the method signature.(reporting syntax error)
Can anyone please suggest me what can I do regarding this?
I am very new to Map Reduce Programming.

Here is the code:

class Map<V> extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable>
{
   String myfilename;
  private Text word = new Text();
  private IntWritable var = new IntWritable();

  public void map(LongWritable key, Text value, Context context, OutputCollector<Text, IntWritable> output, Reporter reporter) //here is the error
throws IOException
{
         Configuration conf = context.getConfiguration();
             myfilename = conf.get("filename");
             -- - - -- - -- -
        }
 public  void compute() throws Exception
  {
 Configuration config = new Configuration();
      config.set("filename", "Syn-100n-8t-2l-2k.vars");

    JobConf conf = new JobConf(Hadoop_GNTS.class);
    -- - - - -- - -
}

Thanks

--
MANISH DUNANI
-THANX
+91 9426881954<tel:%2B91%209426881954>,+91 8460656443<tel:%2B91%208460656443>
manishd207@gmail.com<ma...@gmail.com>

--
MANISH DUNANI
-THANX
+91 9426881954<tel:%2B91%209426881954>,+91 8460656443<tel:%2B91%208460656443>
manishd207@gmail.com<ma...@gmail.com>

Re: Context Object in Map Reduce

Posted by Tanniru Govardhan <go...@gmail.com>.

Thanks Manish. you are correct. I figured it out somehow.
This is what i had to change.

In the main class
*
Configuration config = new Configuration();
config.set("filename", "Syn-100n-8t-2l-2k.vars");
*
*JobConf conf = new JobConf(config,Hadoop_GNTS.class);*
*
*
In the Map class
*
public void configure(JobConf job)
  {
   myfilename = job.get("filename");
  }
*
*
*
Thanks
*
*


On Fri, Jul 26, 2013 at 10:56 AM, manish dunani <ma...@gmail.com>wrote:

> If u r using OutputCollector,Reporter then no need to use Context..Context
> works same as OutputCollector and Reporter.
>
> That is used to collect mapper's <key ,value> pair.
>
>
>
> On Fri, Jul 26, 2013 at 8:03 PM, manish dunani <ma...@gmail.com>wrote:
>
>> Can u please elaborate what is the code and what is error.??
>>
>> Then it will be much better to give the answer.
>>
>> Regards Manish Dunani..
>>
>>
>> On Fri, Jul 26, 2013 at 7:07 PM, Tanniru Govardhan <
>> govardhan5126@gmail.com> wrote:
>>
>>> Hi everyone,
>>> I am trying to pass a string variable from Driver class to Mapper class.
>>> I came to know that i need to use context variable for this purpose.
>>> But the Mapper method i have used has a signature without context object.
>>> It is not allowing to change the method signature.(reporting syntax
>>> error)
>>> Can anyone please suggest me what can I do regarding this?
>>> I am very new to Map Reduce Programming.
>>>
>>> Here is the code:
>>>
>>> class Map<V> extends MapReduceBase implements Mapper<LongWritable, Text,
>>> Text, IntWritable>
>>> {
>>>     String myfilename;
>>>   private Text word = new Text();
>>>   private IntWritable var = new IntWritable();
>>>
>>>   public void map(LongWritable key, Text value, *Context context,*OutputCollector<Text, IntWritable> output, Reporter reporter)
>>> *//here is the error *
>>>  throws IOException
>>> {
>>>          Configuration conf = context.getConfiguration();
>>>              myfilename = conf.get("filename");
>>>              -- - - -- - -- -
>>>         }
>>>  public  void compute() throws Exception
>>>   {
>>>   Configuration config = new Configuration();
>>>       config.set("filename", "Syn-100n-8t-2l-2k.vars");
>>>
>>>     JobConf conf = new JobConf(Hadoop_GNTS.class);
>>>     -- - - - -- - -
>>> }
>>>
>>> Thanks
>>>
>>>
>>
>>
>> --
>> MANISH DUNANI
>> -THANX
>> +91 9426881954,+91 8460656443
>> manishd207@gmail.com
>>
>
>
>
> --
> MANISH DUNANI
> -THANX
> +91 9426881954,+91 8460656443
> manishd207@gmail.com
>

Re: Context Object in Map Reduce

Posted by Tanniru Govardhan <go...@gmail.com>.

Thanks Manish. you are correct. I figured it out somehow.
This is what i had to change.

In the main class
*
Configuration config = new Configuration();
config.set("filename", "Syn-100n-8t-2l-2k.vars");
*
*JobConf conf = new JobConf(config,Hadoop_GNTS.class);*
*
*
In the Map class
*
public void configure(JobConf job)
  {
   myfilename = job.get("filename");
  }
*
*
*
Thanks
*
*


On Fri, Jul 26, 2013 at 10:56 AM, manish dunani <ma...@gmail.com>wrote:

> If u r using OutputCollector,Reporter then no need to use Context..Context
> works same as OutputCollector and Reporter.
>
> That is used to collect mapper's <key ,value> pair.
>
>
>
> On Fri, Jul 26, 2013 at 8:03 PM, manish dunani <ma...@gmail.com>wrote:
>
>> Can u please elaborate what is the code and what is error.??
>>
>> Then it will be much better to give the answer.
>>
>> Regards Manish Dunani..
>>
>>
>> On Fri, Jul 26, 2013 at 7:07 PM, Tanniru Govardhan <
>> govardhan5126@gmail.com> wrote:
>>
>>> Hi everyone,
>>> I am trying to pass a string variable from Driver class to Mapper class.
>>> I came to know that i need to use context variable for this purpose.
>>> But the Mapper method i have used has a signature without context object.
>>> It is not allowing to change the method signature.(reporting syntax
>>> error)
>>> Can anyone please suggest me what can I do regarding this?
>>> I am very new to Map Reduce Programming.
>>>
>>> Here is the code:
>>>
>>> class Map<V> extends MapReduceBase implements Mapper<LongWritable, Text,
>>> Text, IntWritable>
>>> {
>>>     String myfilename;
>>>   private Text word = new Text();
>>>   private IntWritable var = new IntWritable();
>>>
>>>   public void map(LongWritable key, Text value, *Context context,*OutputCollector<Text, IntWritable> output, Reporter reporter)
>>> *//here is the error *
>>>  throws IOException
>>> {
>>>          Configuration conf = context.getConfiguration();
>>>              myfilename = conf.get("filename");
>>>              -- - - -- - -- -
>>>         }
>>>  public  void compute() throws Exception
>>>   {
>>>   Configuration config = new Configuration();
>>>       config.set("filename", "Syn-100n-8t-2l-2k.vars");
>>>
>>>     JobConf conf = new JobConf(Hadoop_GNTS.class);
>>>     -- - - - -- - -
>>> }
>>>
>>> Thanks
>>>
>>>
>>
>>
>> --
>> MANISH DUNANI
>> -THANX
>> +91 9426881954,+91 8460656443
>> manishd207@gmail.com
>>
>
>
>
> --
> MANISH DUNANI
> -THANX
> +91 9426881954,+91 8460656443
> manishd207@gmail.com
>

Re: Context Object in Map Reduce

Posted by Tanniru Govardhan <go...@gmail.com>.

Thanks Manish. you are correct. I figured it out somehow.
This is what i had to change.

In the main class
*
Configuration config = new Configuration();
config.set("filename", "Syn-100n-8t-2l-2k.vars");
*
*JobConf conf = new JobConf(config,Hadoop_GNTS.class);*
*
*
In the Map class
*
public void configure(JobConf job)
  {
   myfilename = job.get("filename");
  }
*
*
*
Thanks
*
*


On Fri, Jul 26, 2013 at 10:56 AM, manish dunani <ma...@gmail.com>wrote:

> If u r using OutputCollector,Reporter then no need to use Context..Context
> works same as OutputCollector and Reporter.
>
> That is used to collect mapper's <key ,value> pair.
>
>
>
> On Fri, Jul 26, 2013 at 8:03 PM, manish dunani <ma...@gmail.com>wrote:
>
>> Can u please elaborate what is the code and what is error.??
>>
>> Then it will be much better to give the answer.
>>
>> Regards Manish Dunani..
>>
>>
>> On Fri, Jul 26, 2013 at 7:07 PM, Tanniru Govardhan <
>> govardhan5126@gmail.com> wrote:
>>
>>> Hi everyone,
>>> I am trying to pass a string variable from Driver class to Mapper class.
>>> I came to know that i need to use context variable for this purpose.
>>> But the Mapper method i have used has a signature without context object.
>>> It is not allowing to change the method signature.(reporting syntax
>>> error)
>>> Can anyone please suggest me what can I do regarding this?
>>> I am very new to Map Reduce Programming.
>>>
>>> Here is the code:
>>>
>>> class Map<V> extends MapReduceBase implements Mapper<LongWritable, Text,
>>> Text, IntWritable>
>>> {
>>>     String myfilename;
>>>   private Text word = new Text();
>>>   private IntWritable var = new IntWritable();
>>>
>>>   public void map(LongWritable key, Text value, *Context context,*OutputCollector<Text, IntWritable> output, Reporter reporter)
>>> *//here is the error *
>>>  throws IOException
>>> {
>>>          Configuration conf = context.getConfiguration();
>>>              myfilename = conf.get("filename");
>>>              -- - - -- - -- -
>>>         }
>>>  public  void compute() throws Exception
>>>   {
>>>   Configuration config = new Configuration();
>>>       config.set("filename", "Syn-100n-8t-2l-2k.vars");
>>>
>>>     JobConf conf = new JobConf(Hadoop_GNTS.class);
>>>     -- - - - -- - -
>>> }
>>>
>>> Thanks
>>>
>>>
>>
>>
>> --
>> MANISH DUNANI
>> -THANX
>> +91 9426881954,+91 8460656443
>> manishd207@gmail.com
>>
>
>
>
> --
> MANISH DUNANI
> -THANX
> +91 9426881954,+91 8460656443
> manishd207@gmail.com
>