You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Abhinav M Kulkarni <ab...@gmail.com> on 2013/04/02 18:46:56 UTC
Multiple mappers, different parameter values
Hi,
I have a situation wherein I have multiple mappers. I am using
MultipleInputs class to add them.
Now, I need to pass different parameters to different mappers. For e.g.
lets say I have a parameter 'num.iterations' that is set differently for
different mappers. One way to pass parameters to mappers is to set them
in a Configuration object:
Configuration conf = new Configuration();
conf.set("num.iterations", "100");
However using this I can't set the value differently for different
mappers. How do I achieve this?
Thanks,
Abhinav
Re: Multiple mappers, different parameter values
Posted by Abhinav M Kulkarni <ab...@gmail.com>.
Hi Mirko,
Thanks for the reply.
Yeah that's one solution.
--
Abhinav
On 04/02/2013 10:04 AM, Mirko Kämpf wrote:
> Hi,
>
> I would add an id to the parameter name: "num.iterations.ID
> <http://num.iterations.ID>=something"
> If your mapper knows what ID it has it can just pick up this value
> from the context. But the question is:
> How does the mapper know about it's ID? Is it related to the input?
> Thank it can be calculated but this is
> a domain specific issue.
>
> Best wishes
> Mirko
>
>
> 2013/4/2 Abhinav M Kulkarni <abhinavkulkarni@gmail.com
> <ma...@gmail.com>>
>
> Hi,
>
> I have a situation wherein I have multiple mappers. I am using
> MultipleInputs class to add them.
>
> Now, I need to pass different parameters to different mappers. For
> e.g. lets say I have a parameter 'num.iterations' that is set
> differently for different mappers. One way to pass parameters to
> mappers is to set them in a Configuration object:
>
> Configuration conf = new Configuration();
> conf.set("num.iterations", "100");
>
> However using this I can't set the value differently for different
> mappers. How do I achieve this?
>
> Thanks,
> Abhinav
>
>
Re: Multiple mappers, different parameter values
Posted by Abhinav M Kulkarni <ab...@gmail.com>.
Hi Mirko,
Thanks for the reply.
Yeah that's one solution.
--
Abhinav
On 04/02/2013 10:04 AM, Mirko Kämpf wrote:
> Hi,
>
> I would add an id to the parameter name: "num.iterations.ID
> <http://num.iterations.ID>=something"
> If your mapper knows what ID it has it can just pick up this value
> from the context. But the question is:
> How does the mapper know about it's ID? Is it related to the input?
> Thank it can be calculated but this is
> a domain specific issue.
>
> Best wishes
> Mirko
>
>
> 2013/4/2 Abhinav M Kulkarni <abhinavkulkarni@gmail.com
> <ma...@gmail.com>>
>
> Hi,
>
> I have a situation wherein I have multiple mappers. I am using
> MultipleInputs class to add them.
>
> Now, I need to pass different parameters to different mappers. For
> e.g. lets say I have a parameter 'num.iterations' that is set
> differently for different mappers. One way to pass parameters to
> mappers is to set them in a Configuration object:
>
> Configuration conf = new Configuration();
> conf.set("num.iterations", "100");
>
> However using this I can't set the value differently for different
> mappers. How do I achieve this?
>
> Thanks,
> Abhinav
>
>
Re: Multiple mappers, different parameter values
Posted by Abhinav M Kulkarni <ab...@gmail.com>.
Hi Mirko,
Thanks for the reply.
Yeah that's one solution.
--
Abhinav
On 04/02/2013 10:04 AM, Mirko Kämpf wrote:
> Hi,
>
> I would add an id to the parameter name: "num.iterations.ID
> <http://num.iterations.ID>=something"
> If your mapper knows what ID it has it can just pick up this value
> from the context. But the question is:
> How does the mapper know about it's ID? Is it related to the input?
> Thank it can be calculated but this is
> a domain specific issue.
>
> Best wishes
> Mirko
>
>
> 2013/4/2 Abhinav M Kulkarni <abhinavkulkarni@gmail.com
> <ma...@gmail.com>>
>
> Hi,
>
> I have a situation wherein I have multiple mappers. I am using
> MultipleInputs class to add them.
>
> Now, I need to pass different parameters to different mappers. For
> e.g. lets say I have a parameter 'num.iterations' that is set
> differently for different mappers. One way to pass parameters to
> mappers is to set them in a Configuration object:
>
> Configuration conf = new Configuration();
> conf.set("num.iterations", "100");
>
> However using this I can't set the value differently for different
> mappers. How do I achieve this?
>
> Thanks,
> Abhinav
>
>
Re: Multiple mappers, different parameter values
Posted by Abhinav M Kulkarni <ab...@gmail.com>.
Hi Mirko,
Thanks for the reply.
Yeah that's one solution.
--
Abhinav
On 04/02/2013 10:04 AM, Mirko Kämpf wrote:
> Hi,
>
> I would add an id to the parameter name: "num.iterations.ID
> <http://num.iterations.ID>=something"
> If your mapper knows what ID it has it can just pick up this value
> from the context. But the question is:
> How does the mapper know about it's ID? Is it related to the input?
> Thank it can be calculated but this is
> a domain specific issue.
>
> Best wishes
> Mirko
>
>
> 2013/4/2 Abhinav M Kulkarni <abhinavkulkarni@gmail.com
> <ma...@gmail.com>>
>
> Hi,
>
> I have a situation wherein I have multiple mappers. I am using
> MultipleInputs class to add them.
>
> Now, I need to pass different parameters to different mappers. For
> e.g. lets say I have a parameter 'num.iterations' that is set
> differently for different mappers. One way to pass parameters to
> mappers is to set them in a Configuration object:
>
> Configuration conf = new Configuration();
> conf.set("num.iterations", "100");
>
> However using this I can't set the value differently for different
> mappers. How do I achieve this?
>
> Thanks,
> Abhinav
>
>
Re: Multiple mappers, different parameter values
Posted by Mirko Kämpf <mi...@gmail.com>.
Hi,
I would add an id to the parameter name: "num.iterations.ID=something"
If your mapper knows what ID it has it can just pick up this value from the
context. But the question is:
How does the mapper know about it's ID? Is it related to the input? Thank
it can be calculated but this is
a domain specific issue.
Best wishes
Mirko
2013/4/2 Abhinav M Kulkarni <ab...@gmail.com>
> Hi,
>
> I have a situation wherein I have multiple mappers. I am using
> MultipleInputs class to add them.
>
> Now, I need to pass different parameters to different mappers. For e.g.
> lets say I have a parameter 'num.iterations' that is set differently for
> different mappers. One way to pass parameters to mappers is to set them in
> a Configuration object:
>
> Configuration conf = new Configuration();
> conf.set("num.iterations", "100");
>
> However using this I can't set the value differently for different
> mappers. How do I achieve this?
>
> Thanks,
> Abhinav
>
Re: Multiple mappers, different parameter values
Posted by Mirko Kämpf <mi...@gmail.com>.
Hi,
I would add an id to the parameter name: "num.iterations.ID=something"
If your mapper knows what ID it has it can just pick up this value from the
context. But the question is:
How does the mapper know about it's ID? Is it related to the input? Thank
it can be calculated but this is
a domain specific issue.
Best wishes
Mirko
2013/4/2 Abhinav M Kulkarni <ab...@gmail.com>
> Hi,
>
> I have a situation wherein I have multiple mappers. I am using
> MultipleInputs class to add them.
>
> Now, I need to pass different parameters to different mappers. For e.g.
> lets say I have a parameter 'num.iterations' that is set differently for
> different mappers. One way to pass parameters to mappers is to set them in
> a Configuration object:
>
> Configuration conf = new Configuration();
> conf.set("num.iterations", "100");
>
> However using this I can't set the value differently for different
> mappers. How do I achieve this?
>
> Thanks,
> Abhinav
>
Re: Multiple mappers, different parameter values
Posted by Mirko Kämpf <mi...@gmail.com>.
Hi,
I would add an id to the parameter name: "num.iterations.ID=something"
If your mapper knows what ID it has it can just pick up this value from the
context. But the question is:
How does the mapper know about it's ID? Is it related to the input? Thank
it can be calculated but this is
a domain specific issue.
Best wishes
Mirko
2013/4/2 Abhinav M Kulkarni <ab...@gmail.com>
> Hi,
>
> I have a situation wherein I have multiple mappers. I am using
> MultipleInputs class to add them.
>
> Now, I need to pass different parameters to different mappers. For e.g.
> lets say I have a parameter 'num.iterations' that is set differently for
> different mappers. One way to pass parameters to mappers is to set them in
> a Configuration object:
>
> Configuration conf = new Configuration();
> conf.set("num.iterations", "100");
>
> However using this I can't set the value differently for different
> mappers. How do I achieve this?
>
> Thanks,
> Abhinav
>
Re: Multiple mappers, different parameter values
Posted by Mirko Kämpf <mi...@gmail.com>.
Hi,
I would add an id to the parameter name: "num.iterations.ID=something"
If your mapper knows what ID it has it can just pick up this value from the
context. But the question is:
How does the mapper know about it's ID? Is it related to the input? Thank
it can be calculated but this is
a domain specific issue.
Best wishes
Mirko
2013/4/2 Abhinav M Kulkarni <ab...@gmail.com>
> Hi,
>
> I have a situation wherein I have multiple mappers. I am using
> MultipleInputs class to add them.
>
> Now, I need to pass different parameters to different mappers. For e.g.
> lets say I have a parameter 'num.iterations' that is set differently for
> different mappers. One way to pass parameters to mappers is to set them in
> a Configuration object:
>
> Configuration conf = new Configuration();
> conf.set("num.iterations", "100");
>
> However using this I can't set the value differently for different
> mappers. How do I achieve this?
>
> Thanks,
> Abhinav
>