You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Aditya Desai <ad...@gmail.com> on 2009/01/18 18:58:53 UTC

Calling a mapreduce job from inside another

Is it possible to call a mapreduce job from inside another, if yes how?
and is it possible to disable the reducer completely that is suspend the job
immediately after call to map has been terminated.
I have tried -reducer "NONE". I am using the streaming api to code in python

Regards,
Aditya Desai.

Re: Calling a mapreduce job from inside another

Posted by Amareshwari Sriramadasu <am...@yahoo-inc.com>.
You can use Job Control.
See
http://hadoop.apache.org/core/docs/r0.19.0/mapred_tutorial.html#Job+Control
http://hadoop.apache.org/core/docs/r0.19.0/api/org/apache/hadoop/mapred/jobcontrol/Job.html
and
http://hadoop.apache.org/core/docs/r0.19.0/api/org/apache/hadoop/mapred/jobcontrol/JobControl.html

Thanks
Amareshwari
Aditya Desai wrote:
> Is it possible to call a mapreduce job from inside another, if yes how?
> and is it possible to disable the reducer completely that is suspend the job
> immediately after call to map has been terminated.
> I have tried -reducer "NONE". I am using the streaming api to code in python
>
> Regards,
> Aditya Desai.
>
>

Calling a mapreduce job from inside another

Posted by Aditya Desai <ad...@gmail.com>.
Is it possible to call a mapreduce job from inside another, if yes how?
and is it possible to disable the reducer completely that is suspend the job
immediately after call to map has been terminated.
I have tried -reducer "NONE". I am using the streaming api to code in python

Regards,
Aditya Desai.

Re: Calling a mapreduce job from inside another

Posted by Sagar Naik <sn...@attributor.com>.
You can also play with the priority of the jobs to have the innermost 
job finish first

-Sagar

Devaraj Das wrote:
> You can chain job submissions at the client. Also, you can run more than one
> job in parallel (if you have enough task slots). An example of chaining jobs
> is there in src/examples/org/apache/hadoop/examples/Grep.java where the jobs
> grep-search and grep-sort are chained..
>
>
> On 1/18/09 9:58 AM, "Aditya Desai" <ad...@gmail.com> wrote:
>
>   
>> Is it possible to call a mapreduce job from inside another, if yes how?
>> and is it possible to disable the reducer completely that is suspend the job
>> immediately after call to map has been terminated.
>> I have tried -reducer "NONE". I am using the streaming api to code in python
>>
>> Regards,
>> Aditya Desai.
>>     
>
>
>

Re: Calling a mapreduce job from inside another

Posted by Devaraj Das <dd...@yahoo-inc.com>.
You can chain job submissions at the client. Also, you can run more than one
job in parallel (if you have enough task slots). An example of chaining jobs
is there in src/examples/org/apache/hadoop/examples/Grep.java where the jobs
grep-search and grep-sort are chained..


On 1/18/09 9:58 AM, "Aditya Desai" <ad...@gmail.com> wrote:

> Is it possible to call a mapreduce job from inside another, if yes how?
> and is it possible to disable the reducer completely that is suspend the job
> immediately after call to map has been terminated.
> I have tried -reducer "NONE". I am using the streaming api to code in python
> 
> Regards,
> Aditya Desai.