You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by "tesmai4@gmail.com" <te...@gmail.com> on 2015/02/21 12:31:55 UTC

Running MapReduce jobs in batch mode on different data sets

Hi,

Is it possible to run jobs on Hadoop in batch mode?

I have 5 different datasets in HDFS and need to run the same MapReduce
application on these datasets sets one after the other.

Right now I am doing it manually How can I automate this?

How can I save the log of each execution in text files for later processing?

Regards,
Tariq

Re: Running MapReduce jobs in batch mode on different data sets

Posted by Artem Ervits <ar...@gmail.com>.
Take a look at Apache Oozie

Artem Ervits
On Feb 21, 2015 6:35 AM, "tesmai4@gmail.com" <te...@gmail.com> wrote:

> Hi,
>
> Is it possible to run jobs on Hadoop in batch mode?
>
> I have 5 different datasets in HDFS and need to run the same MapReduce
> application on these datasets sets one after the other.
>
> Right now I am doing it manually How can I automate this?
>
> How can I save the log of each execution in text files for later
> processing?
>
> Regards,
> Tariq
>

Re: Running MapReduce jobs in batch mode on different data sets

Posted by Artem Ervits <ar...@gmail.com>.
Take a look at Apache Oozie

Artem Ervits
On Feb 21, 2015 6:35 AM, "tesmai4@gmail.com" <te...@gmail.com> wrote:

> Hi,
>
> Is it possible to run jobs on Hadoop in batch mode?
>
> I have 5 different datasets in HDFS and need to run the same MapReduce
> application on these datasets sets one after the other.
>
> Right now I am doing it manually How can I automate this?
>
> How can I save the log of each execution in text files for later
> processing?
>
> Regards,
> Tariq
>

Re: Running MapReduce jobs in batch mode on different data sets

Posted by Artem Ervits <ar...@gmail.com>.
Take a look at Apache Oozie

Artem Ervits
On Feb 21, 2015 6:35 AM, "tesmai4@gmail.com" <te...@gmail.com> wrote:

> Hi,
>
> Is it possible to run jobs on Hadoop in batch mode?
>
> I have 5 different datasets in HDFS and need to run the same MapReduce
> application on these datasets sets one after the other.
>
> Right now I am doing it manually How can I automate this?
>
> How can I save the log of each execution in text files for later
> processing?
>
> Regards,
> Tariq
>

Re: Running MapReduce jobs in batch mode on different data sets

Posted by Artem Ervits <ar...@gmail.com>.
Take a look at Apache Oozie

Artem Ervits
On Feb 21, 2015 6:35 AM, "tesmai4@gmail.com" <te...@gmail.com> wrote:

> Hi,
>
> Is it possible to run jobs on Hadoop in batch mode?
>
> I have 5 different datasets in HDFS and need to run the same MapReduce
> application on these datasets sets one after the other.
>
> Right now I am doing it manually How can I automate this?
>
> How can I save the log of each execution in text files for later
> processing?
>
> Regards,
> Tariq
>