You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by "tesmai4@gmail.com" <te...@gmail.com> on 2015/02/21 12:31:55 UTC
Running MapReduce jobs in batch mode on different data sets
Hi,
Is it possible to run jobs on Hadoop in batch mode?
I have 5 different datasets in HDFS and need to run the same MapReduce
application on these datasets sets one after the other.
Right now I am doing it manually How can I automate this?
How can I save the log of each execution in text files for later processing?
Regards,
Tariq
Re: Running MapReduce jobs in batch mode on different data sets
Posted by Artem Ervits <ar...@gmail.com>.
Take a look at Apache Oozie
Artem Ervits
On Feb 21, 2015 6:35 AM, "tesmai4@gmail.com" <te...@gmail.com> wrote:
> Hi,
>
> Is it possible to run jobs on Hadoop in batch mode?
>
> I have 5 different datasets in HDFS and need to run the same MapReduce
> application on these datasets sets one after the other.
>
> Right now I am doing it manually How can I automate this?
>
> How can I save the log of each execution in text files for later
> processing?
>
> Regards,
> Tariq
>
Re: Running MapReduce jobs in batch mode on different data sets
Posted by Artem Ervits <ar...@gmail.com>.
Take a look at Apache Oozie
Artem Ervits
On Feb 21, 2015 6:35 AM, "tesmai4@gmail.com" <te...@gmail.com> wrote:
> Hi,
>
> Is it possible to run jobs on Hadoop in batch mode?
>
> I have 5 different datasets in HDFS and need to run the same MapReduce
> application on these datasets sets one after the other.
>
> Right now I am doing it manually How can I automate this?
>
> How can I save the log of each execution in text files for later
> processing?
>
> Regards,
> Tariq
>
Re: Running MapReduce jobs in batch mode on different data sets
Posted by Artem Ervits <ar...@gmail.com>.
Take a look at Apache Oozie
Artem Ervits
On Feb 21, 2015 6:35 AM, "tesmai4@gmail.com" <te...@gmail.com> wrote:
> Hi,
>
> Is it possible to run jobs on Hadoop in batch mode?
>
> I have 5 different datasets in HDFS and need to run the same MapReduce
> application on these datasets sets one after the other.
>
> Right now I am doing it manually How can I automate this?
>
> How can I save the log of each execution in text files for later
> processing?
>
> Regards,
> Tariq
>
Re: Running MapReduce jobs in batch mode on different data sets
Posted by Artem Ervits <ar...@gmail.com>.
Take a look at Apache Oozie
Artem Ervits
On Feb 21, 2015 6:35 AM, "tesmai4@gmail.com" <te...@gmail.com> wrote:
> Hi,
>
> Is it possible to run jobs on Hadoop in batch mode?
>
> I have 5 different datasets in HDFS and need to run the same MapReduce
> application on these datasets sets one after the other.
>
> Right now I am doing it manually How can I automate this?
>
> How can I save the log of each execution in text files for later
> processing?
>
> Regards,
> Tariq
>