oozie and task retries

[Joe Crobak <jo...@gmail.com>]

We experienced a tasktracker die yesterday during an oozie workflow run.
The tasktracker was running a mapper for the oozie driver, so the hadoop
framework retried the task on another node. This job was a java job that
included a <prepare> section to delete the output path and then invoke a
main class that starts a mapreduce job. It seems that the prepare was not
run again... is that the architecture of oozie (to run prepare before
launching the task)?  If so, is there a best practice for avoiding this
problem?

Second question - do people use oozie with fair-scheduler preemption? If
so, how? We thought about having a separate pool for the oozie launcher
tasks where minMaps = maxMaps, but then we need to make sure to set
oozie.launcher.pool.name in every single workflow that we define. Any
recommendations?

We're using oozie from cdh3u3 if that makes a difference (maybe a newer
version has some improvements that we should know about?).

Thanks,
Joe

Re: oozie and task retries

[Alejandro Abdelnur <tu...@gmail.com>]

Joe,

Currently the <prepare> block is run by Oozie server before starting
the launcher job, thus task retry will not run it. This has been
addressed in trunk and it will be part of Oozie 3.2.

Regarding having to specify the pool name in every action, this is
still the case in Oozie 3. There are some proposed changed that would
simplify this (global configs) but this has not been addressed yet.

Thxs.

Alejandro

On Fri, Mar 23, 2012 at 8:38 AM, Joe Crobak <jo...@gmail.com> wrote:
> We experienced a tasktracker die yesterday during an oozie workflow run.
> The tasktracker was running a mapper for the oozie driver, so the hadoop
> framework retried the task on another node. This job was a java job that
> included a <prepare> section to delete the output path and then invoke a
> main class that starts a mapreduce job. It seems that the prepare was not
> run again... is that the architecture of oozie (to run prepare before
> launching the task)?  If so, is there a best practice for avoiding this
> problem?
>
> Second question - do people use oozie with fair-scheduler preemption? If
> so, how? We thought about having a separate pool for the oozie launcher
> tasks where minMaps = maxMaps, but then we need to make sure to set
> oozie.launcher.pool.name in every single workflow that we define. Any
> recommendations?
>
> We're using oozie from cdh3u3 if that makes a difference (maybe a newer
> version has some improvements that we should know about?).
>
> Thanks,
> Joe