Fwd: How to automate running pig scripts

[Prashant Kommireddi <pr...@gmail.com>]

Forwarding your question to the Oozie mailing list.  cc Pig users

Does Oozie work with pig jobs running in local mode or only in MR mode?

Thanks,

---------- Forwarded message ----------
From: Juan Martin Pampliega <jp...@gmail.com>
Date: Sat, May 26, 2012 at 12:23 PM
Subject: Re: How to automate running pig scripts
To: user@pig.apache.org


Thanks a lot Prashant. This seems exactly what I was looking for. Just one
doubt I have and it doesn't seem to be clear from the documentation. Do
this workflow managers work with pig jobs running in local mode or only in
MR mode?

On Sat, May 26, 2012 at 3:19 PM, Prashant Kommireddi <prash1784@gmail.com
>wrote:

> Take a look at Oozie or Azkaban.
>
> http://incubator.apache.org/oozie/
>
> I prefer Oozie, its a lot more comprehensive. AZK does not seem to be
fully
> baked (though installation and getting started could be easier).
>
> On Sat, May 26, 2012 at 11:09 AM, Juan Martin Pampliega <
> jpampliega@gmail.com> wrote:
>
> > Hi,
> >
> > I need to have pig scripts run automatically every certain amount of
> time.
> > Also, I need to know if a script is still running so that the next one
> does
> > not start until the previous one has finished and also I need to
register
> > somewhere the end state of each script run to display it in a table on a
> > web page.
> > My last requirement is to have the scripts receive some parameters that
> > will be from which id to start processing. So for example the second
time
> > the script is run it should know which was the last id that was
processed
> > and start from there onwards.
> >
> > Does anyone have any suggestions on how to approach this?
> >
> > I was thinking of using Jython to run the Pig scripts programmatically
> but
> > I'm having trouble calling all these from Java and I'm always getting
> > success status even when the pig script fails.
> >
> > Any other suggestions?
> >
>

Re: How to automate running pig scripts

[Alejandro Abdelnur <tu...@cloudera.com>]

MR mode only

thx

On Sat, May 26, 2012 at 12:35 PM, Prashant Kommireddi
<pr...@gmail.com>wrote:

> Forwarding your question to the Oozie mailing list.  cc Pig users
>
> Does Oozie work with pig jobs running in local mode or only in MR mode?
>
> Thanks,
>
> ---------- Forwarded message ----------
> From: Juan Martin Pampliega <jp...@gmail.com>
> Date: Sat, May 26, 2012 at 12:23 PM
> Subject: Re: How to automate running pig scripts
> To: user@pig.apache.org
>
>
> Thanks a lot Prashant. This seems exactly what I was looking for. Just one
> doubt I have and it doesn't seem to be clear from the documentation. Do
> this workflow managers work with pig jobs running in local mode or only in
> MR mode?
>
> On Sat, May 26, 2012 at 3:19 PM, Prashant Kommireddi <prash1784@gmail.com
> >wrote:
>
> > Take a look at Oozie or Azkaban.
> >
> > http://incubator.apache.org/oozie/
> >
> > I prefer Oozie, its a lot more comprehensive. AZK does not seem to be
> fully
> > baked (though installation and getting started could be easier).
> >
> > On Sat, May 26, 2012 at 11:09 AM, Juan Martin Pampliega <
> > jpampliega@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I need to have pig scripts run automatically every certain amount of
> > time.
> > > Also, I need to know if a script is still running so that the next one
> > does
> > > not start until the previous one has finished and also I need to
> register
> > > somewhere the end state of each script run to display it in a table on
> a
> > > web page.
> > > My last requirement is to have the scripts receive some parameters that
> > > will be from which id to start processing. So for example the second
> time
> > > the script is run it should know which was the last id that was
> processed
> > > and start from there onwards.
> > >
> > > Does anyone have any suggestions on how to approach this?
> > >
> > > I was thinking of using Jython to run the Pig scripts programmatically
> > but
> > > I'm having trouble calling all these from Java and I'm always getting
> > > success status even when the pig script fails.
> > >
> > > Any other suggestions?
> > >
> >
>



-- 
Alejandro