You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Roshan James <ro...@gmail.com> on 2009/06/17 21:43:04 UTC
JobControl for Pipes?
Hello, Is there any way to express dependencies between map-reduce jobs
(such as in org.apache.hadoop.mapred.jobcontrol) for pipes jobs? The
provided header Pipes.hh does not seem to reflect any such capabilities.
best,
Roshan
Re: JobControl for Pipes?
Posted by jason hadoop <ja...@gmail.com>.
http://www.cascading.org/
https://issues.apache.org/jira/browse/HADOOP-5303 (oozie)
On Thu, Jun 18, 2009 at 6:19 AM, Roshan James <
roshan.james.subscription@gmail.com> wrote:
> Can you give me a url or so to both? I cant seem to find either one after a
> couple of basic web searches.
>
> Also, when you say JobControl is coming to Hadoop - I can already see the
> Java JobControl classes that lets one express dependancies between jobs. So
> I assume this already works in Java - does it not? I was asking if this
> functionality is exposed via Pipes in some way.
>
> Roshan
>
>
> On Wed, Jun 17, 2009 at 10:59 PM, jason hadoop <jason.hadoop@gmail.com
> >wrote:
>
> > Job control is coming with the Hadoop WorkFlow manager, in the mean time
> > there is cascade by chris wensel. I do not have any personal experience
> > with
> > either. I do not know how pipes interacts with either.
> >
> > On Wed, Jun 17, 2009 at 12:43 PM, Roshan James <
> > roshan.james.subscription@gmail.com> wrote:
> >
> > > Hello, Is there any way to express dependencies between map-reduce jobs
> > > (such as in org.apache.hadoop.mapred.jobcontrol) for pipes jobs? The
> > > provided header Pipes.hh does not seem to reflect any such
> capabilities.
> > >
> > > best,
> > > Roshan
> > >
> >
> >
> >
> > --
> > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > www.prohadoopbook.com a community for Hadoop Professionals
> >
>
--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals
Re: JobControl for Pipes?
Posted by Roshan James <ro...@gmail.com>.
Can you give me a url or so to both? I cant seem to find either one after a
couple of basic web searches.
Also, when you say JobControl is coming to Hadoop - I can already see the
Java JobControl classes that lets one express dependancies between jobs. So
I assume this already works in Java - does it not? I was asking if this
functionality is exposed via Pipes in some way.
Roshan
On Wed, Jun 17, 2009 at 10:59 PM, jason hadoop <ja...@gmail.com>wrote:
> Job control is coming with the Hadoop WorkFlow manager, in the mean time
> there is cascade by chris wensel. I do not have any personal experience
> with
> either. I do not know how pipes interacts with either.
>
> On Wed, Jun 17, 2009 at 12:43 PM, Roshan James <
> roshan.james.subscription@gmail.com> wrote:
>
> > Hello, Is there any way to express dependencies between map-reduce jobs
> > (such as in org.apache.hadoop.mapred.jobcontrol) for pipes jobs? The
> > provided header Pipes.hh does not seem to reflect any such capabilities.
> >
> > best,
> > Roshan
> >
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>
Re: JobControl for Pipes?
Posted by jason hadoop <ja...@gmail.com>.
Job control is coming with the Hadoop WorkFlow manager, in the mean time
there is cascade by chris wensel. I do not have any personal experience with
either. I do not know how pipes interacts with either.
On Wed, Jun 17, 2009 at 12:43 PM, Roshan James <
roshan.james.subscription@gmail.com> wrote:
> Hello, Is there any way to express dependencies between map-reduce jobs
> (such as in org.apache.hadoop.mapred.jobcontrol) for pipes jobs? The
> provided header Pipes.hh does not seem to reflect any such capabilities.
>
> best,
> Roshan
>
--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals