You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by Ross Laidlaw <rl...@gmail.com> on 2015/05/08 20:14:21 UTC
Request for advice on workflows
Hi All,
I'm working on an OODT setup (v0.8.1) with several workflow tasks defined
and working well individually. I can use the wmgr-client to fire off jobs
separately and all appears to be working as planned. I'd like to combine
these tasks into a workflow so that I don't have to run each set of jobs.
For example, in the workflow I'd like to run several 'task A' jobs and when
all these have finished I'd like the workflow to automatically start
several 'task B' jobs, etc.
I found some example workflows in the distribution, plus several guides up
on the wiki. Is there a current recommended way of setting up a simple
workflow? Should I use workflow v2 or is it best to stick with v1? Once
I've set up a workflow, how do I start it off? It looks like it might be
possible to nest workflows, is that functionality available yet?
Additionally, what's the best way to check on the status of currently
running jobs? I've used the pcs_stat tool a lot and it's great for
checking on the overall status, but is there a recommended way to check
e.g. if any 'task A' jobs are still running for a certain workflow and what
state they're currently in?
Many thanks in advance for your advice,
Ross
Re: Request for advice on workflows
Posted by Ross Laidlaw <rl...@gmail.com>.
Hi Chris,
Many thanks for your advice. I'll check out the PackagedWorkflowRepository and will let you know how it goes.
Ross
> On May 8, 2015, at 12:58 PM, Mattmann, Chris A (3980) <ch...@jpl.nasa.gov> wrote:
>
> Hey Ross:
>
> Great work! Try the PackagedWorkflowRepository. The best documentation
> that exists of it is in OODT-70 [1].
>
> I would use that to support the type of workflows that you’re doing.
> If this works we should make it the default since it’s supposed to
> be back compat with the XMLWorkflowRepositoryManager.
>
> Cheers,
> Chris
>
> P.S. see some of the ASO scripts I wrote to check on specific task/
> PGE status, etc.
>
>
> [1]https://issues.apache.org/jira/browse/OODT-70
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
> -----Original Message-----
> From: Ross Laidlaw <rl...@gmail.com>
> Reply-To: "dev@oodt.apache.org" <de...@oodt.apache.org>
> Date: Friday, May 8, 2015 at 8:14 AM
> To: "dev@oodt.apache.org" <de...@oodt.apache.org>
> Subject: Request for advice on workflows
>
>> Hi All,
>>
>> I'm working on an OODT setup (v0.8.1) with several workflow tasks defined
>> and working well individually. I can use the wmgr-client to fire off jobs
>> separately and all appears to be working as planned. I'd like to combine
>> these tasks into a workflow so that I don't have to run each set of jobs.
>> For example, in the workflow I'd like to run several 'task A' jobs and
>> when
>> all these have finished I'd like the workflow to automatically start
>> several 'task B' jobs, etc.
>>
>> I found some example workflows in the distribution, plus several guides up
>> on the wiki. Is there a current recommended way of setting up a simple
>> workflow? Should I use workflow v2 or is it best to stick with v1? Once
>> I've set up a workflow, how do I start it off? It looks like it might be
>> possible to nest workflows, is that functionality available yet?
>>
>> Additionally, what's the best way to check on the status of currently
>> running jobs? I've used the pcs_stat tool a lot and it's great for
>> checking on the overall status, but is there a recommended way to check
>> e.g. if any 'task A' jobs are still running for a certain workflow and
>> what
>> state they're currently in?
>>
>> Many thanks in advance for your advice,
>>
>>
>> Ross
>
Re: Request for advice on workflows
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
Hey Ross:
Great work! Try the PackagedWorkflowRepository. The best documentation
that exists of it is in OODT-70 [1].
I would use that to support the type of workflows that you’re doing.
If this works we should make it the default since it’s supposed to
be back compat with the XMLWorkflowRepositoryManager.
Cheers,
Chris
P.S. see some of the ASO scripts I wrote to check on specific task/
PGE status, etc.
[1]https://issues.apache.org/jira/browse/OODT-70
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Ross Laidlaw <rl...@gmail.com>
Reply-To: "dev@oodt.apache.org" <de...@oodt.apache.org>
Date: Friday, May 8, 2015 at 8:14 AM
To: "dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: Request for advice on workflows
>Hi All,
>
>I'm working on an OODT setup (v0.8.1) with several workflow tasks defined
>and working well individually. I can use the wmgr-client to fire off jobs
>separately and all appears to be working as planned. I'd like to combine
>these tasks into a workflow so that I don't have to run each set of jobs.
>For example, in the workflow I'd like to run several 'task A' jobs and
>when
>all these have finished I'd like the workflow to automatically start
>several 'task B' jobs, etc.
>
>I found some example workflows in the distribution, plus several guides up
>on the wiki. Is there a current recommended way of setting up a simple
>workflow? Should I use workflow v2 or is it best to stick with v1? Once
>I've set up a workflow, how do I start it off? It looks like it might be
>possible to nest workflows, is that functionality available yet?
>
>Additionally, what's the best way to check on the status of currently
>running jobs? I've used the pcs_stat tool a lot and it's great for
>checking on the overall status, but is there a recommended way to check
>e.g. if any 'task A' jobs are still running for a certain workflow and
>what
>state they're currently in?
>
>Many thanks in advance for your advice,
>
>
>Ross