You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> on 2012/03/02 04:59:43 UTC
Re: Three Workflow questions
Hi Keith,
Thanks for your questions! Happy to answer them below:
On Mar 1, 2012, at 9:54 AM, Keith Cummings wrote:
> Hi Chris,
> I have a few questions that I hope won't take you but a minute or two to
> answer. If you prefer I send such questions to the OODT email list just
> let me know.
>
> 1. The example code with the Workflow Manager shows examples of
> pre-conditions to tasks. How do you configure post-conditions to tasks?
In the new wengine-style XML workflow documents, available in
trunk with the PackagedWorkflowRepository that came with OODT-70 [1],
you specify them with something like this:
<conditions execution="parallel" type="pre">
<condition id-ref="urn:npp:MOA_IASI_L1C_Daily"/>
<condition id-ref="urn:npp:MOA_MHS_L1B_Daily"/>
<condition id-ref="urn:npp:MOA_AMSUA_L1B_Daily"/>
</conditions>
Note that the conditions tag block can have an attribute 'type',
that you can set to post or to pre. At the moment, it simply reads
this, but as soon as the wengine is fully integrated, it will support
pre or post execution of these conditions. That should ship with 0.4.
>
> 2. Is it possible to branch a workflow? If so, can you point me to an
> example?
Sure, it's possible even in the 0.1-incubating, 0.2 and 0.3 versions.
The way that you branch a workflow is to:
1. add a simple WorkflowTaskInstance implementation, whose goal
is to send an Event *back* to the same workflow manager, to kick off
another workflow. You can find a simple example of such a redirector
here [2].
2. wire up an event for the next stage of the branch to kick off the
branch workflow. So, if you had a simple workflow:
----> t2---->t4
t1-/
\
---->t3
You could support this with the following:
w1: t1, branch redirector (seeded with e2,e3)
w2: t2, t4
w3: t3
e1: w1
e2: w2
e3: w3
Where w=workflow, e=event, t=task
That make sense? If you need to fan in, you can support
this with a precondition on the incoming fan task, holding until
the other tasks in the workflow have finished, and then going.
>
> 3. How do you interrupt a workflow from within? Externally, you can
> just issue a stop command on the command-line for this workflow. If a
> task (in the Java code) found that an error condition occurred, how
> could that be communicated back to the workflow manager? (I found that
> throwing an Exception does the trick, but I was wondering if there was a
> less obtrusive way to signal the workflow to stop.)
You can, inside of a WorkflowTaskInstance, do something like:
XmlRpcWorfklowManager wm = new XmlRpcWorkflowManager(new URL(met.getMetadata("WorkflowUrl"));
wm.stopWorkflowInstance(met.getMetadata("WorkflowInstId"));
That will basically have the same effect as the external stop. Exceptions work fine too,
even if they don't look elegant :) Would be nice to have a WorkflowStoppedException, which extends
some generic WorkflowTaskInstanceException, but for now, a good ol' Exception will do it itoo.
HTH!
Cheers,
Chris
[1] https://issues.apache.org/jira/browse/OODT-70
[2] http://s.apache.org/NeK
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Three Workflow questions
Posted by Bruce Barkstrom <br...@gmail.com>.
I think I said "yes" - is there a special form of e-mail you want?
On Fri, Mar 2, 2012 at 1:46 PM, Mattmann, Chris A (388J)
<ch...@jpl.nasa.gov> wrote:
> Hey Bruce,
>
> Well right now there is some standardization in that our science algorithm
> wrapper tries to both detect error codes and to use Workflow or Task specific
> context metadata information to determine if the algorithm set a failure flag.
> Your specifics below are similar too since you always can get the error log
> or error message from the integrated algorithm and yes it does speed up
> debugging.
>
> Now, to go back to the other discussion. Can you please respond to my
> invite email to the OODT community? Emails like the below are plenty
> enough credit (not that you need any more) and reason for you to accept
> the invite and keep hanging around the community. :)
>
> Thanks,
> Chris
>
> On Mar 2, 2012, at 6:50 AM, Bruce Barkstrom wrote:
>
>> As a generic question, has there been any standardization of
>> using exceptions to identify where something happened?
>> I have been coding so that every procedure has a Boolean
>> variable named 'OK' and a variable length string named 'Err_Msg'.
>> The string is built so that it identifies the procedure and the
>> last good location in the procedure. That really speeds up
>> debugging.
>>
>> Bruce B.
>>
>> On Thu, Mar 1, 2012 at 10:59 PM, Mattmann, Chris A (388J)
>> <ch...@jpl.nasa.gov> wrote:
>>> Hi Keith,
>>>
>>> Thanks for your questions! Happy to answer them below:
>>>
>>> On Mar 1, 2012, at 9:54 AM, Keith Cummings wrote:
>>>
>>>> Hi Chris,
>>>> I have a few questions that I hope won't take you but a minute or two to
>>>> answer. If you prefer I send such questions to the OODT email list just
>>>> let me know.
>>>>
>>>> 1. The example code with the Workflow Manager shows examples of
>>>> pre-conditions to tasks. How do you configure post-conditions to tasks?
>>>
>>> In the new wengine-style XML workflow documents, available in
>>> trunk with the PackagedWorkflowRepository that came with OODT-70 [1],
>>> you specify them with something like this:
>>>
>>> <conditions execution="parallel" type="pre">
>>> <condition id-ref="urn:npp:MOA_IASI_L1C_Daily"/>
>>> <condition id-ref="urn:npp:MOA_MHS_L1B_Daily"/>
>>> <condition id-ref="urn:npp:MOA_AMSUA_L1B_Daily"/>
>>> </conditions>
>>>
>>> Note that the conditions tag block can have an attribute 'type',
>>> that you can set to post or to pre. At the moment, it simply reads
>>> this, but as soon as the wengine is fully integrated, it will support
>>> pre or post execution of these conditions. That should ship with 0.4.
>>>
>>>>
>>>> 2. Is it possible to branch a workflow? If so, can you point me to an
>>>> example?
>>>
>>> Sure, it's possible even in the 0.1-incubating, 0.2 and 0.3 versions.
>>> The way that you branch a workflow is to:
>>>
>>> 1. add a simple WorkflowTaskInstance implementation, whose goal
>>> is to send an Event *back* to the same workflow manager, to kick off
>>> another workflow. You can find a simple example of such a redirector
>>> here [2].
>>>
>>> 2. wire up an event for the next stage of the branch to kick off the
>>> branch workflow. So, if you had a simple workflow:
>>>
>>> ----> t2---->t4
>>> t1-/
>>> \
>>> ---->t3
>>>
>>> You could support this with the following:
>>>
>>> w1: t1, branch redirector (seeded with e2,e3)
>>> w2: t2, t4
>>> w3: t3
>>>
>>> e1: w1
>>> e2: w2
>>> e3: w3
>>>
>>> Where w=workflow, e=event, t=task
>>>
>>> That make sense? If you need to fan in, you can support
>>> this with a precondition on the incoming fan task, holding until
>>> the other tasks in the workflow have finished, and then going.
>>>
>>>>
>>>> 3. How do you interrupt a workflow from within? Externally, you can
>>>> just issue a stop command on the command-line for this workflow. If a
>>>> task (in the Java code) found that an error condition occurred, how
>>>> could that be communicated back to the workflow manager? (I found that
>>>> throwing an Exception does the trick, but I was wondering if there was a
>>>> less obtrusive way to signal the workflow to stop.)
>>>
>>> You can, inside of a WorkflowTaskInstance, do something like:
>>>
>>> XmlRpcWorfklowManager wm = new XmlRpcWorkflowManager(new URL(met.getMetadata("WorkflowUrl"));
>>> wm.stopWorkflowInstance(met.getMetadata("WorkflowInstId"));
>>>
>>> That will basically have the same effect as the external stop. Exceptions work fine too,
>>> even if they don't look elegant :) Would be nice to have a WorkflowStoppedException, which extends
>>> some generic WorkflowTaskInstanceException, but for now, a good ol' Exception will do it itoo.
>>>
>>> HTH!
>>>
>>> Cheers,
>>> Chris
>>>
>>> [1] https://issues.apache.org/jira/browse/OODT-70
>>> [2] http://s.apache.org/NeK
>>>
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattmann@nasa.gov
>>> WWW: http://sunset.usc.edu/~mattmann/
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> Phone: +1 (818) 354-8810
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
Re: Three Workflow questions
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Bruce,
Well right now there is some standardization in that our science algorithm
wrapper tries to both detect error codes and to use Workflow or Task specific
context metadata information to determine if the algorithm set a failure flag.
Your specifics below are similar too since you always can get the error log
or error message from the integrated algorithm and yes it does speed up
debugging.
Now, to go back to the other discussion. Can you please respond to my
invite email to the OODT community? Emails like the below are plenty
enough credit (not that you need any more) and reason for you to accept
the invite and keep hanging around the community. :)
Thanks,
Chris
On Mar 2, 2012, at 6:50 AM, Bruce Barkstrom wrote:
> As a generic question, has there been any standardization of
> using exceptions to identify where something happened?
> I have been coding so that every procedure has a Boolean
> variable named 'OK' and a variable length string named 'Err_Msg'.
> The string is built so that it identifies the procedure and the
> last good location in the procedure. That really speeds up
> debugging.
>
> Bruce B.
>
> On Thu, Mar 1, 2012 at 10:59 PM, Mattmann, Chris A (388J)
> <ch...@jpl.nasa.gov> wrote:
>> Hi Keith,
>>
>> Thanks for your questions! Happy to answer them below:
>>
>> On Mar 1, 2012, at 9:54 AM, Keith Cummings wrote:
>>
>>> Hi Chris,
>>> I have a few questions that I hope won't take you but a minute or two to
>>> answer. If you prefer I send such questions to the OODT email list just
>>> let me know.
>>>
>>> 1. The example code with the Workflow Manager shows examples of
>>> pre-conditions to tasks. How do you configure post-conditions to tasks?
>>
>> In the new wengine-style XML workflow documents, available in
>> trunk with the PackagedWorkflowRepository that came with OODT-70 [1],
>> you specify them with something like this:
>>
>> <conditions execution="parallel" type="pre">
>> <condition id-ref="urn:npp:MOA_IASI_L1C_Daily"/>
>> <condition id-ref="urn:npp:MOA_MHS_L1B_Daily"/>
>> <condition id-ref="urn:npp:MOA_AMSUA_L1B_Daily"/>
>> </conditions>
>>
>> Note that the conditions tag block can have an attribute 'type',
>> that you can set to post or to pre. At the moment, it simply reads
>> this, but as soon as the wengine is fully integrated, it will support
>> pre or post execution of these conditions. That should ship with 0.4.
>>
>>>
>>> 2. Is it possible to branch a workflow? If so, can you point me to an
>>> example?
>>
>> Sure, it's possible even in the 0.1-incubating, 0.2 and 0.3 versions.
>> The way that you branch a workflow is to:
>>
>> 1. add a simple WorkflowTaskInstance implementation, whose goal
>> is to send an Event *back* to the same workflow manager, to kick off
>> another workflow. You can find a simple example of such a redirector
>> here [2].
>>
>> 2. wire up an event for the next stage of the branch to kick off the
>> branch workflow. So, if you had a simple workflow:
>>
>> ----> t2---->t4
>> t1-/
>> \
>> ---->t3
>>
>> You could support this with the following:
>>
>> w1: t1, branch redirector (seeded with e2,e3)
>> w2: t2, t4
>> w3: t3
>>
>> e1: w1
>> e2: w2
>> e3: w3
>>
>> Where w=workflow, e=event, t=task
>>
>> That make sense? If you need to fan in, you can support
>> this with a precondition on the incoming fan task, holding until
>> the other tasks in the workflow have finished, and then going.
>>
>>>
>>> 3. How do you interrupt a workflow from within? Externally, you can
>>> just issue a stop command on the command-line for this workflow. If a
>>> task (in the Java code) found that an error condition occurred, how
>>> could that be communicated back to the workflow manager? (I found that
>>> throwing an Exception does the trick, but I was wondering if there was a
>>> less obtrusive way to signal the workflow to stop.)
>>
>> You can, inside of a WorkflowTaskInstance, do something like:
>>
>> XmlRpcWorfklowManager wm = new XmlRpcWorkflowManager(new URL(met.getMetadata("WorkflowUrl"));
>> wm.stopWorkflowInstance(met.getMetadata("WorkflowInstId"));
>>
>> That will basically have the same effect as the external stop. Exceptions work fine too,
>> even if they don't look elegant :) Would be nice to have a WorkflowStoppedException, which extends
>> some generic WorkflowTaskInstanceException, but for now, a good ol' Exception will do it itoo.
>>
>> HTH!
>>
>> Cheers,
>> Chris
>>
>> [1] https://issues.apache.org/jira/browse/OODT-70
>> [2] http://s.apache.org/NeK
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
Phone: +1 (818) 354-8810
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Three Workflow questions
Posted by Bruce Barkstrom <br...@gmail.com>.
As a generic question, has there been any standardization of
using exceptions to identify where something happened?
I have been coding so that every procedure has a Boolean
variable named 'OK' and a variable length string named 'Err_Msg'.
The string is built so that it identifies the procedure and the
last good location in the procedure. That really speeds up
debugging.
Bruce B.
On Thu, Mar 1, 2012 at 10:59 PM, Mattmann, Chris A (388J)
<ch...@jpl.nasa.gov> wrote:
> Hi Keith,
>
> Thanks for your questions! Happy to answer them below:
>
> On Mar 1, 2012, at 9:54 AM, Keith Cummings wrote:
>
>> Hi Chris,
>> I have a few questions that I hope won't take you but a minute or two to
>> answer. If you prefer I send such questions to the OODT email list just
>> let me know.
>>
>> 1. The example code with the Workflow Manager shows examples of
>> pre-conditions to tasks. How do you configure post-conditions to tasks?
>
> In the new wengine-style XML workflow documents, available in
> trunk with the PackagedWorkflowRepository that came with OODT-70 [1],
> you specify them with something like this:
>
> <conditions execution="parallel" type="pre">
> <condition id-ref="urn:npp:MOA_IASI_L1C_Daily"/>
> <condition id-ref="urn:npp:MOA_MHS_L1B_Daily"/>
> <condition id-ref="urn:npp:MOA_AMSUA_L1B_Daily"/>
> </conditions>
>
> Note that the conditions tag block can have an attribute 'type',
> that you can set to post or to pre. At the moment, it simply reads
> this, but as soon as the wengine is fully integrated, it will support
> pre or post execution of these conditions. That should ship with 0.4.
>
>>
>> 2. Is it possible to branch a workflow? If so, can you point me to an
>> example?
>
> Sure, it's possible even in the 0.1-incubating, 0.2 and 0.3 versions.
> The way that you branch a workflow is to:
>
> 1. add a simple WorkflowTaskInstance implementation, whose goal
> is to send an Event *back* to the same workflow manager, to kick off
> another workflow. You can find a simple example of such a redirector
> here [2].
>
> 2. wire up an event for the next stage of the branch to kick off the
> branch workflow. So, if you had a simple workflow:
>
> ----> t2---->t4
> t1-/
> \
> ---->t3
>
> You could support this with the following:
>
> w1: t1, branch redirector (seeded with e2,e3)
> w2: t2, t4
> w3: t3
>
> e1: w1
> e2: w2
> e3: w3
>
> Where w=workflow, e=event, t=task
>
> That make sense? If you need to fan in, you can support
> this with a precondition on the incoming fan task, holding until
> the other tasks in the workflow have finished, and then going.
>
>>
>> 3. How do you interrupt a workflow from within? Externally, you can
>> just issue a stop command on the command-line for this workflow. If a
>> task (in the Java code) found that an error condition occurred, how
>> could that be communicated back to the workflow manager? (I found that
>> throwing an Exception does the trick, but I was wondering if there was a
>> less obtrusive way to signal the workflow to stop.)
>
> You can, inside of a WorkflowTaskInstance, do something like:
>
> XmlRpcWorfklowManager wm = new XmlRpcWorkflowManager(new URL(met.getMetadata("WorkflowUrl"));
> wm.stopWorkflowInstance(met.getMetadata("WorkflowInstId"));
>
> That will basically have the same effect as the external stop. Exceptions work fine too,
> even if they don't look elegant :) Would be nice to have a WorkflowStoppedException, which extends
> some generic WorkflowTaskInstanceException, but for now, a good ol' Exception will do it itoo.
>
> HTH!
>
> Cheers,
> Chris
>
> [1] https://issues.apache.org/jira/browse/OODT-70
> [2] http://s.apache.org/NeK
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>