You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by Chris Mattmann <ma...@apache.org> on 2013/10/15 05:48:00 UTC
Re: Question regarding OODT patch 563 (Workflow API additions)
Hey Varun,
-----Original Message-----
From: Varun Ratnakar <va...@gmail.com>
Date: Monday, October 14, 2013 2:11 PM
To: jpluser <ch...@jpl.nasa.gov>
Subject: Re: Question regarding OODT patch 563 (Workflow API additions)
>Ah I see Chris. Thanks !
>
>So, I'm assuming that I add this functionality in pcs-services ?
PCS services would be a good place, or maybe creating like a workflow
JAX-RS services package.
PCS services typically are services that extend beyond multiple individual
CAS components.
> i.e. extend the API in there to interface with
>PackagedWorkflowRepository in order to be able to add/remove dynamic
>tasks.
See above, I would work on defining a new API e.g., and create a
webapp/wm-services/ App based on Apache CXF/JAX-RS similar to
the new FMprod stuff.
>
>
>I had some questions, if you don't mind:
>1. What do you mean by dynamic tasks in this context ?
Tasks that are added e.g., when a Dynamic workflow is created (see
OODT-215 and OODT-491)
>
>2. After the tasks are added, how will the workflow be run ?
The same way as if the workflow originally existed in the repo.
>3. Will the workflow manager will have to configured in a certain way to
>load tasks from this PackagedWorkflowRepository directory ?
Nope, the only issue is how to save dynamic workflows after restart, which
I'm working on.
Cheers,
Chris
>
>Regards,
>Varun
Cheers,
Chris
>
> On 14 Oct 2013, at 19:33, "Mattmann, Chris A (398J)"
><ch...@jpl.nasa.gov> wrote:
>
>> Hey Varun,
>>
>> The point about the PackagedWorkflowRepository is that it adds dynamic
>> tasks
>> to the workflow repo -- thoughts?
>>
>> Cheers,
>> Chris
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: Varun Ratnakar <va...@gmail.com>
>> Date: Monday, October 14, 2013 9:39 AM
>> To: jpluser <ch...@jpl.nasa.gov>
>> Subject: Re: Question regarding OODT patch 563 (Workflow API additions)
>>
>>> Hi Chris,
>>>
>>> just took a look at PackagedWorkflowRepository & the files in the
>>> tar-ball you sent a link to.
>>>
>>> I noticed that the PackagedWorkflowRepository loads workflow files and
>>> allows one to:
>>> - Add a task
>>> - Add a workflow
>>>
>>> I also see that the files you sent have code for
>>>PoicyAwareWorkflowTasks.
>>>
>>> I'm afraid I'm unable to connect the two. What did you have in mind ?
>>>
>>> Kind regards,
>>> Varun
>>>
>>>
>>>
>>> On 12 Oct 2013, at 20:23, "Mattmann, Chris A (398J)"
>>> <ch...@jpl.nasa.gov> wrote:
>>>
>>>> Hey Varun,
>>>>
>>>> See the PackagedWorkflowRepository (I do it there).
>>>>
>>>> Also see the work done here:
>>>>
>>>> https://issues.apache.org/jira/browse/OODT-213
>>>>
>>>>
>>>> I uploaded a series of the Java files that implement this from
>>>> OCO in 2007 (needs to be ALv2'ed, and cleaned up). It's a tar-ball
>>>> on that issue.
>>>>
>>>> Check those out and let's talk.
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> Chris Mattmann, Ph.D.
>>>> Senior Computer Scientist
>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>> Office: 171-266B, Mailstop: 171-246
>>>> Email: chris.a.mattmann@nasa.gov
>>>> WWW: http://sunset.usc.edu/~mattmann/
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> Adjunct Assistant Professor, Computer Science Department
>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Varun Ratnakar <va...@gmail.com>
>>>> Date: Friday, October 11, 2013 4:55 PM
>>>> To: Chris Mattmann <ch...@nasa.gov>
>>>> Subject: Question regarding OODT patch 563 (Workflow API additions)
>>>>
>>>>> Hi Chris,
>>>>>
>>>>> You had mentioned making the home for patch 563 in pcs-services. Did
>>>>> you
>>>>> have any pointers for me to look at regarding this ?
>>>>>
>>>>> What we basically need is:
>>>>>
>>>>> a) A way to add tasks with some configuration (inputs, outputs, etc)
>>>>> b) A way to invoke a workflow which includes the added tasks
>>>>>
>>>>> Note that the tasks in a) above doesn't need to be persistent. In
>>>>>fact,
>>>>> making it persistent might be counter-productive as creating a new
>>>>>task
>>>>> for each instantiation of a component might result in a large number
>>>>>of
>>>>> tasks being added in.
>>>>>
>>>>> Regards,
>>>>> Varun
>>>>>
>>>>
>>>> <winmail.dat>
>>>
>>
>> <winmail.dat>
>
Re: Question regarding OODT patch 563 (Workflow API additions)
Posted by Chris Mattmann <ma...@apache.org>.
Hey Varun,
Thanks, yep it is possible. Those parameters are the dynamic "metadata"
parameters
and should be used to override anything statically set from the workflow
config.
Cheers,
Chris
-----Original Message-----
From: Varun Ratnakar <va...@gmail.com>
Reply-To: <de...@oodt.apache.org>
Date: Tuesday, October 15, 2013 2:24 AM
To: Chris Mattmann <ch...@nasa.gov>
Cc: <de...@oodt.apache.org>
Subject: Re: Question regarding OODT patch 563 (Workflow API additions)
>Thanks Chris !
>
>I'll get to work on creating the webapp/wm-services JAX-RS module.
>
>Question:
>Is it possible to run a stored workflow and specify some configuration
>parameters of it's tasks at run-time (i.e. just before submitting the
>workflow for execution) ?
>
>Regards,
>Varun
>
>
>On 15 Oct 2013, at 04:48, Chris Mattmann <ma...@apache.org> wrote:
>
>> Hey Varun,
>>
>> -----Original Message-----
>>
>> From: Varun Ratnakar <va...@gmail.com>
>> Date: Monday, October 14, 2013 2:11 PM
>> To: jpluser <ch...@jpl.nasa.gov>
>> Subject: Re: Question regarding OODT patch 563 (Workflow API additions)
>>
>>> Ah I see Chris. Thanks !
>>>
>>> So, I'm assuming that I add this functionality in pcs-services ?
>>
>> PCS services would be a good place, or maybe creating like a workflow
>> JAX-RS services package.
>> PCS services typically are services that extend beyond multiple
>>individual
>> CAS components.
>>
>>> i.e. extend the API in there to interface with
>>> PackagedWorkflowRepository in order to be able to add/remove dynamic
>>> tasks.
>>
>> See above, I would work on defining a new API e.g., and create a
>> webapp/wm-services/ App based on Apache CXF/JAX-RS similar to
>> the new FMprod stuff.
>>
>>>
>>>
>>> I had some questions, if you don't mind:
>>> 1. What do you mean by dynamic tasks in this context ?
>>
>> Tasks that are added e.g., when a Dynamic workflow is created (see
>> OODT-215 and OODT-491)
>>
>>>
>>> 2. After the tasks are added, how will the workflow be run ?
>>
>> The same way as if the workflow originally existed in the repo.
>>
>>> 3. Will the workflow manager will have to configured in a certain way
>>>to
>>> load tasks from this PackagedWorkflowRepository directory ?
>>
>> Nope, the only issue is how to save dynamic workflows after restart,
>>which
>> I'm working on.
>>
>> Cheers,
>> Chris
>>
>>>
>>> Regards,
>>> Varun
>>
>> Cheers,
>> Chris
>>
>>>
>>> On 14 Oct 2013, at 19:33, "Mattmann, Chris A (398J)"
>>> <ch...@jpl.nasa.gov> wrote:
>>>
>>>> Hey Varun,
>>>>
>>>> The point about the PackagedWorkflowRepository is that it adds dynamic
>>>> tasks
>>>> to the workflow repo -- thoughts?
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> Chris Mattmann, Ph.D.
>>>> Senior Computer Scientist
>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>> Office: 171-266B, Mailstop: 171-246
>>>> Email: chris.a.mattmann@nasa.gov
>>>> WWW: http://sunset.usc.edu/~mattmann/
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> Adjunct Assistant Professor, Computer Science Department
>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Varun Ratnakar <va...@gmail.com>
>>>> Date: Monday, October 14, 2013 9:39 AM
>>>> To: jpluser <ch...@jpl.nasa.gov>
>>>> Subject: Re: Question regarding OODT patch 563 (Workflow API
>>>>additions)
>>>>
>>>>> Hi Chris,
>>>>>
>>>>> just took a look at PackagedWorkflowRepository & the files in the
>>>>> tar-ball you sent a link to.
>>>>>
>>>>> I noticed that the PackagedWorkflowRepository loads workflow files
>>>>>and
>>>>> allows one to:
>>>>> - Add a task
>>>>> - Add a workflow
>>>>>
>>>>> I also see that the files you sent have code for
>>>>> PoicyAwareWorkflowTasks.
>>>>>
>>>>> I'm afraid I'm unable to connect the two. What did you have in mind ?
>>>>>
>>>>> Kind regards,
>>>>> Varun
>>>>>
>>>>>
>>>>>
>>>>> On 12 Oct 2013, at 20:23, "Mattmann, Chris A (398J)"
>>>>> <ch...@jpl.nasa.gov> wrote:
>>>>>
>>>>>> Hey Varun,
>>>>>>
>>>>>> See the PackagedWorkflowRepository (I do it there).
>>>>>>
>>>>>> Also see the work done here:
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/OODT-213
>>>>>>
>>>>>>
>>>>>> I uploaded a series of the Java files that implement this from
>>>>>> OCO in 2007 (needs to be ALv2'ed, and cleaned up). It's a tar-ball
>>>>>> on that issue.
>>>>>>
>>>>>> Check those out and let's talk.
>>>>>>
>>>>>> Cheers,
>>>>>> Chris
>>>>>>
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> Chris Mattmann, Ph.D.
>>>>>> Senior Computer Scientist
>>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>>> Office: 171-266B, Mailstop: 171-246
>>>>>> Email: chris.a.mattmann@nasa.gov
>>>>>> WWW: http://sunset.usc.edu/~mattmann/
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> Adjunct Assistant Professor, Computer Science Department
>>>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Varun Ratnakar <va...@gmail.com>
>>>>>> Date: Friday, October 11, 2013 4:55 PM
>>>>>> To: Chris Mattmann <ch...@nasa.gov>
>>>>>> Subject: Question regarding OODT patch 563 (Workflow API additions)
>>>>>>
>>>>>>> Hi Chris,
>>>>>>>
>>>>>>> You had mentioned making the home for patch 563 in pcs-services.
>>>>>>>Did
>>>>>>> you
>>>>>>> have any pointers for me to look at regarding this ?
>>>>>>>
>>>>>>> What we basically need is:
>>>>>>>
>>>>>>> a) A way to add tasks with some configuration (inputs, outputs,
>>>>>>>etc)
>>>>>>> b) A way to invoke a workflow which includes the added tasks
>>>>>>>
>>>>>>> Note that the tasks in a) above doesn't need to be persistent. In
>>>>>>> fact,
>>>>>>> making it persistent might be counter-productive as creating a new
>>>>>>> task
>>>>>>> for each instantiation of a component might result in a large
>>>>>>>number
>>>>>>> of
>>>>>>> tasks being added in.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Varun
>>>>>>>
>>>>>>
>>>>>> <winmail.dat>
>>>>>
>>>>
>>>> <winmail.dat>
>>>
>>
>>
>
Re: Question regarding OODT patch 563 (Workflow API additions)
Posted by Varun Ratnakar <va...@gmail.com>.
Thanks Chris !
I'll get to work on creating the webapp/wm-services JAX-RS module.
Question:
Is it possible to run a stored workflow and specify some configuration parameters of it's tasks at run-time (i.e. just before submitting the workflow for execution) ?
Regards,
Varun
On 15 Oct 2013, at 04:48, Chris Mattmann <ma...@apache.org> wrote:
> Hey Varun,
>
> -----Original Message-----
>
> From: Varun Ratnakar <va...@gmail.com>
> Date: Monday, October 14, 2013 2:11 PM
> To: jpluser <ch...@jpl.nasa.gov>
> Subject: Re: Question regarding OODT patch 563 (Workflow API additions)
>
>> Ah I see Chris. Thanks !
>>
>> So, I'm assuming that I add this functionality in pcs-services ?
>
> PCS services would be a good place, or maybe creating like a workflow
> JAX-RS services package.
> PCS services typically are services that extend beyond multiple individual
> CAS components.
>
>> i.e. extend the API in there to interface with
>> PackagedWorkflowRepository in order to be able to add/remove dynamic
>> tasks.
>
> See above, I would work on defining a new API e.g., and create a
> webapp/wm-services/ App based on Apache CXF/JAX-RS similar to
> the new FMprod stuff.
>
>>
>>
>> I had some questions, if you don't mind:
>> 1. What do you mean by dynamic tasks in this context ?
>
> Tasks that are added e.g., when a Dynamic workflow is created (see
> OODT-215 and OODT-491)
>
>>
>> 2. After the tasks are added, how will the workflow be run ?
>
> The same way as if the workflow originally existed in the repo.
>
>> 3. Will the workflow manager will have to configured in a certain way to
>> load tasks from this PackagedWorkflowRepository directory ?
>
> Nope, the only issue is how to save dynamic workflows after restart, which
> I'm working on.
>
> Cheers,
> Chris
>
>>
>> Regards,
>> Varun
>
> Cheers,
> Chris
>
>>
>> On 14 Oct 2013, at 19:33, "Mattmann, Chris A (398J)"
>> <ch...@jpl.nasa.gov> wrote:
>>
>>> Hey Varun,
>>>
>>> The point about the PackagedWorkflowRepository is that it adds dynamic
>>> tasks
>>> to the workflow repo -- thoughts?
>>>
>>> Cheers,
>>> Chris
>>>
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattmann@nasa.gov
>>> WWW: http://sunset.usc.edu/~mattmann/
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>>
>>>
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Varun Ratnakar <va...@gmail.com>
>>> Date: Monday, October 14, 2013 9:39 AM
>>> To: jpluser <ch...@jpl.nasa.gov>
>>> Subject: Re: Question regarding OODT patch 563 (Workflow API additions)
>>>
>>>> Hi Chris,
>>>>
>>>> just took a look at PackagedWorkflowRepository & the files in the
>>>> tar-ball you sent a link to.
>>>>
>>>> I noticed that the PackagedWorkflowRepository loads workflow files and
>>>> allows one to:
>>>> - Add a task
>>>> - Add a workflow
>>>>
>>>> I also see that the files you sent have code for
>>>> PoicyAwareWorkflowTasks.
>>>>
>>>> I'm afraid I'm unable to connect the two. What did you have in mind ?
>>>>
>>>> Kind regards,
>>>> Varun
>>>>
>>>>
>>>>
>>>> On 12 Oct 2013, at 20:23, "Mattmann, Chris A (398J)"
>>>> <ch...@jpl.nasa.gov> wrote:
>>>>
>>>>> Hey Varun,
>>>>>
>>>>> See the PackagedWorkflowRepository (I do it there).
>>>>>
>>>>> Also see the work done here:
>>>>>
>>>>> https://issues.apache.org/jira/browse/OODT-213
>>>>>
>>>>>
>>>>> I uploaded a series of the Java files that implement this from
>>>>> OCO in 2007 (needs to be ALv2'ed, and cleaned up). It's a tar-ball
>>>>> on that issue.
>>>>>
>>>>> Check those out and let's talk.
>>>>>
>>>>> Cheers,
>>>>> Chris
>>>>>
>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>> Chris Mattmann, Ph.D.
>>>>> Senior Computer Scientist
>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>> Office: 171-266B, Mailstop: 171-246
>>>>> Email: chris.a.mattmann@nasa.gov
>>>>> WWW: http://sunset.usc.edu/~mattmann/
>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>> Adjunct Assistant Professor, Computer Science Department
>>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Varun Ratnakar <va...@gmail.com>
>>>>> Date: Friday, October 11, 2013 4:55 PM
>>>>> To: Chris Mattmann <ch...@nasa.gov>
>>>>> Subject: Question regarding OODT patch 563 (Workflow API additions)
>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>> You had mentioned making the home for patch 563 in pcs-services. Did
>>>>>> you
>>>>>> have any pointers for me to look at regarding this ?
>>>>>>
>>>>>> What we basically need is:
>>>>>>
>>>>>> a) A way to add tasks with some configuration (inputs, outputs, etc)
>>>>>> b) A way to invoke a workflow which includes the added tasks
>>>>>>
>>>>>> Note that the tasks in a) above doesn't need to be persistent. In
>>>>>> fact,
>>>>>> making it persistent might be counter-productive as creating a new
>>>>>> task
>>>>>> for each instantiation of a component might result in a large number
>>>>>> of
>>>>>> tasks being added in.
>>>>>>
>>>>>> Regards,
>>>>>> Varun
>>>>>>
>>>>>
>>>>> <winmail.dat>
>>>>
>>>
>>> <winmail.dat>
>>
>
>