You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by "Verma, Rishi (317I)" <Ri...@jpl.nasa.gov> on 2011/09/27 22:32:13 UTC
When to use CAS-PGE
Hi All,
This is something I have been wondering myself, and perhaps other CAS-Workflow users have been wondering as well so I thought I'd put this question out there.
The question is: in which circumstances does it make sense to use PGEs for representing workflow tasks? In other words, which types of situations favor wrapping a workflow task as a PGETaskInstance versus using the generic WorkflowTaskInstance?
Based on my own experience with CAS-PGE, I feel it makes sense to go for using PGEs for the following types of situations:
[1] Extensive use of external programs is required for the execution of a workflow task
[2] Gathering products from, or ingesting products to the CAS-FileManager is needed for the execution of a workflow task
If there are other types of circumstances using PGEs is ideal for or if there are other opinions out there, it would be great to hear them!
Thanks,
Rishi
Re: When to use CAS-PGE
Posted by Sheryl John <sh...@gmail.com>.
Hi Rishi,
I need a PGE for a following situation:
[3] When a task requires workflow context metadata and/or it generates
metadata that it adds to the workflow context metadata to be used later down
in the pipeline.
I'm not sure how to add PGE generated metadata to the Workflow context yet.
Thanks,
On Tue, Sep 27, 2011 at 1:32 PM, Verma, Rishi (317I) <
Rishi.Verma@jpl.nasa.gov> wrote:
> Hi All,
>
> This is something I have been wondering myself, and perhaps other
> CAS-Workflow users have been wondering as well so I thought I'd put this
> question out there.
>
> The question is: in which circumstances does it make sense to use PGEs for
> representing workflow tasks? In other words, which types of situations favor
> wrapping a workflow task as a PGETaskInstance versus using the generic
> WorkflowTaskInstance?
>
> Based on my own experience with CAS-PGE, I feel it makes sense to go for
> using PGEs for the following types of situations:
> [1] Extensive use of external programs is required for the execution of a
> workflow task
> [2] Gathering products from, or ingesting products to the CAS-FileManager
> is needed for the execution of a workflow task
>
> If there are other types of circumstances using PGEs is ideal for or if
> there are other opinions out there, it would be great to hear them!
>
> Thanks,
> Rishi
--
-Sheryl
Re: When to use CAS-PGE
Posted by Sheryl John <sh...@gmail.com>.
Hi Rishi,
I need a PGE for a following situation:
[3] When a task requires workflow context metadata and/or it generates
metadata that it adds to the workflow context metadata to be used later down
in the pipeline.
I'm not sure how to add PGE generated metadata to the Workflow context yet.
Thanks,
On Tue, Sep 27, 2011 at 1:32 PM, Verma, Rishi (317I) <
Rishi.Verma@jpl.nasa.gov> wrote:
> Hi All,
>
> This is something I have been wondering myself, and perhaps other
> CAS-Workflow users have been wondering as well so I thought I'd put this
> question out there.
>
> The question is: in which circumstances does it make sense to use PGEs for
> representing workflow tasks? In other words, which types of situations favor
> wrapping a workflow task as a PGETaskInstance versus using the generic
> WorkflowTaskInstance?
>
> Based on my own experience with CAS-PGE, I feel it makes sense to go for
> using PGEs for the following types of situations:
> [1] Extensive use of external programs is required for the execution of a
> workflow task
> [2] Gathering products from, or ingesting products to the CAS-FileManager
> is needed for the execution of a workflow task
>
> If there are other types of circumstances using PGEs is ideal for or if
> there are other opinions out there, it would be great to hear them!
>
> Thanks,
> Rishi
--
-Sheryl
Re: When to use CAS-PGE
Posted by "Verma, Rishi (317I)" <Ri...@jpl.nasa.gov>.
Hi Sheryl, Chris,
Thank you both for providing detailed suggestions/use-cases of when to use PGEs. This certainly has helped me, and hopefully will help others as well.
Thanks!
Rishi
On Sep 27, 2011, at 7:51 PM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:
> Hi Rishi,
>
> Thanks for your email. My thoughts below:
>
> On Sep 27, 2011, at 1:32 PM, Verma, Rishi (317I) wrote:
>
>> Hi All,
>>
>> This is something I have been wondering myself, and perhaps other CAS-Workflow users have been wondering as well so I thought I'd put this question out there.
>>
>> The question is: in which circumstances does it make sense to use PGEs for representing workflow tasks? In other words, which types of situations favor wrapping a workflow task as a PGETaskInstance versus using the generic WorkflowTaskInstance?
>>
>> Based on my own experience with CAS-PGE, I feel it makes sense to go for using PGEs for the following types of situations:
>> [1] Extensive use of external programs is required for the execution of a workflow task
>> [2] Gathering products from, or ingesting products to the CAS-FileManager is needed for the execution of a workflow task
>
> and to add Sheryl's (also a good one):
>
> [3] When a task requires workflow context metadata and/or it generates metadata that it adds to the workflow context metadata to be used later down in the pipeline.
>
> And here are some of mine:
>
> [4] When you need to generate an input configuration file (or set of configuration files) that serve as input to the PGE and those configuration files are based in some form on CAS metadata, environment variables, command line parameters, etc.
> [5] When you want to extract metadata for the output product files for a PGE.
>
> Basically CAS-PGE implements a specialized, optimized workflow of:
>
> [4], then [2] (gathering part, combined with the ability to get information from CAS-Workflow and CAS-Resource as envisioned), then [3], then [1], then [5], then [2] (ingestion part).
>
> Those are the types of tasks that CAS-PGE is good for.
>
> Other than that, a WorkflowTaskInstance might make more sense. The nice thing about CAS-PGE is that the goal is to not have to write code, but to write XML to be able to integrate underlying science algorithms.
>
> HTH!
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
Re: When to use CAS-PGE
Posted by "Verma, Rishi (317I)" <Ri...@jpl.nasa.gov>.
Hi Sheryl, Chris,
Thank you both for providing detailed suggestions/use-cases of when to use PGEs. This certainly has helped me, and hopefully will help others as well.
Thanks!
Rishi
On Sep 27, 2011, at 7:51 PM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:
> Hi Rishi,
>
> Thanks for your email. My thoughts below:
>
> On Sep 27, 2011, at 1:32 PM, Verma, Rishi (317I) wrote:
>
>> Hi All,
>>
>> This is something I have been wondering myself, and perhaps other CAS-Workflow users have been wondering as well so I thought I'd put this question out there.
>>
>> The question is: in which circumstances does it make sense to use PGEs for representing workflow tasks? In other words, which types of situations favor wrapping a workflow task as a PGETaskInstance versus using the generic WorkflowTaskInstance?
>>
>> Based on my own experience with CAS-PGE, I feel it makes sense to go for using PGEs for the following types of situations:
>> [1] Extensive use of external programs is required for the execution of a workflow task
>> [2] Gathering products from, or ingesting products to the CAS-FileManager is needed for the execution of a workflow task
>
> and to add Sheryl's (also a good one):
>
> [3] When a task requires workflow context metadata and/or it generates metadata that it adds to the workflow context metadata to be used later down in the pipeline.
>
> And here are some of mine:
>
> [4] When you need to generate an input configuration file (or set of configuration files) that serve as input to the PGE and those configuration files are based in some form on CAS metadata, environment variables, command line parameters, etc.
> [5] When you want to extract metadata for the output product files for a PGE.
>
> Basically CAS-PGE implements a specialized, optimized workflow of:
>
> [4], then [2] (gathering part, combined with the ability to get information from CAS-Workflow and CAS-Resource as envisioned), then [3], then [1], then [5], then [2] (ingestion part).
>
> Those are the types of tasks that CAS-PGE is good for.
>
> Other than that, a WorkflowTaskInstance might make more sense. The nice thing about CAS-PGE is that the goal is to not have to write code, but to write XML to be able to integrate underlying science algorithms.
>
> HTH!
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
Re: When to use CAS-PGE
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Rishi,
Thanks for your email. My thoughts below:
On Sep 27, 2011, at 1:32 PM, Verma, Rishi (317I) wrote:
> Hi All,
>
> This is something I have been wondering myself, and perhaps other CAS-Workflow users have been wondering as well so I thought I'd put this question out there.
>
> The question is: in which circumstances does it make sense to use PGEs for representing workflow tasks? In other words, which types of situations favor wrapping a workflow task as a PGETaskInstance versus using the generic WorkflowTaskInstance?
>
> Based on my own experience with CAS-PGE, I feel it makes sense to go for using PGEs for the following types of situations:
> [1] Extensive use of external programs is required for the execution of a workflow task
> [2] Gathering products from, or ingesting products to the CAS-FileManager is needed for the execution of a workflow task
and to add Sheryl's (also a good one):
[3] When a task requires workflow context metadata and/or it generates metadata that it adds to the workflow context metadata to be used later down in the pipeline.
And here are some of mine:
[4] When you need to generate an input configuration file (or set of configuration files) that serve as input to the PGE and those configuration files are based in some form on CAS metadata, environment variables, command line parameters, etc.
[5] When you want to extract metadata for the output product files for a PGE.
Basically CAS-PGE implements a specialized, optimized workflow of:
[4], then [2] (gathering part, combined with the ability to get information from CAS-Workflow and CAS-Resource as envisioned), then [3], then [1], then [5], then [2] (ingestion part).
Those are the types of tasks that CAS-PGE is good for.
Other than that, a WorkflowTaskInstance might make more sense. The nice thing about CAS-PGE is that the goal is to not have to write code, but to write XML to be able to integrate underlying science algorithms.
HTH!
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: When to use CAS-PGE
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Rishi,
Thanks for your email. My thoughts below:
On Sep 27, 2011, at 1:32 PM, Verma, Rishi (317I) wrote:
> Hi All,
>
> This is something I have been wondering myself, and perhaps other CAS-Workflow users have been wondering as well so I thought I'd put this question out there.
>
> The question is: in which circumstances does it make sense to use PGEs for representing workflow tasks? In other words, which types of situations favor wrapping a workflow task as a PGETaskInstance versus using the generic WorkflowTaskInstance?
>
> Based on my own experience with CAS-PGE, I feel it makes sense to go for using PGEs for the following types of situations:
> [1] Extensive use of external programs is required for the execution of a workflow task
> [2] Gathering products from, or ingesting products to the CAS-FileManager is needed for the execution of a workflow task
and to add Sheryl's (also a good one):
[3] When a task requires workflow context metadata and/or it generates metadata that it adds to the workflow context metadata to be used later down in the pipeline.
And here are some of mine:
[4] When you need to generate an input configuration file (or set of configuration files) that serve as input to the PGE and those configuration files are based in some form on CAS metadata, environment variables, command line parameters, etc.
[5] When you want to extract metadata for the output product files for a PGE.
Basically CAS-PGE implements a specialized, optimized workflow of:
[4], then [2] (gathering part, combined with the ability to get information from CAS-Workflow and CAS-Resource as envisioned), then [3], then [1], then [5], then [2] (ingestion part).
Those are the types of tasks that CAS-PGE is good for.
Other than that, a WorkflowTaskInstance might make more sense. The nice thing about CAS-PGE is that the goal is to not have to write code, but to write XML to be able to integrate underlying science algorithms.
HTH!
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++