You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> on 2011/07/05 00:16:29 UTC
[NOTICE] trunk/workflow now supports wengine-style workflow XML
files
Hi Folks,
With the recent work on OODT-70 [1] to add support for wengine-style workflow XML files
to trunk/workflow, the trunk/workflow manager can now load up and execute
wengine-style workflows.
To enable this feature (which only exists in 0.4-SNAPSHOT), download and build
a fresh version of trunk, and then grab the workflow manager deployment from there
and edit $WORKFLOW_HOME/etc/workflow.properties:
- Edit # workflow.repo.factory and set it to org.apache.oodt.cas.workflow.repository.PackagedWorkflowRepositoryFactory
- Set the following property:
org.apache.oodt.cas.workflow.wengine.packagedRepo.dir.path to the root dir path to a directory with wengine-style
workflow files. You can comma-separate the paths and it will load all wengine-style XML files from each dir path
specified (and comma-separated)
Reboot your workflow manager ($WORKFLOW_HOME/bin ; ./wmgr start) and your workflows
should now be loaded. A couple of notes (from [1]):
1. Sub-workflows are supported by an inline BranchEventTask that sends an Event to kick off sub-workflow. The
dynamic Events are built and added to event repo.
2. Parallel workflows are supported right now with event mapped to single-task workflows.
If you're interested in the progress of the Workflow2 integration with trunk, see OODT-215 [2]. Comments, help,
thoughts, patches, welcome :-)
Cheers,
Chris
[1] https://issues.apache.org/jira/browse/OODT-70
[2] https://issues.apache.org/jira/browse/OODT-215
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: [NOTICE] trunk/workflow now supports wengine-style workflow XML
files
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Cecilia,
The biggest difference between the 2 is that the wengine-branch filemgr includes
a Catalog interface for cas-catalog, as well as a small functionality update
to allow it to stage files (for cas-pge).
That's the reason that there's a specialized cas-pge in the branch too. That, and
the specialized cas-pge includes specialized met keys that you must define in your
conf that are delimited by "/" instead of "_". So instead of e.g., PCS_FileManagerUrl
as the property for the cas-pge file manager URL, it uses "/PgeTask/Query/FileManagerUrl".
You can see the diff between the 2 PGE keys by examining these two files:
http://svn.apache.org/repos/asf/oodt/branches/wengine-branch/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PGETaskMetKeys.java
http://svn.apache.org/repos/asf/oodt/trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PcsMetadataKeys.java
One other difference is that Brian wrote a customized CAS-PGE workflow condition as well and wired it
into the CAS-PGE framework. The other major thing is PCS file staging.
I have a diff sitting on my desktop of new CAS-PGE stuff and plan on making some updates to
remove the need for a branched cas-pge as well. It's my goal with 0.4 to remove the need
for a branch, period. Folks can continue to use it, but with the knowledge that it's not
as actively maintained as trunk.
See OODT-215 [1] for the whole plan.
HTH!
Cheers,
Chris
[1] https://issues.apache.org/jia/browse/OODT-215
On Jul 5, 2011, at 3:01 PM, Cheng, Cecilia S (388K) wrote:
> Hi Chris,
>
> Just a quick question ... The wengine branch also has its own filemgr & pge
> componetns. I assume that the wengine-style workflow in release 0.4 uses the
> trunk filemgr. Do you know what's the difference between the two filemgr
> components? I only know that in order to use the wengine branch workflow, I
> must use the wengine branch filemgr.
>
> Thanks,
> Cecilia
>
>
> On 7/4/11 3:16 PM, "Mattmann, Chris A (388J)"
> <ch...@jpl.nasa.gov> wrote:
>
>> Hi Folks,
>>
>> With the recent work on OODT-70 [1] to add support for wengine-style workflow
>> XML files
>> to trunk/workflow, the trunk/workflow manager can now load up and execute
>> wengine-style workflows.
>>
>> To enable this feature (which only exists in 0.4-SNAPSHOT), download and build
>> a fresh version of trunk, and then grab the workflow manager deployment from
>> there
>> and edit $WORKFLOW_HOME/etc/workflow.properties:
>> - Edit # workflow.repo.factory and set it to
>> org.apache.oodt.cas.workflow.repository.PackagedWorkflowRepositoryFactory
>> - Set the following property:
>> org.apache.oodt.cas.workflow.wengine.packagedRepo.dir.path to the root dir
>> path to a directory with wengine-style
>> workflow files. You can comma-separate the paths and it will load all
>> wengine-style XML files from each dir path
>> specified (and comma-separated)
>>
>> Reboot your workflow manager ($WORKFLOW_HOME/bin ; ./wmgr start) and your
>> workflows
>> should now be loaded. A couple of notes (from [1]):
>>
>> 1. Sub-workflows are supported by an inline BranchEventTask that sends an
>> Event to kick off sub-workflow. The
>> dynamic Events are built and added to event repo.
>> 2. Parallel workflows are supported right now with event mapped to single-task
>> workflows.
>>
>> If you're interested in the progress of the Workflow2 integration with trunk,
>> see OODT-215 [2]. Comments, help,
>> thoughts, patches, welcome :-)
>>
>> Cheers,
>> Chris
>>
>> [1] https://issues.apache.org/jira/browse/OODT-70
>> [2] https://issues.apache.org/jira/browse/OODT-215
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: [NOTICE] trunk/workflow now supports wengine-style workflow XML
files
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Cecilia,
The biggest difference between the 2 is that the wengine-branch filemgr includes
a Catalog interface for cas-catalog, as well as a small functionality update
to allow it to stage files (for cas-pge).
That's the reason that there's a specialized cas-pge in the branch too. That, and
the specialized cas-pge includes specialized met keys that you must define in your
conf that are delimited by "/" instead of "_". So instead of e.g., PCS_FileManagerUrl
as the property for the cas-pge file manager URL, it uses "/PgeTask/Query/FileManagerUrl".
You can see the diff between the 2 PGE keys by examining these two files:
http://svn.apache.org/repos/asf/oodt/branches/wengine-branch/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PGETaskMetKeys.java
http://svn.apache.org/repos/asf/oodt/trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PcsMetadataKeys.java
One other difference is that Brian wrote a customized CAS-PGE workflow condition as well and wired it
into the CAS-PGE framework. The other major thing is PCS file staging.
I have a diff sitting on my desktop of new CAS-PGE stuff and plan on making some updates to
remove the need for a branched cas-pge as well. It's my goal with 0.4 to remove the need
for a branch, period. Folks can continue to use it, but with the knowledge that it's not
as actively maintained as trunk.
See OODT-215 [1] for the whole plan.
HTH!
Cheers,
Chris
[1] https://issues.apache.org/jia/browse/OODT-215
On Jul 5, 2011, at 3:01 PM, Cheng, Cecilia S (388K) wrote:
> Hi Chris,
>
> Just a quick question ... The wengine branch also has its own filemgr & pge
> componetns. I assume that the wengine-style workflow in release 0.4 uses the
> trunk filemgr. Do you know what's the difference between the two filemgr
> components? I only know that in order to use the wengine branch workflow, I
> must use the wengine branch filemgr.
>
> Thanks,
> Cecilia
>
>
> On 7/4/11 3:16 PM, "Mattmann, Chris A (388J)"
> <ch...@jpl.nasa.gov> wrote:
>
>> Hi Folks,
>>
>> With the recent work on OODT-70 [1] to add support for wengine-style workflow
>> XML files
>> to trunk/workflow, the trunk/workflow manager can now load up and execute
>> wengine-style workflows.
>>
>> To enable this feature (which only exists in 0.4-SNAPSHOT), download and build
>> a fresh version of trunk, and then grab the workflow manager deployment from
>> there
>> and edit $WORKFLOW_HOME/etc/workflow.properties:
>> - Edit # workflow.repo.factory and set it to
>> org.apache.oodt.cas.workflow.repository.PackagedWorkflowRepositoryFactory
>> - Set the following property:
>> org.apache.oodt.cas.workflow.wengine.packagedRepo.dir.path to the root dir
>> path to a directory with wengine-style
>> workflow files. You can comma-separate the paths and it will load all
>> wengine-style XML files from each dir path
>> specified (and comma-separated)
>>
>> Reboot your workflow manager ($WORKFLOW_HOME/bin ; ./wmgr start) and your
>> workflows
>> should now be loaded. A couple of notes (from [1]):
>>
>> 1. Sub-workflows are supported by an inline BranchEventTask that sends an
>> Event to kick off sub-workflow. The
>> dynamic Events are built and added to event repo.
>> 2. Parallel workflows are supported right now with event mapped to single-task
>> workflows.
>>
>> If you're interested in the progress of the Workflow2 integration with trunk,
>> see OODT-215 [2]. Comments, help,
>> thoughts, patches, welcome :-)
>>
>> Cheers,
>> Chris
>>
>> [1] https://issues.apache.org/jira/browse/OODT-70
>> [2] https://issues.apache.org/jira/browse/OODT-215
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: [NOTICE] trunk/workflow now supports wengine-style workflow XML
files
Posted by "Cheng, Cecilia S (388K)" <ce...@jpl.nasa.gov>.
Hi Chris,
Just a quick question ... The wengine branch also has its own filemgr & pge
componetns. I assume that the wengine-style workflow in release 0.4 uses the
trunk filemgr. Do you know what's the difference between the two filemgr
components? I only know that in order to use the wengine branch workflow, I
must use the wengine branch filemgr.
Thanks,
Cecilia
On 7/4/11 3:16 PM, "Mattmann, Chris A (388J)"
<ch...@jpl.nasa.gov> wrote:
> Hi Folks,
>
> With the recent work on OODT-70 [1] to add support for wengine-style workflow
> XML files
> to trunk/workflow, the trunk/workflow manager can now load up and execute
> wengine-style workflows.
>
> To enable this feature (which only exists in 0.4-SNAPSHOT), download and build
> a fresh version of trunk, and then grab the workflow manager deployment from
> there
> and edit $WORKFLOW_HOME/etc/workflow.properties:
> - Edit # workflow.repo.factory and set it to
> org.apache.oodt.cas.workflow.repository.PackagedWorkflowRepositoryFactory
> - Set the following property:
> org.apache.oodt.cas.workflow.wengine.packagedRepo.dir.path to the root dir
> path to a directory with wengine-style
> workflow files. You can comma-separate the paths and it will load all
> wengine-style XML files from each dir path
> specified (and comma-separated)
>
> Reboot your workflow manager ($WORKFLOW_HOME/bin ; ./wmgr start) and your
> workflows
> should now be loaded. A couple of notes (from [1]):
>
> 1. Sub-workflows are supported by an inline BranchEventTask that sends an
> Event to kick off sub-workflow. The
> dynamic Events are built and added to event repo.
> 2. Parallel workflows are supported right now with event mapped to single-task
> workflows.
>
> If you're interested in the progress of the Workflow2 integration with trunk,
> see OODT-215 [2]. Comments, help,
> thoughts, patches, welcome :-)
>
> Cheers,
> Chris
>
> [1] https://issues.apache.org/jira/browse/OODT-70
> [2] https://issues.apache.org/jira/browse/OODT-215
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
Re: [NOTICE] trunk/workflow now supports wengine-style workflow XML
files
Posted by "Cheng, Cecilia S (388K)" <ce...@jpl.nasa.gov>.
Hi Chris,
Just a quick question ... The wengine branch also has its own filemgr & pge
componetns. I assume that the wengine-style workflow in release 0.4 uses the
trunk filemgr. Do you know what's the difference between the two filemgr
components? I only know that in order to use the wengine branch workflow, I
must use the wengine branch filemgr.
Thanks,
Cecilia
On 7/4/11 3:16 PM, "Mattmann, Chris A (388J)"
<ch...@jpl.nasa.gov> wrote:
> Hi Folks,
>
> With the recent work on OODT-70 [1] to add support for wengine-style workflow
> XML files
> to trunk/workflow, the trunk/workflow manager can now load up and execute
> wengine-style workflows.
>
> To enable this feature (which only exists in 0.4-SNAPSHOT), download and build
> a fresh version of trunk, and then grab the workflow manager deployment from
> there
> and edit $WORKFLOW_HOME/etc/workflow.properties:
> - Edit # workflow.repo.factory and set it to
> org.apache.oodt.cas.workflow.repository.PackagedWorkflowRepositoryFactory
> - Set the following property:
> org.apache.oodt.cas.workflow.wengine.packagedRepo.dir.path to the root dir
> path to a directory with wengine-style
> workflow files. You can comma-separate the paths and it will load all
> wengine-style XML files from each dir path
> specified (and comma-separated)
>
> Reboot your workflow manager ($WORKFLOW_HOME/bin ; ./wmgr start) and your
> workflows
> should now be loaded. A couple of notes (from [1]):
>
> 1. Sub-workflows are supported by an inline BranchEventTask that sends an
> Event to kick off sub-workflow. The
> dynamic Events are built and added to event repo.
> 2. Parallel workflows are supported right now with event mapped to single-task
> workflows.
>
> If you're interested in the progress of the Workflow2 integration with trunk,
> see OODT-215 [2]. Comments, help,
> thoughts, patches, welcome :-)
>
> Cheers,
> Chris
>
> [1] https://issues.apache.org/jira/browse/OODT-70
> [2] https://issues.apache.org/jira/browse/OODT-215
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>