You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by Chris Mattmann <ma...@apache.org> on 2013/06/05 04:35:58 UTC
Re: Review Request: Apache Airvata extension to integrate Apache OODT File
manager using CAS-PGE
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11565/
-----------------------------------------------------------
(Updated June 5, 2013, 2:35 a.m.)
Review request for oodt, Chris Mattmann, Suresh Srinivas, and Shahani Weerawarana.
Description
-------
I have worked on the Apache Airavata integration with Apache OODT. As a first step, I have implemented integration with Apache OODT file manager component.
1. Introduce a new GFac Schema type called OODTProduct which takes APache OODT product IDs as input.
2. Implemented new pre GFac Handler by extending Apache OODT PgeTaskInstance to stage the corresponding file into the working directory.
3. Once file is staged, input parameter with OODT product id is replaced with path of the staged file for downstream processing
I have tested the implementation with Gfac application which wraps /bin/ls command. Application takes product id as input and stage corresponding file into the working directory and /bin/ls is executed against the staged file. Hope this is a valid testing scenario.
Concerns
- Configurations : I have added new configuration file named and oodt-integration.properties in addition to dynamic_metadata.met and pge-config.xml files used by OODT. But at the moment there is no item configured with the oodt-integration.properties.
- Staged File Name - With the current implementation of PgeTaskInstance it is not possible to retrieve path of the staged file. Due to this limitation, I have query the FileManagerServer with product id and retrieve the file name and computed the file path using information of working directory.
- Currently it is not possible to execute the workflow using Xbaya due to validation failure due to new schema type. I have commented out the relevant validation code for testing purpose.
Diffs
-----
https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/java/org/apache/airavata/commons/gfac/type/MappingFactory.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/resources/schemas/GFacParameterTypes.xsd 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/pom.xml 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/assembly/bin-assembly.xml 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/dynamic_metadata.met PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/gfac-config.xml 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/oodt_integration.properties PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/pge-config.xml PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/gfac-core/src/main/java/org/apache/airavata/gfac/utils/GFacUtils.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/pom.xml PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataToApacheOODTConfigurationMapper.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataWorkFlowInstance.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataWorkFlowInstanceImpl.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/OODTProduct.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/OODTProductStager.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/component/ws/WSComponent.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/gpel/script/WorkflowWSDL.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/xsd/GFacSimpleTypesXSD.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/xbaya-gui/src/main/java/org/apache/airavata/xbaya/lead/LEADTypes.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/pom.xml 1487802
Diff: https://reviews.apache.org/r/11565/diff/
Testing
-------
Thanks,
Sanjaya Medonsa
Re: Review Request: Apache Airvata extension to integrate Apache OODT File
manager using CAS-PGE
Posted by Sanjaya Medonsa <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11565/
-----------------------------------------------------------
(Updated June 12, 2013, 1:12 a.m.)
Review request for Airavata, oodt, Chris Mattmann, Suresh Srinivas, and Shahani Weerawarana.
Changes
-------
Added Airavata into review group
Description
-------
I have worked on the Apache Airavata integration with Apache OODT. As a first step, I have implemented integration with Apache OODT file manager component.
1. Introduce a new GFac Schema type called OODTProduct which takes APache OODT product IDs as input.
2. Implemented new pre GFac Handler by extending Apache OODT PgeTaskInstance to stage the corresponding file into the working directory.
3. Once file is staged, input parameter with OODT product id is replaced with path of the staged file for downstream processing
I have tested the implementation with Gfac application which wraps /bin/ls command. Application takes product id as input and stage corresponding file into the working directory and /bin/ls is executed against the staged file. Hope this is a valid testing scenario.
Concerns
- Configurations : I have added new configuration file named and oodt-integration.properties in addition to dynamic_metadata.met and pge-config.xml files used by OODT. But at the moment there is no item configured with the oodt-integration.properties.
- Staged File Name - With the current implementation of PgeTaskInstance it is not possible to retrieve path of the staged file. Due to this limitation, I have query the FileManagerServer with product id and retrieve the file name and computed the file path using information of working directory.
- Currently it is not possible to execute the workflow using Xbaya due to validation failure due to new schema type. I have commented out the relevant validation code for testing purpose.
Diffs
-----
https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/java/org/apache/airavata/commons/gfac/type/MappingFactory.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/resources/schemas/GFacParameterTypes.xsd 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/pom.xml 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/assembly/bin-assembly.xml 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/dynamic_metadata.met PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/gfac-config.xml 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/oodt_integration.properties PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/pge-config.xml PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/gfac-core/src/main/java/org/apache/airavata/gfac/utils/GFacUtils.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/pom.xml PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataToApacheOODTConfigurationMapper.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataWorkFlowInstance.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataWorkFlowInstanceImpl.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/OODTProduct.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/OODTProductStager.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/component/ws/WSComponent.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/gpel/script/WorkflowWSDL.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/xsd/GFacSimpleTypesXSD.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/xbaya-gui/src/main/java/org/apache/airavata/xbaya/lead/LEADTypes.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/pom.xml 1487802
Diff: https://reviews.apache.org/r/11565/diff/
Testing
-------
Thanks,
Sanjaya Medonsa
Re: Review Request: Apache Airvata extension to integrate Apache OODT File
manager using CAS-PGE
Posted by Sanjaya Medonsa <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11565/
-----------------------------------------------------------
(Updated June 12, 2013, 1:12 a.m.)
Review request for Airavata, oodt, Chris Mattmann, Suresh Srinivas, and Shahani Weerawarana.
Changes
-------
Added Airavata into review group
Description
-------
I have worked on the Apache Airavata integration with Apache OODT. As a first step, I have implemented integration with Apache OODT file manager component.
1. Introduce a new GFac Schema type called OODTProduct which takes APache OODT product IDs as input.
2. Implemented new pre GFac Handler by extending Apache OODT PgeTaskInstance to stage the corresponding file into the working directory.
3. Once file is staged, input parameter with OODT product id is replaced with path of the staged file for downstream processing
I have tested the implementation with Gfac application which wraps /bin/ls command. Application takes product id as input and stage corresponding file into the working directory and /bin/ls is executed against the staged file. Hope this is a valid testing scenario.
Concerns
- Configurations : I have added new configuration file named and oodt-integration.properties in addition to dynamic_metadata.met and pge-config.xml files used by OODT. But at the moment there is no item configured with the oodt-integration.properties.
- Staged File Name - With the current implementation of PgeTaskInstance it is not possible to retrieve path of the staged file. Due to this limitation, I have query the FileManagerServer with product id and retrieve the file name and computed the file path using information of working directory.
- Currently it is not possible to execute the workflow using Xbaya due to validation failure due to new schema type. I have commented out the relevant validation code for testing purpose.
Diffs
-----
https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/java/org/apache/airavata/commons/gfac/type/MappingFactory.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/resources/schemas/GFacParameterTypes.xsd 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/pom.xml 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/assembly/bin-assembly.xml 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/dynamic_metadata.met PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/gfac-config.xml 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/oodt_integration.properties PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/pge-config.xml PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/gfac-core/src/main/java/org/apache/airavata/gfac/utils/GFacUtils.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/pom.xml PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataToApacheOODTConfigurationMapper.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataWorkFlowInstance.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataWorkFlowInstanceImpl.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/OODTProduct.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/OODTProductStager.java PRE-CREATION
https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/component/ws/WSComponent.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/gpel/script/WorkflowWSDL.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/xsd/GFacSimpleTypesXSD.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/modules/xbaya-gui/src/main/java/org/apache/airavata/xbaya/lead/LEADTypes.java 1487802
https://svn.apache.org/repos/asf/airavata/trunk/pom.xml 1487802
Diff: https://reviews.apache.org/r/11565/diff/
Testing
-------
Thanks,
Sanjaya Medonsa