You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oodt.apache.org by "Chris A. Mattmann (JIRA)" <ji...@apache.org> on 2011/06/19 01:56:47 UTC
[jira] [Created] (OODT-202) NonBlockWorkflow double processing and
not pausing
NonBlockWorkflow double processing and not pausing
--------------------------------------------------
Key: OODT-202
URL: https://issues.apache.org/jira/browse/OODT-202
Project: OODT
Issue Type: Bug
Components: workflow manager
Environment: from JPL's last internal WM release JIRA
Reporter: Brian Foster
Fix For: 0.4
Here is the case i ran:
4 Workflows:
MoaOrbits : consists of 1 task {MoaOrbits}
MoaIasiGPolygonMap : consists of 2 task {MoaIasiGPolygon, MoaIasiMap}
MoaAmsuaGPolygonMap : consists of 2 task {MoaAmsuaGPolygon, MoaAmsuaMap}
MoaMhsGPolygonMap : consists of 2 task {MoaMhsGPolygon, MoaMhsMap}
I summit MoaOrbits workflow. On ingest of its primary product MOA_ORBITS, it summits (via a cas-crawler action) all 3 GPolygonMap workflows in the order given above.
None of the GPolygonMap workflows are related to each other in any other way.
Required Task Input:
MoaOrbit task: a days worth of IASI files
GPolygon tasks: the Orbit file created by MoaOrbits task and a days worth of the appropriate METOPA L1 files (IASI | AMSUA | MHS)
Map tasks: the GPolygon file created by the GPolgon task in the same workflow
This is how it is expected to run and currently runs with standard workflow engine:
MoaOrbits run successfully and ingests a MOA_ORBITS product
MoaIasiGPolygonMap runs successfully ingesting a MOA_IASI_GPOLYGON product and a MOA_IASI_MAP product
MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and thus the MoaAmsuaMap task fails to ever pass conditions to run
MoaMhsGPolygonMap runs successfully ingesting a MOA_MHS_GPOLYGON product and a MOA_MHS_MAP product
This is how it runs with non-blocking workflow engine:
MoaOrbits run successfully and ingests a MOA_ORBITS product
MoaIasiGPolygonMap runs TWICE successfully ingesting 2 MOA_IASI_GPOLYGON products and 2 MOA_IASI_MAP products
MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and the MoaAmsuaMap task fails to ever pass preconditions to run and
instead of waiting a few seconds between checking conditions it dumps to the logger this message many times a second: INFO: Task: [MoaAmsuaMap] has no required metadata fields
MoaMhsGPolygonMap runs TWICE successfully ingesting 2 MOA_MHS_GPOLYGON products and 2 MOA_MHS_MAP products
The short of it seems like workflows with > 1 task get run twice and workflows with task that block have no wait time between rechecking conditions
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (OODT-202) NonBlockWorkflow double processing and
not pausing
Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris A. Mattmann reassigned OODT-202:
--------------------------------------
Assignee: Chris A. Mattmann
> NonBlockWorkflow double processing and not pausing
> --------------------------------------------------
>
> Key: OODT-202
> URL: https://issues.apache.org/jira/browse/OODT-202
> Project: OODT
> Issue Type: Sub-task
> Components: workflow manager
> Environment: from JPL's last internal WM release JIRA
> Reporter: Brian Foster
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> Here is the case i ran:
> 4 Workflows:
> MoaOrbits : consists of 1 task {MoaOrbits}
> MoaIasiGPolygonMap : consists of 2 task {MoaIasiGPolygon, MoaIasiMap}
> MoaAmsuaGPolygonMap : consists of 2 task {MoaAmsuaGPolygon, MoaAmsuaMap}
> MoaMhsGPolygonMap : consists of 2 task {MoaMhsGPolygon, MoaMhsMap}
> I summit MoaOrbits workflow. On ingest of its primary product MOA_ORBITS, it summits (via a cas-crawler action) all 3 GPolygonMap workflows in the order given above.
> None of the GPolygonMap workflows are related to each other in any other way.
> Required Task Input:
> MoaOrbit task: a days worth of IASI files
> GPolygon tasks: the Orbit file created by MoaOrbits task and a days worth of the appropriate METOPA L1 files (IASI | AMSUA | MHS)
> Map tasks: the GPolygon file created by the GPolgon task in the same workflow
> This is how it is expected to run and currently runs with standard workflow engine:
> MoaOrbits run successfully and ingests a MOA_ORBITS product
> MoaIasiGPolygonMap runs successfully ingesting a MOA_IASI_GPOLYGON product and a MOA_IASI_MAP product
> MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and thus the MoaAmsuaMap task fails to ever pass conditions to run
> MoaMhsGPolygonMap runs successfully ingesting a MOA_MHS_GPOLYGON product and a MOA_MHS_MAP product
> This is how it runs with non-blocking workflow engine:
> MoaOrbits run successfully and ingests a MOA_ORBITS product
> MoaIasiGPolygonMap runs TWICE successfully ingesting 2 MOA_IASI_GPOLYGON products and 2 MOA_IASI_MAP products
> MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and the MoaAmsuaMap task fails to ever pass preconditions to run and
> instead of waiting a few seconds between checking conditions it dumps to the logger this message many times a second: INFO: Task: [MoaAmsuaMap] has no required metadata fields
> MoaMhsGPolygonMap runs TWICE successfully ingesting 2 MOA_MHS_GPOLYGON products and 2 MOA_MHS_MAP products
> The short of it seems like workflows with > 1 task get run twice and workflows with task that block have no wait time between rechecking conditions
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OODT-202) NonBlockWorkflow double processing and
not pausing
Posted by "Chris A. Mattmann (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris A. Mattmann updated OODT-202:
-----------------------------------
Fix Version/s: (was: 0.4)
0.5
- push to 0.5
> NonBlockWorkflow double processing and not pausing
> --------------------------------------------------
>
> Key: OODT-202
> URL: https://issues.apache.org/jira/browse/OODT-202
> Project: OODT
> Issue Type: Sub-task
> Components: workflow manager
> Environment: from JPL's last internal WM release JIRA
> Reporter: Brian Foster
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
>
> Here is the case i ran:
> 4 Workflows:
> MoaOrbits : consists of 1 task {MoaOrbits}
> MoaIasiGPolygonMap : consists of 2 task {MoaIasiGPolygon, MoaIasiMap}
> MoaAmsuaGPolygonMap : consists of 2 task {MoaAmsuaGPolygon, MoaAmsuaMap}
> MoaMhsGPolygonMap : consists of 2 task {MoaMhsGPolygon, MoaMhsMap}
> I summit MoaOrbits workflow. On ingest of its primary product MOA_ORBITS, it summits (via a cas-crawler action) all 3 GPolygonMap workflows in the order given above.
> None of the GPolygonMap workflows are related to each other in any other way.
> Required Task Input:
> MoaOrbit task: a days worth of IASI files
> GPolygon tasks: the Orbit file created by MoaOrbits task and a days worth of the appropriate METOPA L1 files (IASI | AMSUA | MHS)
> Map tasks: the GPolygon file created by the GPolgon task in the same workflow
> This is how it is expected to run and currently runs with standard workflow engine:
> MoaOrbits run successfully and ingests a MOA_ORBITS product
> MoaIasiGPolygonMap runs successfully ingesting a MOA_IASI_GPOLYGON product and a MOA_IASI_MAP product
> MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and thus the MoaAmsuaMap task fails to ever pass conditions to run
> MoaMhsGPolygonMap runs successfully ingesting a MOA_MHS_GPOLYGON product and a MOA_MHS_MAP product
> This is how it runs with non-blocking workflow engine:
> MoaOrbits run successfully and ingests a MOA_ORBITS product
> MoaIasiGPolygonMap runs TWICE successfully ingesting 2 MOA_IASI_GPOLYGON products and 2 MOA_IASI_MAP products
> MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and the MoaAmsuaMap task fails to ever pass preconditions to run and
> instead of waiting a few seconds between checking conditions it dumps to the logger this message many times a second: INFO: Task: [MoaAmsuaMap] has no required metadata fields
> MoaMhsGPolygonMap runs TWICE successfully ingesting 2 MOA_MHS_GPOLYGON products and 2 MOA_MHS_MAP products
> The short of it seems like workflows with > 1 task get run twice and workflows with task that block have no wait time between rechecking conditions
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OODT-202) NonBlockWorkflow double processing and
not pausing
Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris A. Mattmann updated OODT-202:
-----------------------------------
Issue Type: Sub-task (was: Bug)
Parent: OODT-215
> NonBlockWorkflow double processing and not pausing
> --------------------------------------------------
>
> Key: OODT-202
> URL: https://issues.apache.org/jira/browse/OODT-202
> Project: OODT
> Issue Type: Sub-task
> Components: workflow manager
> Environment: from JPL's last internal WM release JIRA
> Reporter: Brian Foster
> Fix For: 0.4
>
>
> Here is the case i ran:
> 4 Workflows:
> MoaOrbits : consists of 1 task {MoaOrbits}
> MoaIasiGPolygonMap : consists of 2 task {MoaIasiGPolygon, MoaIasiMap}
> MoaAmsuaGPolygonMap : consists of 2 task {MoaAmsuaGPolygon, MoaAmsuaMap}
> MoaMhsGPolygonMap : consists of 2 task {MoaMhsGPolygon, MoaMhsMap}
> I summit MoaOrbits workflow. On ingest of its primary product MOA_ORBITS, it summits (via a cas-crawler action) all 3 GPolygonMap workflows in the order given above.
> None of the GPolygonMap workflows are related to each other in any other way.
> Required Task Input:
> MoaOrbit task: a days worth of IASI files
> GPolygon tasks: the Orbit file created by MoaOrbits task and a days worth of the appropriate METOPA L1 files (IASI | AMSUA | MHS)
> Map tasks: the GPolygon file created by the GPolgon task in the same workflow
> This is how it is expected to run and currently runs with standard workflow engine:
> MoaOrbits run successfully and ingests a MOA_ORBITS product
> MoaIasiGPolygonMap runs successfully ingesting a MOA_IASI_GPOLYGON product and a MOA_IASI_MAP product
> MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and thus the MoaAmsuaMap task fails to ever pass conditions to run
> MoaMhsGPolygonMap runs successfully ingesting a MOA_MHS_GPOLYGON product and a MOA_MHS_MAP product
> This is how it runs with non-blocking workflow engine:
> MoaOrbits run successfully and ingests a MOA_ORBITS product
> MoaIasiGPolygonMap runs TWICE successfully ingesting 2 MOA_IASI_GPOLYGON products and 2 MOA_IASI_MAP products
> MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and the MoaAmsuaMap task fails to ever pass preconditions to run and
> instead of waiting a few seconds between checking conditions it dumps to the logger this message many times a second: INFO: Task: [MoaAmsuaMap] has no required metadata fields
> MoaMhsGPolygonMap runs TWICE successfully ingesting 2 MOA_MHS_GPOLYGON products and 2 MOA_MHS_MAP products
> The short of it seems like workflows with > 1 task get run twice and workflows with task that block have no wait time between rechecking conditions
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OODT-202) NonBlockWorkflow double processing and
not pausing
Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris A. Mattmann resolved OODT-202.
------------------------------------
Resolution: Not A Problem
- the old class that exhibited this behavior was removed. Once the last few wengine items are wrapped for 0.5 this should be fine.
> NonBlockWorkflow double processing and not pausing
> --------------------------------------------------
>
> Key: OODT-202
> URL: https://issues.apache.org/jira/browse/OODT-202
> Project: OODT
> Issue Type: Sub-task
> Components: workflow manager
> Environment: from JPL's last internal WM release JIRA
> Reporter: Brian Foster
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
>
> Here is the case i ran:
> 4 Workflows:
> MoaOrbits : consists of 1 task {MoaOrbits}
> MoaIasiGPolygonMap : consists of 2 task {MoaIasiGPolygon, MoaIasiMap}
> MoaAmsuaGPolygonMap : consists of 2 task {MoaAmsuaGPolygon, MoaAmsuaMap}
> MoaMhsGPolygonMap : consists of 2 task {MoaMhsGPolygon, MoaMhsMap}
> I summit MoaOrbits workflow. On ingest of its primary product MOA_ORBITS, it summits (via a cas-crawler action) all 3 GPolygonMap workflows in the order given above.
> None of the GPolygonMap workflows are related to each other in any other way.
> Required Task Input:
> MoaOrbit task: a days worth of IASI files
> GPolygon tasks: the Orbit file created by MoaOrbits task and a days worth of the appropriate METOPA L1 files (IASI | AMSUA | MHS)
> Map tasks: the GPolygon file created by the GPolgon task in the same workflow
> This is how it is expected to run and currently runs with standard workflow engine:
> MoaOrbits run successfully and ingests a MOA_ORBITS product
> MoaIasiGPolygonMap runs successfully ingesting a MOA_IASI_GPOLYGON product and a MOA_IASI_MAP product
> MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and thus the MoaAmsuaMap task fails to ever pass conditions to run
> MoaMhsGPolygonMap runs successfully ingesting a MOA_MHS_GPOLYGON product and a MOA_MHS_MAP product
> This is how it runs with non-blocking workflow engine:
> MoaOrbits run successfully and ingests a MOA_ORBITS product
> MoaIasiGPolygonMap runs TWICE successfully ingesting 2 MOA_IASI_GPOLYGON products and 2 MOA_IASI_MAP products
> MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and the MoaAmsuaMap task fails to ever pass preconditions to run and
> instead of waiting a few seconds between checking conditions it dumps to the logger this message many times a second: INFO: Task: [MoaAmsuaMap] has no required metadata fields
> MoaMhsGPolygonMap runs TWICE successfully ingesting 2 MOA_MHS_GPOLYGON products and 2 MOA_MHS_MAP products
> The short of it seems like workflows with > 1 task get run twice and workflows with task that block have no wait time between rechecking conditions
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OODT-202) NonBlockWorkflow double processing and
not pausing
Posted by "Chris A. Mattmann (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris A. Mattmann updated OODT-202:
-----------------------------------
- push out to 0.5
> NonBlockWorkflow double processing and not pausing
> --------------------------------------------------
>
> Key: OODT-202
> URL: https://issues.apache.org/jira/browse/OODT-202
> Project: OODT
> Issue Type: Sub-task
> Components: workflow manager
> Environment: from JPL's last internal WM release JIRA
> Reporter: Brian Foster
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
>
> Here is the case i ran:
> 4 Workflows:
> MoaOrbits : consists of 1 task {MoaOrbits}
> MoaIasiGPolygonMap : consists of 2 task {MoaIasiGPolygon, MoaIasiMap}
> MoaAmsuaGPolygonMap : consists of 2 task {MoaAmsuaGPolygon, MoaAmsuaMap}
> MoaMhsGPolygonMap : consists of 2 task {MoaMhsGPolygon, MoaMhsMap}
> I summit MoaOrbits workflow. On ingest of its primary product MOA_ORBITS, it summits (via a cas-crawler action) all 3 GPolygonMap workflows in the order given above.
> None of the GPolygonMap workflows are related to each other in any other way.
> Required Task Input:
> MoaOrbit task: a days worth of IASI files
> GPolygon tasks: the Orbit file created by MoaOrbits task and a days worth of the appropriate METOPA L1 files (IASI | AMSUA | MHS)
> Map tasks: the GPolygon file created by the GPolgon task in the same workflow
> This is how it is expected to run and currently runs with standard workflow engine:
> MoaOrbits run successfully and ingests a MOA_ORBITS product
> MoaIasiGPolygonMap runs successfully ingesting a MOA_IASI_GPOLYGON product and a MOA_IASI_MAP product
> MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and thus the MoaAmsuaMap task fails to ever pass conditions to run
> MoaMhsGPolygonMap runs successfully ingesting a MOA_MHS_GPOLYGON product and a MOA_MHS_MAP product
> This is how it runs with non-blocking workflow engine:
> MoaOrbits run successfully and ingests a MOA_ORBITS product
> MoaIasiGPolygonMap runs TWICE successfully ingesting 2 MOA_IASI_GPOLYGON products and 2 MOA_IASI_MAP products
> MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and the MoaAmsuaMap task fails to ever pass preconditions to run and
> instead of waiting a few seconds between checking conditions it dumps to the logger this message many times a second: INFO: Task: [MoaAmsuaMap] has no required metadata fields
> MoaMhsGPolygonMap runs TWICE successfully ingesting 2 MOA_MHS_GPOLYGON products and 2 MOA_MHS_MAP products
> The short of it seems like workflows with > 1 task get run twice and workflows with task that block have no wait time between rechecking conditions
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira