You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oodt.apache.org by "Chris A. Mattmann (JIRA)" <ji...@apache.org> on 2012/09/02 19:03:07 UTC
[jira] [Created] (OODT-491) Finish line tasks for Wengine
integration
Chris A. Mattmann created OODT-491:
--------------------------------------
Summary: Finish line tasks for Wengine integration
Key: OODT-491
URL: https://issues.apache.org/jira/browse/OODT-491
Project: OODT
Issue Type: Umbrella
Components: workflow manager
Reporter: Chris A. Mattmann
Assignee: Chris A. Mattmann
Fix For: 0.5
The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Brian Foster (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480215#comment-13480215 ]
Brian Foster commented on OODT-491:
-----------------------------------
okay so i've been digging more into the WorkflowProcessor stuff... here are a few things i think we should change:
1) the cache is actually holding all the WorkflowProcessors in memory at the same time, this will cause memory issues really quick, so here is was i purpose: WorkflowInstance is holding the state of each WorkflowProcessor (and anything that isn't, move to the WorkflowInstance -- see #2 about this though), so let not even keep the WorkflowProcessors in memory (they can be rebuilt when they are needed from the WorkflowInstance objects). This will required another ID field to be added to WorkflowInstance though (maybe just a metadata field -- see #2 below)... i just need something such that given a WorkflowInstance I am able to determine other WorkflowInstance(s) which it is connected to via WorkflowProcessors... this model would allow for me in the future to cache static instances of WorkflowProcessors and just use them to process WorkflowInstance when needed (would be a huge speed gain -- but that would come as an improvement at a later date).
2) Also i think WorkflowInstance class is getting cluttered with member variables that don't always apply to a Workflow... also variables like currentTaskId now makes no sense for parallel workflows (this is very much a sequential workflow field)... what if we come up with a reserved metadata group (i.e. workflow/) which a WorkflowProcessor can dump its custom things into and then read back out (i.e. priority, number of retries, current task(s), etc...)... that way WorkflowInstance can just be kept simple (start and end date, etc...)... then we deprecate currentTaskId.
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476193#comment-13476193 ]
Chris A. Mattmann commented on OODT-491:
----------------------------------------
Hey BFost, totally open to what you propose...in 0.6 :) I'd like to simply finish what I started though here and I'm getting closer and closer to being able to say "yes" on that. Yes it's complicated and in the end I think I'd rather simply use the ThreadPoolWorkflowEngine until we move to what you are suggesting in 0.6+ and beyond. But, I'd like to deliver a version of the workflow manager with a workflow PrioritizedQueueBasedWorkflowEngine and call it "done" for 0.5. I also agree that the greatest thing out of this was all the improvements that came about because of the Workflow2 (CAS-CLI; dynamic workflows; global workflow conditions (pre + post); Runner framework, etc.)
So yes, I'm game to try what you are proposing, but if you are willing to help me wrap up 0.5 as is that would be great. Also, if you don't have time or energy to work on it with me that's fine too b/c I see a light at the end of the tunnel and know precisely how to wrap these finish line tasks up and get this thing working, so hang in there :)
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476628#comment-13476628 ]
Chris A. Mattmann commented on OODT-491:
----------------------------------------
Hey Brian:
{quote}
Still not sure how this is going to work... each WorkflowProcessor will usually have multiple WorkflowIds,
{quote}
This may have been true in wengine but isn't strictly true in 0.5. Each Workflow Processor handles a single model id (or Workflow Id in trunk terminology). Sure, it may have sub processors which themselves have model IDs, etc., though, which is I think what you are talking below.
{quote}
so for an instance to update it will need: updateStatus(workflowInstanceId, workflowId, status) and updateMetadata(workflowInstanceId, workflowId, status) or combine them (which actually is a hugh speed boost): updateWorkflowInstance(workflowInstanceId, workflowId, status, metadata)...
{quote}
Yep I like the above. I'll file a JIRA issue to support that as part of OODT-491. Care to take a whack and help me on the above?
{quote}
these methods then have to notify the WorkflowProcessor from which the WorkflowTaskInstance came from, which then notifies it's parent WorkflowProcessors to update their status and metadata as well.
{quote}
Yep.
{quote}
There's a method here: updateWorkflowInstanceStatus that we can use for this.
This isn't going to work for parallel workflows... multiple posts from different tasks in the same workflow are going to clobber each others writes (since they are unaware of each other's updates to the WorkflowInstance)... i think the server should be responsible for updating the WorkflowInstance itself (what this method appears to do is a blank update)... here is an example of what i mean: Task1 has WorkflowInstance object and Task2 has the same WorkflowInstance object, both which have Task1 and Task2 as running... now Task1 finishes and sends an update to WorkflowManager, then Task2 finishes and sends in an update to the WorkflowManager, the WorkflowManager will perform Task1 update request, then Task2 update request – WorkflowManager will now think that Task1 is still running again, because that is the state of it in Task2 WorkflowInstance... or you will have to request a fresh copy of each WorkflowInstance before updating but this will be very time consuming and although will decrease the number of Tasks clobbering each other's writes it won't completely prevent it
{quote}
Yeah I see what you mean. Take a look at [persist in WorkflowProcessorQueue|https://builds.apache.org/job/oodt-trunk/javadoc/org/apache/oodt/cas/workflow/engine/processor/WorkflowProcessorQueue.html#persist(org.apache.oodt.cas.workflow.structs.WorkflowInstance)] which is what I was trying to do to accomplish the same thing you're talking about (I was running into Lucene issues before that with all the parallel updating). However it's fixed now and unit-tested :) and they are passing.
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Brian Foster (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477604#comment-13477604 ]
Brian Foster commented on OODT-491:
-----------------------------------
i think we are talking around each other like always... lol... we'll have to talk in person... in the meantime i'll get to work putting the final touches on WorkflowProcessor stuff
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477622#comment-13477622 ]
Chris A. Mattmann commented on OODT-491:
----------------------------------------
Wouldn't be the first time :) How about we agree that we are in bro love and yes, we will need to chat in person which is fine b/c we'll bring the decisions back to the list and to Review Board and stuff :)
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476378#comment-13476378 ]
Chris A. Mattmann edited comment on OODT-491 at 10/15/12 8:03 PM:
------------------------------------------------------------------
Hey BFost:
Thanks, here is what I was thinking:
bq. how do you plan on handling WorkflowInstances updating the appropriate modelId in the WorkflowProcessor they came from (the metadata needs to be flowed through as well)
For us, model ID is WorkflowId, we should set this probably when the workflow is initially created, and then later in fromWorkflowInstance, maybe?
bq. changing the status of a modelId in a WorkflowProcessor? it also looks like you lost support for state change propagating up the parent WorkflowProcessor. A WorkflowInstance will update a status on a leaf WorkflowProcessor and all parent WorkflowProcessors need to then update their statuses.
There's a method here: [updateWorkflowInstanceStatus|http://oodt.apache.org/components/maven/xref/org/apache/oodt/cas/workflow/system/XmlRpcWorkflowManagerClient.html#401] that we can use for this.
bq. Also it appears you lost support for rolling back status to a queued status for a failed workflow – you also need backward status calculation in the WorkflowProcessor. (i know the original long method for determining next state for a WorkflowProcessor, with a ton of if statements, was messy and hard to comprehend, but that is what it accomplished)
Yeah maybe we can make a function to handle this. Thoughts? Or make it part of OODT-495 maybe.
was (Author: chrismattmann):
Hey BFost:
Thanks, here is what I was thinking:
bq. how do you plan on handling WorkflowInstances updating the appropriate modelId in the WorkflowProcessor they came from (the metadata needs to be flowed through as well)
For us, model ID is WorkflowId, we should set this probably when the workflow is initially created, and then later in fromWorkflowInstance, maybe?
bq. changing the status of a modelId in a WorkflowProcessor? it also looks like you lost support for state change propagating up the parent WorkflowProcessor. A WorkflowInstance will update a status on a leaf WorkflowProcessor and all parent WorkflowProcessors need to then update their statuses.
There's a method here: [http://oodt.apache.org/components/maven/xref/org/apache/oodt/cas/workflow/system/XmlRpcWorkflowManagerClient.html#401|updateWorkflowInstanceStatus] that we can use for this.
bq. Also it appears you lost support for rolling back status to a queued status for a failed workflow – you also need backward status calculation in the WorkflowProcessor. (i know the original long method for determining next state for a WorkflowProcessor, with a ton of if statements, was messy and hard to comprehend, but that is what it accomplished)
Yeah maybe we can make a function to handle this. Thoughts? Or make it part of OODT-495 maybe.
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Brian Foster (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Foster updated OODT-491:
------------------------------
Comment: was deleted
(was: i'll help, got a few questions though, seems like there are several (difficult) things missing from trunk at the moment for this port, a few of them are:
1) how do you plan on handling WorkflowInstances updating the appropriate modelId in the WorkflowProcessor they came from (the metadata needs to be flowed through as well)
2) changing the status of a modelId in a WorkflowProcessor? it also looks like you lost support for state change propagating up the parent WorkflowProcessor. A WorkflowInstance will update a status on a leaf WorkflowProcessor and all parent WorkflowProcessors need to then update their statuses. Also it appears you lost support for rolling back status to a queued status for a failed workflow -- you also need backward status calculation in the WorkflowProcessor. (i know the original long method for determining next state for a WorkflowProcessor, with a ton of if statements, was messy and hard to comprehend, but that is what it accomplished)
)
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Brian Foster (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476310#comment-13476310 ]
Brian Foster commented on OODT-491:
-----------------------------------
i'll help, got a few questions though, seems like there are several (difficult) things missing from trunk at the moment for this port, a few of them are:
1) how do you plan on handling WorkflowInstances updating the appropriate modelId in the WorkflowProcessor they came from (the metadata needs to be flowed through as well)
2) changing the status of a modelId in a WorkflowProcessor? it also looks like you lost support for state change propagating up the parent WorkflowProcessor. A WorkflowInstance will update a status on a leaf WorkflowProcessor and all parent WorkflowProcessors need to then update their statuses. Also it appears you lost support for rolling back status to a queued status for a failed workflow -- you also need backward status calculation in the WorkflowProcessor. (i know the original long method for determining next state for a WorkflowProcessor, with a ton of if statements, was messy and hard to comprehend, but that is what it accomplished)
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Brian Foster (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476553#comment-13476553 ]
Brian Foster edited comment on OODT-491 at 10/15/12 11:13 PM:
--------------------------------------------------------------
1) For us, model ID is WorkflowId, we should set this probably when the workflow is initially created, and then later in fromWorkflowInstance, maybe?
- Still not sure how this is going to work... each WorkflowProcessor will usually have multiple WorkflowIds, so for an instance to update it will need: updateStatus(workflowInstanceId, workflowId, status) and updateMetadata(workflowInstanceId, workflowId, status) or combine them (which actually is a hugh speed boost): updateWorkflowInstance(workflowInstanceId, workflowId, status, metadata)... these methods then have to notify the WorkflowProcessor from which the WorkflowTaskInstance came from, which then notifies it's parent WorkflowProcessors to update their status and metadata as well.
2) There's a method here: updateWorkflowInstanceStatus that we can use for this.
- This isn't going to work for parallel workflows... multiple posts from different tasks in the same workflow are going to clobber each others writes (since they are unaware of each other's updates to the WorkflowInstance)... i think the server should be responsible for updating the WorkflowInstance itself (what this method appears to do is a blank update)... here is an example of what i mean: Task1 has WorkflowInstance object and Task2 has the same WorkflowInstance object, both which have Task1 and Task2 as running... now Task1 finishes and sends an update to WorkflowManager, then Task2 finishes and sends in an update to the WorkflowManager, the WorkflowManager will perform Task1 update request, then Task2 update request -- WorkflowManager will now think that Task1 is still running again, because that is the state of it in Task2 WorkflowInstance... or you will have to request a fresh copy of each WorkflowInstance before updating but this will be very time consuming and although will decrease the number of Tasks clobbering each other's writes it won't completely prevent it
3) Yeah maybe we can make a function to handle this. Thoughts? Or make it part of OODT-495 maybe.
- Determining status, whether moving forward or backwards is really the same thing, a WorkflowProcessor has a set of children WorkflowProcessor and based on their statuses, it has a status... so when update status is call on any WorkflowProcessor it should just analyze it's children WorkflowProcessor's status and if it needs to update it's status because one or more of them changed then it should notify it's parent WorkflowProcessor to do the same. This could be a task that is part of OODT-495 i believe.
was (Author: bfoster):
1) For us, model ID is WorkflowId, we should set this probably when the workflow is initially created, and then later in fromWorkflowInstance, maybe?
- Still not sure how this is going to work... each workflow may have multiple WorkflowIds, so for an instance to update it will need: updateStatus(workflowInstanceId, workflowId, status) and updateMetadata(workflowInstanceId, workflowId, status) or combine them (which actually is a hugh speed boost): updateWorkflowInstance(workflowInstanceId, workflowId, status, metadata)... these methods then have to notify the WorkflowProcessor from which the WorkflowTaskInstance came from, which then notifies it's parent WorkflowProcessors to update their status and metadata as well.
2) There's a method here: updateWorkflowInstanceStatus that we can use for this.
- This isn't going to work for parallel workflows... multiple posts from different tasks in the same workflow are going to clobber each others writes (since they are unaware of each other's updates to the WorkflowInstance)... i think the server should be responsible for updating the WorkflowInstance itself (what this method appears to do is a blank update)... here is an example of what i mean: Task1 has WorkflowInstance object and Task2 has the same WorkflowInstance object, both which have Task1 and Task2 as running... now Task1 finishes and sends an update to WorkflowManager, then Task2 finishes and sends in an update to the WorkflowManager, the WorkflowManager will perform Task1 update request, then Task2 update request -- WorkflowManager will now think that Task1 is still running again, because that is the state of it in Task2 WorkflowInstance... or you will have to request a fresh copy of each WorkflowInstance before updating but this will be very time consuming and although will decrease the number of Tasks clobbering each other's writes it won't completely prevent it
3) Yeah maybe we can make a function to handle this. Thoughts? Or make it part of OODT-495 maybe.
- Determining status, whether moving forward or backwards is really the same thing, a WorkflowProcessor has a set of children WorkflowProcessor and based on their statuses, it has a status... so when update status is call on any WorkflowProcessor it should just analyze it's children WorkflowProcessor's status and if it needs to update it's status because one or more of them changed then it should notify it's parent WorkflowProcessor to do the same. This could be a task that is part of OODT-495 i believe.
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Brian Foster (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476553#comment-13476553 ]
Brian Foster commented on OODT-491:
-----------------------------------
1) For us, model ID is WorkflowId, we should set this probably when the workflow is initially created, and then later in fromWorkflowInstance, maybe?
- Still not sure how this is going to work... each workflow may have multiple WorkflowIds, so for an instance to update it will need: updateStatus(workflowInstanceId, workflowId, status) and updateMetadata(workflowInstanceId, workflowId, status) or combine them (which actually is a hugh speed boost): updateWorkflowInstance(workflowInstanceId, workflowId, status, metadata)... these methods then have to notify the WorkflowProcessor from which the WorkflowTaskInstance came from, which then notifies it's parent WorkflowProcessors to update their status and metadata as well.
2) There's a method here: updateWorkflowInstanceStatus that we can use for this.
- This isn't going to work for parallel workflows... multiple posts from different tasks in the same workflow are going to clobber each others writes (since they are unaware of each other's updates to the WorkflowInstance)... i think the server should be responsible for updating the WorkflowInstance itself (what this method appears to do is a blank update)... here is an example of what i mean: Task1 has WorkflowInstance object and Task2 has the same WorkflowInstance object, both which have Task1 and Task2 as running... now Task1 finishes and sends an update to WorkflowManager, then Task2 finishes and sends in an update to the WorkflowManager, the WorkflowManager will perform Task1 update request, then Task2 update request -- WorkflowManager will now think that Task1 is still running again, because that is the state of it in Task2 WorkflowInstance... or you will have to request a fresh copy of each WorkflowInstance before updating but this will be very time consuming and although will decrease the number of Tasks clobbering each other's writes it won't completely prevent it
3) Yeah maybe we can make a function to handle this. Thoughts? Or make it part of OODT-495 maybe.
- Determining status, whether moving forward or backwards is really the same thing, a WorkflowProcessor has a set of children WorkflowProcessor and based on their statuses, it has a status... so when update status is call on any WorkflowProcessor it should just analyze it's children WorkflowProcessor's status and if it needs to update it's status because one or more of them changed then it should notify it's parent WorkflowProcessor to do the same. This could be a task that is part of OODT-495 i believe.
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476378#comment-13476378 ]
Chris A. Mattmann commented on OODT-491:
----------------------------------------
Hey BFost:
Thanks, here is what I was thinking:
bq. how do you plan on handling WorkflowInstances updating the appropriate modelId in the WorkflowProcessor they came from (the metadata needs to be flowed through as well)
For us, model ID is WorkflowId, we should set this probably when the workflow is initially created, and then later in fromWorkflowInstance, maybe?
bq. changing the status of a modelId in a WorkflowProcessor? it also looks like you lost support for state change propagating up the parent WorkflowProcessor. A WorkflowInstance will update a status on a leaf WorkflowProcessor and all parent WorkflowProcessors need to then update their statuses.
There's a method here: [http://oodt.apache.org/components/maven/xref/org/apache/oodt/cas/workflow/system/XmlRpcWorkflowManagerClient.html#401|updateWorkflowInstanceStatus] that we can use for this.
bq. Also it appears you lost support for rolling back status to a queued status for a failed workflow – you also need backward status calculation in the WorkflowProcessor. (i know the original long method for determining next state for a WorkflowProcessor, with a ton of if statements, was messy and hard to comprehend, but that is what it accomplished)
Yeah maybe we can make a function to handle this. Thoughts? Or make it part of OODT-495 maybe.
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Brian Foster (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476311#comment-13476311 ]
Brian Foster commented on OODT-491:
-----------------------------------
i'll help, got a few questions though, seems like there are several (difficult) things missing from trunk at the moment for this port, a few of them are:
1) how do you plan on handling WorkflowInstances updating the appropriate modelId in the WorkflowProcessor they came from (the metadata needs to be flowed through as well)
2) changing the status of a modelId in a WorkflowProcessor? it also looks like you lost support for state change propagating up the parent WorkflowProcessor. A WorkflowInstance will update a status on a leaf WorkflowProcessor and all parent WorkflowProcessors need to then update their statuses. Also it appears you lost support for rolling back status to a queued status for a failed workflow -- you also need backward status calculation in the WorkflowProcessor. (i know the original long method for determining next state for a WorkflowProcessor, with a ton of if statements, was messy and hard to comprehend, but that is what it accomplished)
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OODT-491) Finish line tasks for Wengine
integration
Posted by "Brian Foster (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Foster updated OODT-491:
------------------------------
Attachment: NewWorkflowModel.patch.txt
okay chris so i've taken a look over the current wengine port to trunk code state… and hear me all the way out on this, i think by the time you make it to the end of this comment you will agree with me (i hope… lol)… i think the wengine port stuff in trunk is still missing a lot of stuff to make it work… i also think the WorkflowProcessor part of the port is making the trunk messy (i.e. for one ParentChildWorkflow is causing unwanted instanceof checking which also makes for confusing code)… kinda showing now why i decided to write WEngine as a separate component… i think a lot of good changes have come out of the attempt of this wengine port: separation of Runner, allowing for the workflow engine to not get held up by stalled workflows, CAS-CLI was ironed out and refactored into it's own component based on it's original use in winging there are several others… so here is what i am proposing:
I think for now let's use the WEngine WorkflowProcessor stuff to add design improvements to the current Workflow model to incorporate the features which WEngine WorkflowProcessor provided but with a deferent model approach… I have attached a patch which modifies the struct class Workflow to allow for this integration and also introduces the idea (which Chris and i talked about a long time ago) of chaining workflows… this is basically how I see it working: We will introduce the idea of a WorkflowLink, which will be a wrapper around a Workflow which will attach Events to a Workflow (a wait for Event, a trigger on success Event, and a trigger on failure Event)… currently all the trunk workflows will translate easily to this model as just a workflow chain with 1 WorkflowLink (with a wait for Event)… this bring me to the next point, these WorkflowLinks will be added to a WorkflowChain which will handle figuring out which WorkflowLinks should be run next given a fired Event. This will keep the trunk workflow completely Event driven like it has always been and will not require WorkflowProcessor based changes to poke holes in the client and server XML-RPC classes which are currently missing to make the WorkflowProcessor stuff work… i believe we should be able to rewrite the XML parser for the WEngine workflow configuration to build the model i've created in the patch attach if current when users what to port there stuff to this new model… anyway, let me know what you think
> Finish line tasks for Wengine integration
> -----------------------------------------
>
> Key: OODT-491
> URL: https://issues.apache.org/jira/browse/OODT-491
> Project: OODT
> Issue Type: Umbrella
> Components: workflow manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.5
>
> Attachments: NewWorkflowModel.patch.txt
>
>
> The final tasks to wrap up wengine integration into trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira