You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by "Cayanan, Michael D (388J)" <mi...@jpl.nasa.gov> on 2013/01/07 17:24:24 UTC
Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov" />
<property name="mail.from" value="oodt@jpl.nasa.gov" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Cayanan, Michael D (388J)" <mi...@jpl.nasa.gov>.
Hey Chris,
Okay great. I will be sure to do that.
-Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 14, 2013 9:22 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
Thanks for your replies below — go ahead and file the JIRA issue. It would be great if you could provide a unit test that:
1. Loads your test workflow policy
2. Asserts expected behavior
That way whenever we fix this we'll have a test that going forward will detect this.
Thanks Mike!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 14, 2013 9:06 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My answers are below in blue
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 11:00 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hi Mike,
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 5:54 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
Great! Please let me know what you find.
Will do.
Regarding the timeout feature, how does it work exactly? From the description in OODT-207, it appears that if a condition keeps failing for some configurable specified amount of time, then it will just return true and move on to the task?
You got it, that's what it does.
Or will it simply stop the workflow event all together due to continuous failing?
Nope it will only stop the failing on that task — is that the behavior you are seeing?
Ahhh — you may be seeing both behaviors. You are doing a global workflow condition timeout, right? So what you might be seeing with global workflow conditions are:
1. Global workflow condition adds a dynamic workflow condition ahead of the first task — you may be seeing (correctly) a timeout there, but;
So with the 'TestParallel' workflow defined in the example policy provided, I believe that global conditions aren't being evaluated at all since the 'HelloWorld', 'GoodbyeWorld' and 'Intensive' tasks all run to completion quite fast even though the timeout is set to '600'. Another test I did was modified the policy to have a sequential Workflow with global conditions like the following,
<sequential id="urn:oodt:TestSequential" name="TestSequential">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
</sequential>
What's interesting here is that the Workflow appears to be recognizing the global conditions at this point as indicated by the 'TestSequential-global-conditions-eval' found in the log,
Adding condition: [Timeout Condition] to parent workflow: [TestSequential]
...
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow TestSequential retrieved for event urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [TestSequential-global-conditions-eval] has no required metadata fields
However, it appears to be stuck at this point as it never moves onto the tasks.
1. Since the timeout is at the task level it simply moves on and into the next task, which makes you think it's an error.
I did another test where I put the timeout condition under the task level (GoodbyeWorld task in this case) and the condition didn't appear to be evaluated at all and just executed the GoodbyeWorld task right away.
Is that what you are seeing?
If so, how would you think it should behave differently?
I think the Timeout feature should operate as it is described in OODT-207.
Since it appears that the timeout feature is not working, should I file a JIRA ticket?
Cheers,
Chris
Thanks,
Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Cayanan, Michael D (388J)" <mi...@jpl.nasa.gov>.
Hey Chris,
Okay great. I will be sure to do that.
-Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 14, 2013 9:22 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
Thanks for your replies below — go ahead and file the JIRA issue. It would be great if you could provide a unit test that:
1. Loads your test workflow policy
2. Asserts expected behavior
That way whenever we fix this we'll have a test that going forward will detect this.
Thanks Mike!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 14, 2013 9:06 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My answers are below in blue
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 11:00 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hi Mike,
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 5:54 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
Great! Please let me know what you find.
Will do.
Regarding the timeout feature, how does it work exactly? From the description in OODT-207, it appears that if a condition keeps failing for some configurable specified amount of time, then it will just return true and move on to the task?
You got it, that's what it does.
Or will it simply stop the workflow event all together due to continuous failing?
Nope it will only stop the failing on that task — is that the behavior you are seeing?
Ahhh — you may be seeing both behaviors. You are doing a global workflow condition timeout, right? So what you might be seeing with global workflow conditions are:
1. Global workflow condition adds a dynamic workflow condition ahead of the first task — you may be seeing (correctly) a timeout there, but;
So with the 'TestParallel' workflow defined in the example policy provided, I believe that global conditions aren't being evaluated at all since the 'HelloWorld', 'GoodbyeWorld' and 'Intensive' tasks all run to completion quite fast even though the timeout is set to '600'. Another test I did was modified the policy to have a sequential Workflow with global conditions like the following,
<sequential id="urn:oodt:TestSequential" name="TestSequential">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
</sequential>
What's interesting here is that the Workflow appears to be recognizing the global conditions at this point as indicated by the 'TestSequential-global-conditions-eval' found in the log,
Adding condition: [Timeout Condition] to parent workflow: [TestSequential]
...
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow TestSequential retrieved for event urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [TestSequential-global-conditions-eval] has no required metadata fields
However, it appears to be stuck at this point as it never moves onto the tasks.
1. Since the timeout is at the task level it simply moves on and into the next task, which makes you think it's an error.
I did another test where I put the timeout condition under the task level (GoodbyeWorld task in this case) and the condition didn't appear to be evaluated at all and just executed the GoodbyeWorld task right away.
Is that what you are seeing?
If so, how would you think it should behave differently?
I think the Timeout feature should operate as it is described in OODT-207.
Since it appears that the timeout feature is not working, should I file a JIRA ticket?
Cheers,
Chris
Thanks,
Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Mike,
Thanks for your replies below — go ahead and file the JIRA issue. It would be great if you could provide a unit test that:
1. Loads your test workflow policy
2. Asserts expected behavior
That way whenever we fix this we'll have a test that going forward will detect this.
Thanks Mike!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 14, 2013 9:06 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My answers are below in blue
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 11:00 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hi Mike,
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 5:54 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
Great! Please let me know what you find.
Will do.
Regarding the timeout feature, how does it work exactly? From the description in OODT-207, it appears that if a condition keeps failing for some configurable specified amount of time, then it will just return true and move on to the task?
You got it, that's what it does.
Or will it simply stop the workflow event all together due to continuous failing?
Nope it will only stop the failing on that task — is that the behavior you are seeing?
Ahhh — you may be seeing both behaviors. You are doing a global workflow condition timeout, right? So what you might be seeing with global workflow conditions are:
1. Global workflow condition adds a dynamic workflow condition ahead of the first task — you may be seeing (correctly) a timeout there, but;
So with the 'TestParallel' workflow defined in the example policy provided, I believe that global conditions aren't being evaluated at all since the 'HelloWorld', 'GoodbyeWorld' and 'Intensive' tasks all run to completion quite fast even though the timeout is set to '600'. Another test I did was modified the policy to have a sequential Workflow with global conditions like the following,
<sequential id="urn:oodt:TestSequential" name="TestSequential">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
</sequential>
What's interesting here is that the Workflow appears to be recognizing the global conditions at this point as indicated by the 'TestSequential-global-conditions-eval' found in the log,
Adding condition: [Timeout Condition] to parent workflow: [TestSequential]
...
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow TestSequential retrieved for event urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [TestSequential-global-conditions-eval] has no required metadata fields
However, it appears to be stuck at this point as it never moves onto the tasks.
1. Since the timeout is at the task level it simply moves on and into the next task, which makes you think it's an error.
I did another test where I put the timeout condition under the task level (GoodbyeWorld task in this case) and the condition didn't appear to be evaluated at all and just executed the GoodbyeWorld task right away.
Is that what you are seeing?
If so, how would you think it should behave differently?
I think the Timeout feature should operate as it is described in OODT-207.
Since it appears that the timeout feature is not working, should I file a JIRA ticket?
Cheers,
Chris
Thanks,
Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Mike,
Thanks for your replies below — go ahead and file the JIRA issue. It would be great if you could provide a unit test that:
1. Loads your test workflow policy
2. Asserts expected behavior
That way whenever we fix this we'll have a test that going forward will detect this.
Thanks Mike!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 14, 2013 9:06 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My answers are below in blue
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 11:00 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hi Mike,
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 5:54 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
Great! Please let me know what you find.
Will do.
Regarding the timeout feature, how does it work exactly? From the description in OODT-207, it appears that if a condition keeps failing for some configurable specified amount of time, then it will just return true and move on to the task?
You got it, that's what it does.
Or will it simply stop the workflow event all together due to continuous failing?
Nope it will only stop the failing on that task — is that the behavior you are seeing?
Ahhh — you may be seeing both behaviors. You are doing a global workflow condition timeout, right? So what you might be seeing with global workflow conditions are:
1. Global workflow condition adds a dynamic workflow condition ahead of the first task — you may be seeing (correctly) a timeout there, but;
So with the 'TestParallel' workflow defined in the example policy provided, I believe that global conditions aren't being evaluated at all since the 'HelloWorld', 'GoodbyeWorld' and 'Intensive' tasks all run to completion quite fast even though the timeout is set to '600'. Another test I did was modified the policy to have a sequential Workflow with global conditions like the following,
<sequential id="urn:oodt:TestSequential" name="TestSequential">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
</sequential>
What's interesting here is that the Workflow appears to be recognizing the global conditions at this point as indicated by the 'TestSequential-global-conditions-eval' found in the log,
Adding condition: [Timeout Condition] to parent workflow: [TestSequential]
...
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow TestSequential retrieved for event urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [TestSequential-global-conditions-eval] has no required metadata fields
However, it appears to be stuck at this point as it never moves onto the tasks.
1. Since the timeout is at the task level it simply moves on and into the next task, which makes you think it's an error.
I did another test where I put the timeout condition under the task level (GoodbyeWorld task in this case) and the condition didn't appear to be evaluated at all and just executed the GoodbyeWorld task right away.
Is that what you are seeing?
If so, how would you think it should behave differently?
I think the Timeout feature should operate as it is described in OODT-207.
Since it appears that the timeout feature is not working, should I file a JIRA ticket?
Cheers,
Chris
Thanks,
Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Cayanan, Michael D (388J)" <mi...@jpl.nasa.gov>.
Hey Chris,
My answers are below in blue
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 11:00 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hi Mike,
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 5:54 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
Great! Please let me know what you find.
Will do.
Regarding the timeout feature, how does it work exactly? From the description in OODT-207, it appears that if a condition keeps failing for some configurable specified amount of time, then it will just return true and move on to the task?
You got it, that's what it does.
Or will it simply stop the workflow event all together due to continuous failing?
Nope it will only stop the failing on that task — is that the behavior you are seeing?
Ahhh — you may be seeing both behaviors. You are doing a global workflow condition timeout, right? So what you might be seeing with global workflow conditions are:
1. Global workflow condition adds a dynamic workflow condition ahead of the first task — you may be seeing (correctly) a timeout there, but;
So with the 'TestParallel' workflow defined in the example policy provided, I believe that global conditions aren't being evaluated at all since the 'HelloWorld', 'GoodbyeWorld' and 'Intensive' tasks all run to completion quite fast even though the timeout is set to '600'. Another test I did was modified the policy to have a sequential Workflow with global conditions like the following,
<sequential id="urn:oodt:TestSequential" name="TestSequential">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
</sequential>
What's interesting here is that the Workflow appears to be recognizing the global conditions at this point as indicated by the 'TestSequential-global-conditions-eval' found in the log,
Adding condition: [Timeout Condition] to parent workflow: [TestSequential]
...
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow TestSequential retrieved for event urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [TestSequential-global-conditions-eval] has no required metadata fields
However, it appears to be stuck at this point as it never moves onto the tasks.
1. Since the timeout is at the task level it simply moves on and into the next task, which makes you think it's an error.
I did another test where I put the timeout condition under the task level (GoodbyeWorld task in this case) and the condition didn't appear to be evaluated at all and just executed the GoodbyeWorld task right away.
Is that what you are seeing?
If so, how would you think it should behave differently?
I think the Timeout feature should operate as it is described in OODT-207.
Since it appears that the timeout feature is not working, should I file a JIRA ticket?
Cheers,
Chris
Thanks,
Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Cayanan, Michael D (388J)" <mi...@jpl.nasa.gov>.
Hey Chris,
My answers are below in blue
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 11:00 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hi Mike,
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 5:54 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
Great! Please let me know what you find.
Will do.
Regarding the timeout feature, how does it work exactly? From the description in OODT-207, it appears that if a condition keeps failing for some configurable specified amount of time, then it will just return true and move on to the task?
You got it, that's what it does.
Or will it simply stop the workflow event all together due to continuous failing?
Nope it will only stop the failing on that task — is that the behavior you are seeing?
Ahhh — you may be seeing both behaviors. You are doing a global workflow condition timeout, right? So what you might be seeing with global workflow conditions are:
1. Global workflow condition adds a dynamic workflow condition ahead of the first task — you may be seeing (correctly) a timeout there, but;
So with the 'TestParallel' workflow defined in the example policy provided, I believe that global conditions aren't being evaluated at all since the 'HelloWorld', 'GoodbyeWorld' and 'Intensive' tasks all run to completion quite fast even though the timeout is set to '600'. Another test I did was modified the policy to have a sequential Workflow with global conditions like the following,
<sequential id="urn:oodt:TestSequential" name="TestSequential">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
</sequential>
What's interesting here is that the Workflow appears to be recognizing the global conditions at this point as indicated by the 'TestSequential-global-conditions-eval' found in the log,
Adding condition: [Timeout Condition] to parent workflow: [TestSequential]
...
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow TestSequential retrieved for event urn:oodt:TestSequential
Jan 14, 2013 8:37:21 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [TestSequential-global-conditions-eval] has no required metadata fields
However, it appears to be stuck at this point as it never moves onto the tasks.
1. Since the timeout is at the task level it simply moves on and into the next task, which makes you think it's an error.
I did another test where I put the timeout condition under the task level (GoodbyeWorld task in this case) and the condition didn't appear to be evaluated at all and just executed the GoodbyeWorld task right away.
Is that what you are seeing?
If so, how would you think it should behave differently?
I think the Timeout feature should operate as it is described in OODT-207.
Since it appears that the timeout feature is not working, should I file a JIRA ticket?
Cheers,
Chris
Thanks,
Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Mike,
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 5:54 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
Great! Please let me know what you find.
Will do.
Regarding the timeout feature, how does it work exactly? From the description in OODT-207, it appears that if a condition keeps failing for some configurable specified amount of time, then it will just return true and move on to the task?
You got it, that's what it does.
Or will it simply stop the workflow event all together due to continuous failing?
Nope it will only stop the failing on that task — is that the behavior you are seeing?
Ahhh — you may be seeing both behaviors. You are doing a global workflow condition timeout, right? So what you might be seeing with global workflow conditions are:
1. Global workflow condition adds a dynamic workflow condition ahead of the first task — you may be seeing (correctly) a timeout there, but;
2. Since the timeout is at the task level it simply moves on and into the next task, which makes you think it's an error.
Is that what you are seeing? If so, how would you think it should behave differently?
Cheers,
Chris
Thanks,
Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Mike,
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Wednesday, January 9, 2013 5:54 PM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
Great! Please let me know what you find.
Will do.
Regarding the timeout feature, how does it work exactly? From the description in OODT-207, it appears that if a condition keeps failing for some configurable specified amount of time, then it will just return true and move on to the task?
You got it, that's what it does.
Or will it simply stop the workflow event all together due to continuous failing?
Nope it will only stop the failing on that task — is that the behavior you are seeing?
Ahhh — you may be seeing both behaviors. You are doing a global workflow condition timeout, right? So what you might be seeing with global workflow conditions are:
1. Global workflow condition adds a dynamic workflow condition ahead of the first task — you may be seeing (correctly) a timeout there, but;
2. Since the timeout is at the task level it simply moves on and into the next task, which makes you think it's an error.
Is that what you are seeing? If so, how would you think it should behave differently?
Cheers,
Chris
Thanks,
Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Cayanan, Michael D (388J)" <mi...@jpl.nasa.gov>.
Hey Chris,
Great! Please let me know what you find.
Regarding the timeout feature, how does it work exactly? From the description in OODT-207, it appears that if a condition keeps failing for some configurable specified amount of time, then it will just return true and move on to the task?
Or will it simply stop the workflow event all together due to continuous failing?
Thanks,
Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Cayanan, Michael D (388J)" <mi...@jpl.nasa.gov>.
Hey Chris,
Great! Please let me know what you find.
Regarding the timeout feature, how does it work exactly? From the description in OODT-207, it appears that if a condition keeps failing for some configurable specified amount of time, then it will just return true and move on to the task?
Or will it simply stop the workflow event all together due to continuous failing?
Thanks,
Mike
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris
Re: Workflow 2
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Thanks Mike, awesome. I'll look at the policy this week and try to track down the issue with Global Condition.
Thanks!
Cheers,
Chris
From: <Cayanan>, "Michael D (388J)" <mi...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Monday, January 7, 2013 8:24 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Chris,
My apologies for getting back to you so late on this. I was out on vacation. Hope you had an awesome holiday break.
My replies are in blue below....
From: <Mattmann>, Chris Mattmann <Ch...@jpl.nasa.gov>>
Reply-To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Date: Friday, December 21, 2012 10:38 AM
To: "user@oodt.apache.org<ma...@oodt.apache.org>" <us...@oodt.apache.org>>
Subject: Re: Workflow 2
Hey Mike,
I ran into a few issues that I was hoping you could help me out with:
1) The timeout condition doesn't appear to be working. I changed the timeout to '600' seconds instead of '20' to let the event run for a longer period of time. However, the event appears to run the same amount of time with the timeout=20 vs timeout=600.
Yep this isn't support in the PrioritizedQueueBasedWorkflowEngine yet. Try using the ThreadPoolWorkflowEngine, with the PackagedWorkflowRepository this should work.
I tried using the ThreadPoolWorkflowEngine instead and it still doesn't appear to be working. Still runs In fact, it doesn't seem to recognize the conditions that I'm setting at all. I modified my policy to where the 'HelloWorld' and 'GoodbyeWorld' task definitions each have a 'FalseCondition' now. By doing this, this means that the tasks would never run, correct? The TestParallel example is still executing the 'HelloWorld' and 'GoodByeWorld' tasks. I'm attaching my modified workflow2 policy to this e-mail.
Can you show the log output? From using the ThreadPoolWorkflowEngine? That should show us some output that shows that it loaded the condition.
Here is the log output using the ThreadPoolWorkflowEngine. It looks like the Timeout Condition gets added to the parent Workflow. However, as I've mentioned previously, the timeout doesn't appear to be working as I've set the timeout to 600 in my policy and the workflow runs pretty fast. Furthermore, the logs indicate that the 'HelloWorld' and 'GoodbyeWorld' tasks ran even though I added 'FalseConditions' to both of them:
% ./wmgr start
-n Starting cas workflow manager:
OK
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Jan 7, 2013 8:06:54 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Jan 7, 2013 8:06:55 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Jan 7, 2013 8:07:52 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Hello World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Hello World] locally
Hello World: Chris
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Processor Intensive Task] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Processor Intensive Task] locally
Task: Num seconds: 10
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread checkTaskRequiredMetadata
INFO: Task: [Goodbye World] has no required metadata fields
Jan 7, 2013 8:07:53 AM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
INFO: Executing task: [Goodbye World] locally
Goodbye World: Chris
Is it possible that the workflow policy isn't set up correctly? The policy was attached in an earlier e-mail to the user list. I'll re-paste it on here for completeness sake:
<cas:workflows xmlns="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:cas="http://oodt.jpl.nasa.gov/2.0/cas"
xmlns:p="http://oodt.jpl.nasa.gov/2.0/cas/property">
<parallel id="urn:oodt:TestParallel" name="TestParallel">
<sequential id="urn:oodt:SayHelloAndGoodBye" name="HelloGoodbye">
<task id-ref="urn:oodt:HelloWorld" />
<task id-ref="urn:oodt:GoodbyeWorld" />
</sequential>
<conditions>
<condition id-ref="urn:oodt:TimeoutCondition"/>
</conditions>
<task id-ref="urn:oodt:IntensiveTask" />
</parallel>
<task id="urn:oodt:HelloWorld" name="Hello World"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:GoodbyeWorld" name="Goodbye World"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:YoureStuck" name="You're Stuck"
class="org.apache.oodt.cas.workflow.examples.GoodbyeWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:IntensiveTask" name="Processor Intensive Task"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration/>
</task>
<task id="urn:oodt:IntensiveTaskWithWait" name="Intensive Waiter"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential">
<condition id-ref="urn:oodt:LongCondition" />
</conditions>
<configuration />
</task>
<task id="urn:oodt:PropReplaceTask" name="PRTask"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration>
<property name="PathToReplace" value="[HOME]/my/path"
envReplace="true" />
<property name="DontReplaceMe" value="[HOME]/my/path"
envReplace="false" />
<property name="DontReplaceMeNoSpec" value="[HOME]/my/path" />
</configuration>
</task>
<task id="urn:oodt:NumIncrementTask" name="Num Increment Task"
class="org.apache.oodt.cas.workflow.examples.NumIncrementTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:RandomStatusUpdateTask" name="Random Status Update"
class="org.apache.oodt.cas.workflow.examples.RandomStatusUpdateTask">
<conditions execution="sequential" />
<configuration />
</task>
<task id="urn:oodt:TestMultiConditionTask" name="Test Multi Condition"
class="org.apache.oodt.cas.workflow.examples.HelloWorld">
<conditions execution="sequential">
<condition id-ref="urn:oodt:TrueCondition" />
<condition id-ref="urn:oodt:FalseCondition" />
</conditions>
<configuration>
<property name="Person" value="Chris" />
</configuration>
</task>
<task id="urn:oodt:TestReqMetFieldsTask" name="Test Met Fields"
class="org.apache.oodt.cas.workflow.examples.LongTask">
<conditions execution="sequential" />
<configuration />
<requiredMetFields>
<metfield name="numSeconds" />
</requiredMetFields>
</task>
<task id="urn:oodt:ExternalScriptTask" name="External Script Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance">
<conditions execution="sequential" />
<configuration>
<property name="ShellType" value="/bin/sh" /> <!-- or /bin/tcsh if you have tcsh shell envionment; or /usr/bin/ if you
have a compiled program -->
<property name="PathToScript" value="/usr/local/sampleScript.sh" /> <!-- This should include the file name as well -->
</configuration>
</task>
<task id="urn:oodt:mailTask" name="Ingest e-Mail"
class="org.apache.oodt.cas.workflow.examples.MailTask">
<conditions execution="sequential" />
<configuration>
<property name="user.name" value="Costin" />
<property name="msg.subject" value="Ingest Notification" />
<property name="mail.to" value="chris.mattmann@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
<property name="mail.from" value="oodt@jpl.nasa.gov<ma...@jpl.nasa.gov>" />
</configuration>
</task>
<condition id="urn:oodt:TrueCondition" name="True Condition"
class="org.apache.oodt.cas.workflow.examples.TrueCondition" />
<condition id="urn:oodt:FalseCondition" name="False Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" />
<condition id="urn:oodt:LongCondition" name="Long Condition"
class="org.apache.oodt.cas.workflow.examples.LongCondition" />
<condition id="urn:oodt:CheckForMetadataKeys" name="Metadata Key Check"
class="org.apache.oodt.cas.workflow.examples.CheckForMetadataKeys">
<configuration>
<property name="reqMetKeys" value="Met1,Met2,Met3" />
</configuration>
</condition>
<condition id="urn:oodt:TimeoutCondition" name="Timeout Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" timeout="600"/>
<condition id="urn:oodt:OptionalCondition" name="Optional Condition"
class="org.apache.oodt.cas.workflow.examples.FalseCondition" optional="true"/>
</cas:workflows>
2) I'm trying to pass in a 'numSeconds' metadata to the workflow event so that the intensive task can run for 60 seconds instead of the default 10 seconds. But the log keeps indicating that the number of seconds is set at 10. This is how I'm triggering the event on the command-line:
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 60 -op -se
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow HelloGoodbye retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:34 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Parallel Single Task Processor Intensive Task retrieved for event urn:oodt:TestParallel
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Task: Num seconds: 10
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Hello World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Hello World] for instance id: [f06e0357-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.TaskQuerier run
INFO: Added processor with priority: [MEDIUM : 5.0]
Goodbye World: Chris
Dec 20, 2012 4:25:36 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Goodbye World] for instance id: [f0738198-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Dec 20, 2012 4:25:46 PM org.apache.oodt.cas.workflow.engine.runner.AsynchronousLocalEngineRunner$1 run
INFO: Task: [Processor Intensive Task] for instance id: [f0632de6-4b04-11e2-95a4-edbfb4ca9bb3] completed successfully
Yep it's not honoring these properties yet — but the ThreadPoolWorkflowEngine will. See latest status in OODT-491 [1].
Okay, with the ThreadPoolWorkflowEngine, it is recognizing the numSeconds metadata that I'm passing in.
+1, great.
3) When I try to look at the list of Workflow Instances that have ran or are running currently, I get an exception thrown:
% ./wmgr-client -u http://localhost:9001 -op -winsts
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getWorkflowInstances(XmlRpcWorkflowManagerClient.java:653)
at org.apache.oodt.cas.workflow.cli.action.GetWorkflowInstsCliAction.execute(GetWorkflowInstsCliAction.java:42)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get workflow instances from URL 'http://localhost:9001' : java.lang.Exception: org.apache.oodt.cas.workflow.structs.exceptions.EngineException: Exception getting workflow instances from workflow engine: Message: null
In the Workflow log, it shows the following exception:
Dec 20, 2012 4:26:52 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager getWorkflowInstances
INFO: Getting workflow instances: retrieved: 10 instances
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:394)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflow(XmlRpcStructFactory.java:227)
at org.apache.oodt.cas.workflow.util.XmlRpcStructFactory.getXmlRpcWorkflowInstance(XmlRpcStructFactory.java:114)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.getWorkflowInstances(XmlRpcWorkflowManager.java:448)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
I'm guessing I'm forgetting to set something in the policy. However, I'm not sure what is missing.
Nah you're fine. I think this is an issue with the Serialization/Deserialization of ParentChildWorkflows. Can you file a JIRA issue for this one and link it to OODT-491 [1]?
I filed a JIRA issue for this: https://issues.apache.org/jira/browse/OODT-549
Great work Mike!
A different way that works and that will give you the same result is to run:
./wmgr-client —url http://localhost:9001 —operation —getFirstPage
That worked for me. However, I found an interesting bug. Basically, once you restart the Workflow Manager Server and then try to run that 'getFirstPage' command, it throws an exception. Below shows the set of commands that you can run to reproduce the issue. Let me know if you think this should be a JIRA issue as well. On a side note, when I stopped the Workflow Manager Server, deleted the Lucene Workflow Instance Repository directory, then started the Worfklow Manager again, the error goes away.
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=0,pageSize=0,totalPages=0]
% ./wmgr-client -u http://localhost:9001 -en urn:oodt:TestParallel --metaData --key numSeconds 15 -op -se
Sending event 'urn:oodt:TestParallel'... SUCCESS
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
Page: [num=1,pageSize=20,totalPages=1]
Instance: [id=4e5a2d0c-4b47-11e2-977b-c39e66f8f1d5, status=STARTED, currentTask=urn:oodt:IntensiveTask, workflow=Parallel Single Task Processor Intensive Task, wallClockTime=0.09376666666666668, currentTaskWallClockTime=0.0]
Instance: [id=4e4e6d3b-4b47-11e2-977b-c39e66f8f1d5, status=FINISHED, currentTask=urn:oodt:GoodbyeWorld, workflow=HelloGoodbye, wallClockTime=0.0049, currentTaskWallClockTime=0.0]
% ./wmgr restart
-n Shutting down cas workflow manager:
OK
-n Starting cas workflow manager:
OK
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from: [../etc/workflow.properties]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs locally. URL: []
Adding condition: [Timeout Condition] to parent workflow: [TestParallel]
Dec 21, 2012 12:21:50 AM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by mcayanan
% ./wmgr-client -u http://localhost:9001 --operation --getFirstPage
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.getFirstPage(XmlRpcWorkflowManagerClient.java:122)
at org.apache.oodt.cas.workflow.cli.action.GetFirstPageCliAction.execute(GetFirstPageCliAction.java:47)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlRpcWorkflowManagerClient.java:678)
ERROR: Failed to get first page of workflows : java.lang.Exception: java.lang.NullPointerException
Ahh yes can you please file an issue for this? I believe something is up with the CLI here…
A JIRA issue was filed for this: https://issues.apache.org/jira/browse/OODT-550
Thanks,
Mike
Cheers,
Chris