You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by Christian Alan Mattmann <ma...@usc.edu> on 2014/11/06 04:07:22 UTC
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Hi Renu,
Can you please show a paste.bin of your tasks.xml?
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Adjunct Associate Professor, Computer Science Department
University of Southern California
Los Angeles, CA 90089 USA
Email: mattmann@usc.edu
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Renu Kanakamedala <ka...@usc.edu>
Date: Wednesday, November 5, 2014 at 7:55 PM
To: Chris Mattmann <ma...@usc.edu>
Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
>Hello Professor,
>
>
>When i am trying to run file concatenator CAS PGE example using the
>following command.
>
>
>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>
>
>
>The message shown after executing this command is "Sending Event
>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
>stub.
>
>
>Batch Stub:
>Caused by:
>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcept
>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>value: null
>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>... 2 more
>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>parse value: null
>at
>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCon
>figBuilder.java:87)
>at
>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.ja
>va:230)
>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>... 3 more
>Caused by: java.lang.Exception: Failed to parse value: null
>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>at
>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCon
>figBuilder.java:77)
>... 5 more
>Caused by: java.lang.NullPointerException
>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>at java.util.regex.Matcher.reset(Matcher.java:308)
>at java.util.regex.Matcher.<init>(Matcher.java:228)
>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>at
>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplaceme
>nt(PathUtils.java:321)
>at
>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils
>.java:96)
>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>... 7 more
>
>
>
>
>
>RES mngr:
>
>
>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>addJob
>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>Nov 05, 2014 5:26:30 PM
>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>Scheduling for execution
>Nov 05, 2014 5:26:30 PM
>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>plainConnect
>FINEST: ProxySelector Request for
>http://localhost:2001/ <http://localhost:2001/>
>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>plainConnect
>FINEST: Proxy used: DIRECT
>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>writeRequests
>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>text/html, image/gif, image/jpeg, *; q=.2, */*;
> q=.2}{Connection: keep-alive}{Content-Length: 105}
>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>getInputStream
>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>text/xml}{Content-Length: 121}
>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>plainConnect
>FINEST: ProxySelector Request for
>http://localhost:2001/ <http://localhost:2001/>
>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>plainConnect
>FINEST: Proxy used: DIRECT
>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>writeRequests
>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>text/html, image/gif, image/jpeg, *; q=.2, */*;
> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>getInputStream
>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>text/xml}{Content-Length: 121}
>Nov 05, 2014 5:26:32 PM
>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
>batchstub.executeJob returned false
>java.lang.Exception: batchstub.executeJob returned false
>at
>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatchM
>grProxy.java:125)
>
>
>
>
>
>Workflow:
>
>
>Nov 05, 2014 5:26:27 PM
>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>checkTaskRequiredMetadata
>INFO: All required metadata fields present for task: [FileConcatenator]
>Nov 05, 2014 5:26:28 PM
>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient submitJob
>FINEST: [{job.queueName=high,
>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>job.name <http://job.name>=urn:oodt:FileConcatenator,
>job.id <http://job.id>=, job.status=, job.load=2,
>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileCon
>catenatorPGETask,
>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/PGE
>Config.xml,
>
>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer.
>LocalDataTransferFactory,
>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xml,
> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>PCS_WorkflowManagerUrl=http://localhost:9001,
> PCS_FileManagerUrl=http://localhost:9000,
>PGETask_Name=FileConcatenator},
>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>WorkflowManagerUrl=[http://ubuntu:9001],
>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
> ProcessingNode=[ubuntu], RunID=[testNumber1],
>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>Nov 05, 2014 5:26:39 PM
>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread stop
>WARNING: Attempt to kill current resmgr job:
>[2014-11-05T17:26:28.785-08:00]: failed
>
>
>When i searched online i found one of your mail archives where you had
>mentioned to change tasks.xml
>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>
>
>
>It did not work. Please let us know how to proceed.
>
>
>
>Regards,
>Renu
>
>
>
>
>
>
>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Renu Kanakamedala <ka...@usc.edu>.
Sorry I missed paste.bin word in your email.Here is the pastebin link for
tasks.xml
http://pastebin.com/MAYDeYNH
Regards,
Renu
On Wed, Nov 5, 2014 at 7:18 PM, Renu Kanakamedala <ka...@usc.edu> wrote:
> Please find below tasks.xml file.
>
> <cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
> <!--
> TODO: Add some examples
> -->
> <task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>
> class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGETask">
>
> <conditions/>
>
> <configuration>
> <property name="PGETask_Name" value="FileConcatenator"/>
> <property name="PGETask_ConfigFilePath"
> value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
> envReplace="true"/>
> <property name="PGETask_DumpMetadata" value="true"/>
> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
> envReplace="true" />
> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
> envReplace="true"/>
> <property name="PCS_MetFileExtension" value="met"/>
> <property name="PCS_ClientTransferServiceFactory"
> value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory"/>
> <property name="PCS_ActionRepoFile"
> value="file:[CRAWLER_HOME]/policy/crawler-config.xml" envReplace="true"/>
> </configuration>
>
> <requiredMetFields>
> <metfield name="RunID"/>
> </requiredMetFields>
>
> </task>
> </cas:tasks>
>
> On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann <ma...@usc.edu>
> wrote:
>
>> Hi Renu,
>>
>> Can you please show a paste.bin of your tasks.xml?
>>
>> Cheers,
>> Chris
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Adjunct Associate Professor, Computer Science Department
>> University of Southern California
>> Los Angeles, CA 90089 USA
>> Email: mattmann@usc.edu
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>> -----Original Message-----
>> From: Renu Kanakamedala <ka...@usc.edu>
>> Date: Wednesday, November 5, 2014 at 7:55 PM
>> To: Chris Mattmann <ma...@usc.edu>
>> Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
>> null Error.
>>
>> >Hello Professor,
>> >
>> >
>> >When i am trying to run file concatenator CAS PGE example using the
>> >following command.
>> >
>> >
>> >./wmgr-client --url http://localhost:9001 --operation --sendEvent
>> >--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>> >
>> >
>> >
>> >The message shown after executing this command is "Sending Event
>> >SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
>> >stub.
>> >
>> >
>> >Batch Stub:
>> >Caused by:
>>
>> >org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcept
>> >ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>> >value: null
>> >at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>> >at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>> >... 2 more
>> >Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>> >parse value: null
>> >at
>>
>> >org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCon
>> >figBuilder.java:87)
>> >at
>>
>> >org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.ja
>> >va:230)
>> >at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>> >... 3 more
>> >Caused by: java.lang.Exception: Failed to parse value: null
>> >at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> >at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> >at
>>
>> >org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCon
>> >figBuilder.java:77)
>> >... 5 more
>> >Caused by: java.lang.NullPointerException
>> >at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>> >at java.util.regex.Matcher.reset(Matcher.java:308)
>> >at java.util.regex.Matcher.<init>(Matcher.java:228)
>> >at java.util.regex.Pattern.matcher(Pattern.java:1088)
>> >at
>>
>> >org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplaceme
>> >nt(PathUtils.java:321)
>> >at
>>
>> >org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils
>> >.java:96)
>> >at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>> >... 7 more
>> >
>> >
>> >
>> >
>> >
>> >RES mngr:
>> >
>> >
>> >Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>> >addJob
>> >INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>> >Nov 05, 2014 5:26:30 PM
>> >org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>> >INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>> >Scheduling for execution
>> >Nov 05, 2014 5:26:30 PM
>> >org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>> >INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >plainConnect
>> >FINEST: ProxySelector Request for
>> >http://localhost:2001/ <http://localhost:2001/>
>> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >plainConnect
>> >FINEST: Proxy used: DIRECT
>> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >writeRequests
>> >FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>> >null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>> >no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>> >text/html, image/gif, image/jpeg, *; q=.2, */*;
>> > q=.2}{Connection: keep-alive}{Content-Length: 105}
>> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >getInputStream
>> >FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>> >OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>> >text/xml}{Content-Length: 121}
>> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >plainConnect
>> >FINEST: ProxySelector Request for
>> >http://localhost:2001/ <http://localhost:2001/>
>> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >plainConnect
>> >FINEST: Proxy used: DIRECT
>> >Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>> >writeRequests
>> >FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>> >null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>> >no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>> >text/html, image/gif, image/jpeg, *; q=.2, */*;
>> > q=.2}{Connection: keep-alive}{Content-Length: 2536}
>> >Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>> >getInputStream
>> >FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>> >OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>> >text/xml}{Content-Length: 121}
>> >Nov 05, 2014 5:26:32 PM
>> >org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>> >SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
>> >batchstub.executeJob returned false
>> >java.lang.Exception: batchstub.executeJob returned false
>> >at
>>
>> >org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatchM
>> >grProxy.java:125)
>> >
>> >
>> >
>> >
>> >
>> >Workflow:
>> >
>> >
>> >Nov 05, 2014 5:26:27 PM
>> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> >checkTaskRequiredMetadata
>> >INFO: All required metadata fields present for task: [FileConcatenator]
>> >Nov 05, 2014 5:26:28 PM
>> >org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient submitJob
>> >FINEST: [{job.queueName=high,
>> >job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>> >job.name <http://job.name>=urn:oodt:FileConcatenator,
>> >job.id <http://job.id>=, job.status=, job.load=2,
>> >job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>>
>> >{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileCon
>> >catenatorPGETask,
>>
>> >task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/PGE
>> >Config.xml,
>> >
>>
>> >PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer.
>> >LocalDataTransferFactory,
>>
>> >PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xml,
>> > PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>> >PCS_WorkflowManagerUrl=http://localhost:9001,
>> > PCS_FileManagerUrl=http://localhost:9000,
>> >PGETask_Name=FileConcatenator},
>> >task.metadata={TaskId=[urn:oodt:FileConcatenator],
>> >WorkflowManagerUrl=[http://ubuntu:9001],
>> >JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>> > ProcessingNode=[ubuntu], RunID=[testNumber1],
>> >WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>> >Nov 05, 2014 5:26:39 PM
>> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread stop
>> >WARNING: Attempt to kill current resmgr job:
>> >[2014-11-05T17:26:28.785-08:00]: failed
>> >
>> >
>> >When i searched online i found one of your mail archives where you had
>> >mentioned to change tasks.xml
>> >https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>> >
>> >
>> >
>> >It did not work. Please let us know how to proceed.
>> >
>> >
>> >
>> >Regards,
>> >Renu
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Chris Mattmann <ch...@gmail.com>.
Thanks Renu, put those lines in the batch stub since it¹s
the Java process actually executing cas-pge - should fix it.
If it does, can you open up a issue on Github and submit a
pull request?
------------------------
Chris Mattmann
chris.mattmann@gmail.com
-----Original Message-----
From: Renu Kanakamedala <ka...@usc.edu>
Reply-To: <de...@oodt.apache.org>
Date: Wednesday, November 5, 2014 at 11:01 PM
To: Chris Mattmann <ma...@usc.edu>
Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
<de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
Kondakindi <ko...@usc.edu>
Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
parse null Error.
>Sorry Professor, the file I had sent previously is resmgr script. But that
>seems like the right file to add the two lines you mentioned. I restarted
>the res mgr after adding the lines, still getting the same exception.
>
>Here is the link for batch_stub
>http://pastebin.com/tfRCtJj0
>
>Thanks!
>
>- Renu
>
>
>
>
>On Wed, Nov 5, 2014 at 10:01 PM, Christian Alan Mattmann
><ma...@usc.edu>
>wrote:
>
>> Can you add lines:
>>
>> -Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \
>> -Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \
>>
>> Starting at line 92 in your batch_stub script, restart
>> the batch stub and see if that fixes it?
>>
>> Cheers,
>> Chris
>>
>>
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Adjunct Associate Professor, Computer Science Department
>> University of Southern California
>> Los Angeles, CA 90089 USA
>> Email: mattmann@usc.edu
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>> -----Original Message-----
>> From: Renu Kanakamedala <ka...@usc.edu>
>> Date: Wednesday, November 5, 2014 at 10:51 PM
>> To: Chris Mattmann <ma...@usc.edu>
>> Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
>> <de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
>> Kondakindi <ko...@usc.edu>
>> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>> parse null Error.
>>
>> >wmgr script
>> >
>> >http://pastebin.com/zE7PWnXL
>> >
>> >
>> >resmgr batch_stub
>> >
>> >
>> >http://pastebin.com/gPUvbE9e
>> >
>> >
>> >- Renu
>> >
>> >
>> >
>> >
>> >
>> >On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann
>> ><ma...@usc.edu> wrote:
>> >
>> >Great. One last question.
>> >
>> >Can you show me the contents (paste.bin) of your wmgr
>> >script and your batch_stub script?
>> >
>> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >Chris Mattmann, Ph.D.
>> >Adjunct Associate Professor, Computer Science Department
>> >University of Southern California
>> >Los Angeles, CA 90089 USA
>> >Email: mattmann@usc.edu
>> >WWW: http://sunset.usc.edu/~mattmann/
>> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >
>> >
>> >
>> >
>> >-----Original Message-----
>> >From: Renu Kanakamedala <ka...@usc.edu>
>> >Date: Wednesday, November 5, 2014 at 10:20 PM
>> >To: Chris Mattmann <ma...@usc.edu>
>> >Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
>> ><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
>> >Kondakindi <ko...@usc.edu>
>> >Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>> >parse null Error.
>> >
>> >>renu@ubuntu:~$ echo $WORKFLOW_HOME
>> >>/home/renu/oodt/workflow/
>> >>renu@ubuntu:~$ echo $FILEMGR_HOME
>> >>/home/renu/oodt/filemgr
>> >>renu@ubuntu:~$ echo $WORKFLOW_URL
>> >>http://localhost:9001
>> >>renu@ubuntu:~$ echo $FILEMGR_URL
>> >>http://localhost:9000
>> >>renu@ubuntu:~$ echo $PGE_ROOT
>> >>/home/renu/oodt/pge/
>> >>
>> >>renu@ubuntu:~$ echo $CRAWLER_HOME
>> >>/home/renu/oodt/crawler/
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann
>> >><ma...@usc.edu> wrote:
>> >>
>> >>Right, which is why I asked to specifically echo the value
>> >>of all environment variables so we would be able to see the
>> >>values. Can Renu please show the echo values?
>> >>
>> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >>Chris Mattmann, Ph.D.
>> >>Adjunct Associate Professor, Computer Science Department
>> >>University of Southern California
>> >>Los Angeles, CA 90089 USA
>> >>Email: mattmann@usc.edu
>> >>WWW: http://sunset.usc.edu/~mattmann/
>> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >>
>> >>
>> >>
>> >>
>> >>-----Original Message-----
>> >>From: Meghamsh Utkur <ut...@usc.edu>
>> >>Date: Wednesday, November 5, 2014 at 9:35 PM
>> >>To: Renu Kanakamedala <ka...@usc.edu>
>> >>Cc: Chris Mattmann <ma...@usc.edu>, "dev@oodt.apache.org"
>> >><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
>> >>Kondakindi <ko...@usc.edu>
>> >>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed
>>to
>> >>parse null Error.
>> >>
>> >>>Professor,
>> >>>
>> >>>
>> >>>To add to what Renu has said, I went through the source code and I
>> >>>suspect that the value here in this method is passed as null.
>> >>>
>> >>>XmlFilePgeConfigBuilder.java
>> >>>
>> >>>1. private String fillIn(String value,Metadata inputMetadata,boolean
>> >>>envReplaceRecur)
>> >>>2. throwsException {
>> >>>3. try{
>> >>>4. while((value =PathUtils.doDynamicReplacement(value,
>> >>>inputMetadata)).contains("[")&& envReplaceRecur);
>> >>>5. if(value.toUpperCase()
>> >>>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
>> >>>7. value
>> >>>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
>> >>>URL(inputMetadata
>> >>>8.
>>
>>>>>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParse
>>>>>r.
>> >>>p
>> >>>a
>> >>>rseSqlQueryMethod(value)));
>> >>>9. return value;
>> >>>10. }catch(Exception e){
>> >>>11. thrownew Exception("Failed to parse value: "+ value, e);
>> >>>12. }
>> >>>13. }
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
>> >>><ka...@usc.edu> wrote:
>> >>>
>> >>>Hello Professor,
>> >>>
>> >>>
>> >>>All the environment variables present in tasks.xml i.e.
>> >>>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
>> >>>
>> >>>
>> >>>Please let us know how to proceed.
>> >>>
>> >>>
>> >>>Regards,
>> >>>Renu
>> >>>
>> >>>
>> >>>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
>> >>><ma...@usc.edu> wrote:
>> >>>
>> >>>Can you please confirm that all of the bracketed environment
>> >>>variables below do in fact exist in the shell that you are
>> >>>running ./oodt start in?
>> >>>
>> >>>E.g., please echo $ENV_VAR below and show their values.
>> >>>
>> >>>Cheers,
>> >>>Chris
>> >>>
>> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >>>Chris Mattmann, Ph.D.
>> >>>Adjunct Associate Professor, Computer Science Department
>> >>>University of Southern California
>> >>>Los Angeles, CA 90089 USA
>> >>>Email: mattmann@usc.edu
>> >>>WWW: http://sunset.usc.edu/~mattmann/
>> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>-----Original Message-----
>> >>>From: Renu Kanakamedala <ka...@usc.edu>
>> >>>Date: Wednesday, November 5, 2014 at 8:18 PM
>> >>>To: Chris Mattmann <ma...@usc.edu>
>> >>>Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
>> >>><pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
>> >>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed
>>to
>> >>>parse null Error.
>> >>>
>> >>>>Please find below tasks.xml file.
>> >>>>
>> >>>>
>> >>>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
>> >>>><!--
>> >>>> TODO: Add some examples
>> >>>>-->
>> >>>><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>> >>>>
>>
>>>>>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenator
>>>>>>PG
>> >>>>E
>> >>>>T
>> >>>>a
>> >>>>sk">
>> >>>>
>> >>>> <conditions/>
>> >>>>
>> >>>> <configuration>
>> >>>> <property name="PGETask_Name" value="FileConcatenator"/>
>> >>>> <property name="PGETask_ConfigFilePath"
>> >>>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>> >>>>envReplace="true"/>
>> >>>> <property name="PGETask_DumpMetadata" value="true"/>
>> >>>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>> >>>>envReplace="true" />
>> >>>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
>> >>>>envReplace="true"/>
>> >>>> <property name="PCS_MetFileExtension" value="met"/>
>> >>>> <property name="PCS_ClientTransferServiceFactory"
>>
>>>>>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFact
>>>>>>or
>> >>>>y
>> >>>>"
>> >>>>/
>> >>>>>
>> >>>> <property name="PCS_ActionRepoFile"
>> >>>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml"
>> >>>>envReplace="true"/>
>> >>>> </configuration>
>> >>>>
>> >>>> <requiredMetFields>
>> >>>> <metfield name="RunID"/>
>> >>>> </requiredMetFields>
>> >>>>
>> >>>></task>
>> >>>></cas:tasks>
>> >>>>
>> >>>>
>> >>>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
>> >>>><ma...@usc.edu> wrote:
>> >>>>
>> >>>>Hi Renu,
>> >>>>
>> >>>>Can you please show a paste.bin of your tasks.xml?
>> >>>>
>> >>>>Cheers,
>> >>>>Chris
>> >>>>
>> >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >>>>Chris Mattmann, Ph.D.
>> >>>>Adjunct Associate Professor, Computer Science Department
>> >>>>University of Southern California
>> >>>>Los Angeles, CA 90089 USA
>> >>>>Email: mattmann@usc.edu
>> >>>>WWW: http://sunset.usc.edu/~mattmann/
>> >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>-----Original Message-----
>> >>>>From: Renu Kanakamedala <ka...@usc.edu>
>> >>>>Date: Wednesday, November 5, 2014 at 7:55 PM
>> >>>>To: Chris Mattmann <ma...@usc.edu>
>> >>>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>> >>>>parse
>> >>>>null Error.
>> >>>>
>> >>>>>Hello Professor,
>> >>>
>> >>>>>
>> >>>>>
>> >>>>>When i am trying to run file concatenator CAS PGE example using the
>> >>>>>following command.
>> >>>>>
>> >>>>>
>> >>>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>> >>>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>The message shown after executing this command is "Sending Event
>> >>>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and
>> >>>>>batch
>> >>>>>stub.
>> >>>>>
>> >>>>>
>> >>>>>Batch Stub:
>> >>>>>Caused by:
>>
>>>>>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstance
>>>>>>>Ex
>> >>>>>c
>> >>>>>e
>> >>>>>p
>> >>>>>t
>> >>>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to
>>parse
>> >>>>>value: null
>> >>>>>at
>>
>>>>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150
>>>>>>>)
>> >>>>>at
>>
>>>>>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:10
>>>>>>>2)
>> >>>>>... 2 more
>> >>>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed
>>to
>> >>>>>parse value: null
>> >>>>>at
>>
>>>>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFile
>>>>>>>Pg
>> >>>>>e
>> >>>>>C
>> >>>>>o
>> >>>>>n
>> >>>>>figBuilder.java:87)
>> >>>>>at
>>
>>>>>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInsta
>>>>>>>nc
>> >>>>>e
>> >>>>>.
>> >>>>>j
>> >>>>>a
>> >>>>>va:230)
>> >>>>>at
>>
>>>>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123
>>>>>>>)
>> >>>>>... 3 more
>> >>>>>Caused by: java.lang.Exception: Failed to parse value: null
>> >>>>>at
>>org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> >>>>>at
>>org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> >>>>>at
>>
>>>>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFile
>>>>>>>Pg
>> >>>>>e
>> >>>>>C
>> >>>>>o
>> >>>>>n
>> >>>>>figBuilder.java:77)
>> >>>>>... 5 more
>> >>>>>Caused by: java.lang.NullPointerException
>> >>>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>> >>>>>at java.util.regex.Matcher.reset(Matcher.java:308)
>> >>>>>at java.util.regex.Matcher.<init>(Matcher.java:228)
>> >>>>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>> >>>>>at
>>
>>>>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRep
>>>>>>>la
>> >>>>>c
>> >>>>>e
>> >>>>>m
>> >>>>>e
>> >>>>>nt(PathUtils.java:321)
>> >>>>>at
>>
>>>>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(Pat
>>>>>>>hU
>> >>>>>t
>> >>>>>i
>> >>>>>l
>> >>>>>s
>> >>>>>.java:96)
>> >>>>>at
>>org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>> >>>>>... 7 more
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>RES mngr:
>> >>>>>
>> >>>>>
>> >>>>>Nov 05, 2014 5:26:28 PM
>>org.apache.oodt.cas.resource.jobqueue.JobStack
>> >>>>>addJob
>> >>>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>> >>>>>Nov 05, 2014 5:26:30 PM
>> >>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>> >>>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>> >>>>>Scheduling for execution
>> >>>>>Nov 05, 2014 5:26:30 PM
>> >>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>> >>>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node:
>>[localhost]
>> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>>>>plainConnect
>> >>>>>FINEST: ProxySelector Request for
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>>>>http://localhost:2001/ <http://localhost:2001/>
>> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>>>>plainConnect
>> >>>>>FINEST: Proxy used: DIRECT
>> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>>>>writeRequests
>> >>>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>> >>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>> >>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>> >>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>> >>>>> q=.2}{Connection: keep-alive}{Content-Length: 105}
>> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>>>>getInputStream
>> >>>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>> >>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>> >>>>>text/xml}{Content-Length: 121}
>> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>>>>plainConnect
>> >>>>>FINEST: ProxySelector Request for
>> >>>>>http://localhost:2001/ <http://localhost:2001/>
>> >>>
>> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>>>>plainConnect
>> >>>>>FINEST: Proxy used: DIRECT
>> >>>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>> >>>>>writeRequests
>> >>>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>> >>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>> >>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>> >>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>> >>>>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>> >>>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>> >>>>>getInputStream
>> >>>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1
>>200
>> >>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>> >>>>>text/xml}{Content-Length: 121}
>> >>>>>Nov 05, 2014 5:26:32 PM
>> >>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>> >>>>>SEVERE: Job execution failed for jobId
>>'2014-11-05T17:26:28.785-08:00'
>> >>>>>:
>> >>>>>batchstub.executeJob returned false
>> >>>>>java.lang.Exception: batchstub.executeJob returned false
>> >>>>>at
>>
>>>>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpc
>>>>>>>Ba
>> >>>>>t
>> >>>>>c
>> >>>>>h
>> >>>>>M
>> >>>>>grProxy.java:125)
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>Workflow:
>> >>>>>
>> >>>>>
>> >>>>>Nov 05, 2014 5:26:27 PM
>>
>>>>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> >>>>>checkTaskRequiredMetadata
>> >>>>>INFO: All required metadata fields present for task:
>> >>>>>[FileConcatenator]
>> >>>>>Nov 05, 2014 5:26:28 PM
>> >>>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>> >>>>>submitJob
>> >>>>>FINEST: [{job.queueName=high,
>> >>>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>>>job.name <http://job.name> <http://job.name> <http://job.name>
>> >>>>><http://job.name>
>> >>>>><http://job.name>=urn:oodt:FileConcatenator,
>> >>
>> >>
>> >>>>>job.id <http://job.id> <http://job.id> <http://job.id>
>><http://job.id
>> >
>> >>>>><http://job.id>=, job.status=,
>> >>>>>job.load=2,
>>
>>>>>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput
>>>>>>>},
>>
>>>>>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.F
>>>>>>>il
>> >>>>>e
>> >>>>>C
>> >>>>>o
>> >>>>>n
>> >>>>>catenatorPGETask,
>>
>>>>>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-confi
>>>>>>>gs
>> >>>>>/
>> >>>>>P
>> >>>>>G
>> >>>>>E
>> >>>>>Config.xml,
>> >>>>>
>>
>>>>>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatra
>>>>>>>ns
>> >>>>>f
>> >>>>>e
>> >>>>>r
>> >>>>>.
>> >>>>>LocalDataTransferFactory,
>>
>>>>>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-confi
>>>>>>>g.
>> >>>>>x
>> >>>>>m
>> >>>>>l
>> >>>>>,
>> >>>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>> >>>>>PCS_WorkflowManagerUrl=http://localhost:9001,
>> >>>>> PCS_FileManagerUrl=http://localhost:9000,
>> >>>>>PGETask_Name=FileConcatenator},
>> >>>>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>> >>>>>WorkflowManagerUrl=[http://ubuntu:9001],
>> >>>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>> >>>>> ProcessingNode=[ubuntu], RunID=[testNumber1],
>> >>>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>> >>>>>Nov 05, 2014 5:26:39 PM
>>
>>>>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> >>>>>stop
>> >>>>>WARNING: Attempt to kill current resmgr job:
>> >>>>>[2014-11-05T17:26:28.785-08:00]: failed
>> >>>>>
>> >>>>>
>> >>>>>When i searched online i found one of your mail archives where you
>>had
>> >>>>>mentioned to change tasks.xml
>> >>>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>It did not work. Please let us know how to proceed.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>Regards,
>> >>>>>Renu
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>--
>> >>
>> >>
>> >>>Meghamsh UtkurGraduate Student,University of Southern California
>> >>>utkur@usc.edu
>> >>>
>> >
>> >
>> >>>(503)881-4791 <tel:%28503%29881-4791> <tel:%28503%29881-4791>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Christian Alan Mattmann <ma...@usc.edu>.
Hi Meghamsh, batch_stub is the one that made the difference.
Congrats!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Adjunct Associate Professor, Computer Science Department
University of Southern California
Los Angeles, CA 90089 USA
Email: mattmann@usc.edu
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Meghamsh Utkur <ut...@usc.edu>
Date: Thursday, November 6, 2014 at 12:39 AM
To: Renu Kanakamedala <ka...@usc.edu>
Cc: Chris Mattmann <ma...@usc.edu>, "dev@oodt.apache.org"
<de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
Kondakindi <ko...@usc.edu>
Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
parse null Error.
>Hello Professor,
>I have finally made the Pge workflow work successfully. As pointed out by
>you I have made the following changes.But I'm not sure which one of them
>actually solved the issue.
>
>
>I have added the following lines :
>
>
>-Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \
>-Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \
>
>
>
>To
>
>
>1)oodt/resmgr/bin/batch_stub
>
>2)oodt/resmgr/bin/resmgr
>3)oodt/workflow/bin/wmgr
>4)oodt/filemgr/bin/filemgr
>
>
>Note: Just adding them to the file batch_stub or resmgr din't actually
>make any difference .
>
>
>Thanks and Regards
>Meghamsh Utkur.
>
>
>On Wed, Nov 5, 2014 at 11:01 PM, Renu Kanakamedala
><ka...@usc.edu> wrote:
>
>Sorry Professor, the file I had sent previously is resmgr script. But
>that seems like the right file to add the two lines you mentioned. I
>restarted the res mgr after adding the lines, still getting the same
>exception.
>
>
>Here is the link for batch_stub
>http://pastebin.com/tfRCtJj0
>
>
>Thanks!
>
>
>- Renu
>
>
>
>
>
>
>
>
>On Wed, Nov 5, 2014 at 10:01 PM, Christian Alan Mattmann
><ma...@usc.edu> wrote:
>
>Can you add lines:
>
>-Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \
>-Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \
>
>Starting at line 92 in your batch_stub script, restart
>the batch stub and see if that fixes it?
>
>Cheers,
>Chris
>
>
>
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>Chris Mattmann, Ph.D.
>Adjunct Associate Professor, Computer Science Department
>University of Southern California
>Los Angeles, CA 90089 USA
>Email: mattmann@usc.edu
>WWW: http://sunset.usc.edu/~mattmann/
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>-----Original Message-----
>From: Renu Kanakamedala <ka...@usc.edu>
>Date: Wednesday, November 5, 2014 at 10:51 PM
>To: Chris Mattmann <ma...@usc.edu>
>Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
>Kondakindi <ko...@usc.edu>
>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>parse null Error.
>
>>wmgr script
>>
>>http://pastebin.com/zE7PWnXL
>>
>>
>>resmgr batch_stub
>>
>>
>>http://pastebin.com/gPUvbE9e
>>
>>
>>- Renu
>>
>>
>>
>>
>>
>>On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann
>><ma...@usc.edu> wrote:
>>
>>Great. One last question.
>>
>>Can you show me the contents (paste.bin) of your wmgr
>>script and your batch_stub script?
>>
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>Chris Mattmann, Ph.D.
>>Adjunct Associate Professor, Computer Science Department
>>University of Southern California
>>Los Angeles, CA 90089 USA
>>Email: mattmann@usc.edu
>>WWW: http://sunset.usc.edu/~mattmann/
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>-----Original Message-----
>>From: Renu Kanakamedala <ka...@usc.edu>
>>Date: Wednesday, November 5, 2014 at 10:20 PM
>>To: Chris Mattmann <ma...@usc.edu>
>>Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
>><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
>>Kondakindi <ko...@usc.edu>
>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>parse null Error.
>>
>>>renu@ubuntu:~$ echo $WORKFLOW_HOME
>>>/home/renu/oodt/workflow/
>>>renu@ubuntu:~$ echo $FILEMGR_HOME
>>>/home/renu/oodt/filemgr
>>>renu@ubuntu:~$ echo $WORKFLOW_URL
>>>http://localhost:9001
>>>renu@ubuntu:~$ echo $FILEMGR_URL
>>>http://localhost:9000
>>>renu@ubuntu:~$ echo $PGE_ROOT
>>>/home/renu/oodt/pge/
>>>
>>>renu@ubuntu:~$ echo $CRAWLER_HOME
>>>/home/renu/oodt/crawler/
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann
>>><ma...@usc.edu> wrote:
>>>
>>>Right, which is why I asked to specifically echo the value
>>>of all environment variables so we would be able to see the
>>>values. Can Renu please show the echo values?
>>>
>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>Chris Mattmann, Ph.D.
>>>Adjunct Associate Professor, Computer Science Department
>>>University of Southern California
>>>Los Angeles, CA 90089 USA
>>>Email: mattmann@usc.edu
>>>WWW: http://sunset.usc.edu/~mattmann/
>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>>
>>>
>>>
>>>-----Original Message-----
>>>From: Meghamsh Utkur <ut...@usc.edu>
>>>Date: Wednesday, November 5, 2014 at 9:35 PM
>>>To: Renu Kanakamedala <ka...@usc.edu>
>>>Cc: Chris Mattmann <ma...@usc.edu>, "dev@oodt.apache.org"
>>><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
>>>Kondakindi <ko...@usc.edu>
>>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>>parse null Error.
>>>
>>>>Professor,
>>>>
>>>>
>>>>To add to what Renu has said, I went through the source code and I
>>>>suspect that the value here in this method is passed as null.
>>>>
>>>>XmlFilePgeConfigBuilder.java
>>>>
>>>>1. private String fillIn(String value,Metadata inputMetadata,boolean
>>>>envReplaceRecur)
>>>>2. throwsException {
>>>>3. try{
>>>>4. while((value =PathUtils.doDynamicReplacement(value,
>>>>inputMetadata)).contains("[")&& envReplaceRecur);
>>>>5. if(value.toUpperCase()
>>>>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
>>>>7. value
>>>>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
>>>>URL(inputMetadata
>>>>8.
>>>>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser
>>>>.
>>>>p
>>>>a
>>>>rseSqlQueryMethod(value)));
>>>>9. return value;
>>>>10. }catch(Exception e){
>>>>11. thrownew Exception("Failed to parse value: "+ value, e);
>>>>12. }
>>>>13. }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
>>>><ka...@usc.edu> wrote:
>>>>
>>>>Hello Professor,
>>>>
>>>>
>>>>All the environment variables present in tasks.xml i.e.
>>>>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
>>>>
>>>>
>>>>Please let us know how to proceed.
>>>>
>>>>
>>>>Regards,
>>>>Renu
>>>>
>>>>
>>>>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
>>>><ma...@usc.edu> wrote:
>>>>
>>>>Can you please confirm that all of the bracketed environment
>>>>variables below do in fact exist in the shell that you are
>>>>running ./oodt start in?
>>>>
>>>>E.g., please echo $ENV_VAR below and show their values.
>>>>
>>>>Cheers,
>>>>Chris
>>>>
>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>Chris Mattmann, Ph.D.
>>>>Adjunct Associate Professor, Computer Science Department
>>>>University of Southern California
>>>>Los Angeles, CA 90089 USA
>>>>Email: mattmann@usc.edu
>>>>WWW: http://sunset.usc.edu/~mattmann/
>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>
>>>>
>>>>
>>>>
>>>>-----Original Message-----
>>>>From: Renu Kanakamedala <ka...@usc.edu>
>>>>Date: Wednesday, November 5, 2014 at 8:18 PM
>>>>To: Chris Mattmann <ma...@usc.edu>
>>>>Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
>>>><pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
>>>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>>>parse null Error.
>>>>
>>>>>Please find below tasks.xml file.
>>>>>
>>>>>
>>>>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
>>>>><!--
>>>>> TODO: Add some examples
>>>>>-->
>>>>><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>>>>>
>>>>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorP
>>>>>G
>>>>>E
>>>>>T
>>>>>a
>>>>>sk">
>>>>>
>>>>> <conditions/>
>>>>>
>>>>> <configuration>
>>>>> <property name="PGETask_Name" value="FileConcatenator"/>
>>>>> <property name="PGETask_ConfigFilePath"
>>>>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>>>>>envReplace="true"/>
>>>>> <property name="PGETask_DumpMetadata" value="true"/>
>>>>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>>>>>envReplace="true" />
>>>>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
>>>>>envReplace="true"/>
>>>>> <property name="PCS_MetFileExtension" value="met"/>
>>>>> <property name="PCS_ClientTransferServiceFactory"
>>>>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFacto
>>>>>r
>>>>>y
>>>>>"
>>>>>/
>>>>>>
>>>>> <property name="PCS_ActionRepoFile"
>>>>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml"
>>>>>envReplace="true"/>
>>>>> </configuration>
>>>>>
>>>>> <requiredMetFields>
>>>>> <metfield name="RunID"/>
>>>>> </requiredMetFields>
>>>>>
>>>>></task>
>>>>></cas:tasks>
>>>>>
>>>>>
>>>>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
>>>>><ma...@usc.edu> wrote:
>>>>>
>>>>>Hi Renu,
>>>>>
>>>>>Can you please show a paste.bin of your tasks.xml?
>>>>>
>>>>>Cheers,
>>>>>Chris
>>>>>
>>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>Chris Mattmann, Ph.D.
>>>>>Adjunct Associate Professor, Computer Science Department
>>>>>University of Southern California
>>>>>Los Angeles, CA 90089 USA
>>>>>Email: mattmann@usc.edu
>>>>>WWW: http://sunset.usc.edu/~mattmann/
>>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>-----Original Message-----
>>>>>From: Renu Kanakamedala <ka...@usc.edu>
>>>>>Date: Wednesday, November 5, 2014 at 7:55 PM
>>>>>To: Chris Mattmann <ma...@usc.edu>
>>>>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>>>>parse
>>>>>null Error.
>>>>>
>>>>>>Hello Professor,
>>>>
>>>>>>
>>>>>>
>>>>>>When i am trying to run file concatenator CAS PGE example using the
>>>>>>following command.
>>>>>>
>>>>>>
>>>>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>>>>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>>>>>>
>>>>>>
>>>>>>
>>>>>>The message shown after executing this command is "Sending Event
>>>>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and
>>>>>>batch
>>>>>>stub.
>>>>>>
>>>>>>
>>>>>>Batch Stub:
>>>>>>Caused by:
>>>>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceE
>>>>>>x
>>>>>>c
>>>>>>e
>>>>>>p
>>>>>>t
>>>>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>>>>>>value: null
>>>>>>at
>>>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>>>>>>at
>>>>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102
>>>>>>)
>>>>>>... 2 more
>>>>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>>>>>>parse value: null
>>>>>>at
>>>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFileP
>>>>>>g
>>>>>>e
>>>>>>C
>>>>>>o
>>>>>>n
>>>>>>figBuilder.java:87)
>>>>>>at
>>>>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstan
>>>>>>c
>>>>>>e
>>>>>>.
>>>>>>j
>>>>>>a
>>>>>>va:230)
>>>>>>at
>>>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>>>>>>... 3 more
>>>>>>Caused by: java.lang.Exception: Failed to parse value: null
>>>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>>>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>>>>>>at
>>>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFileP
>>>>>>g
>>>>>>e
>>>>>>C
>>>>>>o
>>>>>>n
>>>>>>figBuilder.java:77)
>>>>>>... 5 more
>>>>>>Caused by: java.lang.NullPointerException
>>>>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>>>>>>at java.util.regex.Matcher.reset(Matcher.java:308)
>>>>>>at java.util.regex.Matcher.<init>(Matcher.java:228)
>>>>>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>>>>>>at
>>>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepl
>>>>>>a
>>>>>>c
>>>>>>e
>>>>>>m
>>>>>>e
>>>>>>nt(PathUtils.java:321)
>>>>>>at
>>>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(Path
>>>>>>U
>>>>>>t
>>>>>>i
>>>>>>l
>>>>>>s
>>>>>>.java:96)
>>>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>>>>>>... 7 more
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>RES mngr:
>>>>>>
>>>>>>
>>>>>>Nov 05, 2014 5:26:28 PM
>>>>>>org.apache.oodt.cas.resource.jobqueue.JobStack
>>>>>>addJob
>>>>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>>>>>>Nov 05, 2014 5:26:30 PM
>>>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>>>>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>>>>>>Scheduling for execution
>>>>>>Nov 05, 2014 5:26:30 PM
>>>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>>>>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>>plainConnect
>>>>>>FINEST: ProxySelector Request for
>>>>>
>>>>>
>>>>
>>>>
>>>>>>http://localhost:2001/ <http://localhost:2001/>
>>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>>plainConnect
>>>>>>FINEST: Proxy used: DIRECT
>>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>>writeRequests
>>>>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>>>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>>>>> q=.2}{Connection: keep-alive}{Content-Length: 105}
>>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>>getInputStream
>>>>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>>>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>>>>text/xml}{Content-Length: 121}
>>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>>plainConnect
>>>>>>FINEST: ProxySelector Request for
>>>>>>http://localhost:2001/ <http://localhost:2001/>
>>>>
>>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>>plainConnect
>>>>>>FINEST: Proxy used: DIRECT
>>>>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>>writeRequests
>>>>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>>>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>>>>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>>>>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>>getInputStream
>>>>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>>>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>>>>text/xml}{Content-Length: 121}
>>>>>>Nov 05, 2014 5:26:32 PM
>>>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>>>>>>SEVERE: Job execution failed for jobId
>>>>>>'2014-11-05T17:26:28.785-08:00'
>>>>>>:
>>>>>>batchstub.executeJob returned false
>>>>>>java.lang.Exception: batchstub.executeJob returned false
>>>>>>at
>>>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcB
>>>>>>a
>>>>>>t
>>>>>>c
>>>>>>h
>>>>>>M
>>>>>>grProxy.java:125)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>Workflow:
>>>>>>
>>>>>>
>>>>>>Nov 05, 2014 5:26:27 PM
>>>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>>>>checkTaskRequiredMetadata
>>>>>>INFO: All required metadata fields present for task:
>>>>>>[FileConcatenator]
>>>>>>Nov 05, 2014 5:26:28 PM
>>>>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>>>>>>submitJob
>>>>>>FINEST: [{job.queueName=high,
>>>>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>>job.name <http://job.name> <http://job.name> <http://job.name>
>>>>>><http://job.name>
>>>>>><http://job.name>
>>>>>><http://job.name>=urn:oodt:FileConcatenator,
>>>
>>>
>>>>>>job.id <http://job.id> <http://job.id> <http://job.id>
>>>>>><http://job.id> <http://job.id>
>>>>>><http://job.id>=, job.status=,
>>>>>>job.load=2,
>>>>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput}
>>>>>>,
>>>>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.Fi
>>>>>>l
>>>>>>e
>>>>>>C
>>>>>>o
>>>>>>n
>>>>>>catenatorPGETask,
>>>>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-config
>>>>>>s
>>>>>>/
>>>>>>P
>>>>>>G
>>>>>>E
>>>>>>Config.xml,
>>>>>>
>>>>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatran
>>>>>>s
>>>>>>f
>>>>>>e
>>>>>>r
>>>>>>.
>>>>>>LocalDataTransferFactory,
>>>>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config
>>>>>>.
>>>>>>x
>>>>>>m
>>>>>>l
>>>>>>,
>>>>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>>>>>>PCS_WorkflowManagerUrl=http://localhost:9001,
>>>>>> PCS_FileManagerUrl=http://localhost:9000,
>>>>>>PGETask_Name=FileConcatenator},
>>>>>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>>>>>>WorkflowManagerUrl=[http://ubuntu:9001],
>>>>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>>>>>> ProcessingNode=[ubuntu], RunID=[testNumber1],
>>>>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>>>>>>Nov 05, 2014 5:26:39 PM
>>>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>>>>stop
>>>>>>WARNING: Attempt to kill current resmgr job:
>>>>>>[2014-11-05T17:26:28.785-08:00]: failed
>>>>>>
>>>>>>
>>>>>>When i searched online i found one of your mail archives where you
>>>>>>had
>>>>>>mentioned to change tasks.xml
>>>>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>>>>>>
>>>>>>
>>>>>>
>>>>>>It did not work. Please let us know how to proceed.
>>>>>>
>>>>>>
>>>>>>
>>>>>>Regards,
>>>>>>Renu
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>--
>>>
>>>
>>>>Meghamsh UtkurGraduate Student,University of Southern California
>>>>utkur@usc.edu
>>>>
>>
>>
>
>
>>>>(503)881-4791 <tel:%28503%29881-4791> <tel:%28503%29881-4791>
>>>><tel:%28503%29881-4791>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>--
>Meghamsh UtkurGraduate Student,University of Southern California
>utkur@usc.edu
>
>(503)881-4791
>
>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Renu Kanakamedala <ka...@usc.edu>.
Sorry Professor, the file I had sent previously is resmgr script. But that
seems like the right file to add the two lines you mentioned. I restarted
the res mgr after adding the lines, still getting the same exception.
Here is the link for batch_stub
http://pastebin.com/tfRCtJj0
Thanks!
- Renu
On Wed, Nov 5, 2014 at 10:01 PM, Christian Alan Mattmann <ma...@usc.edu>
wrote:
> Can you add lines:
>
> -Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \
> -Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \
>
> Starting at line 92 in your batch_stub script, restart
> the batch stub and see if that fixes it?
>
> Cheers,
> Chris
>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Adjunct Associate Professor, Computer Science Department
> University of Southern California
> Los Angeles, CA 90089 USA
> Email: mattmann@usc.edu
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
> -----Original Message-----
> From: Renu Kanakamedala <ka...@usc.edu>
> Date: Wednesday, November 5, 2014 at 10:51 PM
> To: Chris Mattmann <ma...@usc.edu>
> Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
> <de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
> Kondakindi <ko...@usc.edu>
> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> parse null Error.
>
> >wmgr script
> >
> >http://pastebin.com/zE7PWnXL
> >
> >
> >resmgr batch_stub
> >
> >
> >http://pastebin.com/gPUvbE9e
> >
> >
> >- Renu
> >
> >
> >
> >
> >
> >On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann
> ><ma...@usc.edu> wrote:
> >
> >Great. One last question.
> >
> >Can you show me the contents (paste.bin) of your wmgr
> >script and your batch_stub script?
> >
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >Chris Mattmann, Ph.D.
> >Adjunct Associate Professor, Computer Science Department
> >University of Southern California
> >Los Angeles, CA 90089 USA
> >Email: mattmann@usc.edu
> >WWW: http://sunset.usc.edu/~mattmann/
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> >
> >
> >
> >-----Original Message-----
> >From: Renu Kanakamedala <ka...@usc.edu>
> >Date: Wednesday, November 5, 2014 at 10:20 PM
> >To: Chris Mattmann <ma...@usc.edu>
> >Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
> ><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
> >Kondakindi <ko...@usc.edu>
> >Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >parse null Error.
> >
> >>renu@ubuntu:~$ echo $WORKFLOW_HOME
> >>/home/renu/oodt/workflow/
> >>renu@ubuntu:~$ echo $FILEMGR_HOME
> >>/home/renu/oodt/filemgr
> >>renu@ubuntu:~$ echo $WORKFLOW_URL
> >>http://localhost:9001
> >>renu@ubuntu:~$ echo $FILEMGR_URL
> >>http://localhost:9000
> >>renu@ubuntu:~$ echo $PGE_ROOT
> >>/home/renu/oodt/pge/
> >>
> >>renu@ubuntu:~$ echo $CRAWLER_HOME
> >>/home/renu/oodt/crawler/
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann
> >><ma...@usc.edu> wrote:
> >>
> >>Right, which is why I asked to specifically echo the value
> >>of all environment variables so we would be able to see the
> >>values. Can Renu please show the echo values?
> >>
> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>Chris Mattmann, Ph.D.
> >>Adjunct Associate Professor, Computer Science Department
> >>University of Southern California
> >>Los Angeles, CA 90089 USA
> >>Email: mattmann@usc.edu
> >>WWW: http://sunset.usc.edu/~mattmann/
> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>
> >>
> >>
> >>
> >>-----Original Message-----
> >>From: Meghamsh Utkur <ut...@usc.edu>
> >>Date: Wednesday, November 5, 2014 at 9:35 PM
> >>To: Renu Kanakamedala <ka...@usc.edu>
> >>Cc: Chris Mattmann <ma...@usc.edu>, "dev@oodt.apache.org"
> >><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
> >>Kondakindi <ko...@usc.edu>
> >>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >>parse null Error.
> >>
> >>>Professor,
> >>>
> >>>
> >>>To add to what Renu has said, I went through the source code and I
> >>>suspect that the value here in this method is passed as null.
> >>>
> >>>XmlFilePgeConfigBuilder.java
> >>>
> >>>1. private String fillIn(String value,Metadata inputMetadata,boolean
> >>>envReplaceRecur)
> >>>2. throwsException {
> >>>3. try{
> >>>4. while((value =PathUtils.doDynamicReplacement(value,
> >>>inputMetadata)).contains("[")&& envReplaceRecur);
> >>>5. if(value.toUpperCase()
> >>>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
> >>>7. value
> >>>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
> >>>URL(inputMetadata
> >>>8.
> >>>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.
> >>>p
> >>>a
> >>>rseSqlQueryMethod(value)));
> >>>9. return value;
> >>>10. }catch(Exception e){
> >>>11. thrownew Exception("Failed to parse value: "+ value, e);
> >>>12. }
> >>>13. }
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
> >>><ka...@usc.edu> wrote:
> >>>
> >>>Hello Professor,
> >>>
> >>>
> >>>All the environment variables present in tasks.xml i.e.
> >>>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
> >>>
> >>>
> >>>Please let us know how to proceed.
> >>>
> >>>
> >>>Regards,
> >>>Renu
> >>>
> >>>
> >>>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
> >>><ma...@usc.edu> wrote:
> >>>
> >>>Can you please confirm that all of the bracketed environment
> >>>variables below do in fact exist in the shell that you are
> >>>running ./oodt start in?
> >>>
> >>>E.g., please echo $ENV_VAR below and show their values.
> >>>
> >>>Cheers,
> >>>Chris
> >>>
> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>Chris Mattmann, Ph.D.
> >>>Adjunct Associate Professor, Computer Science Department
> >>>University of Southern California
> >>>Los Angeles, CA 90089 USA
> >>>Email: mattmann@usc.edu
> >>>WWW: http://sunset.usc.edu/~mattmann/
> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>
> >>>
> >>>
> >>>
> >>>-----Original Message-----
> >>>From: Renu Kanakamedala <ka...@usc.edu>
> >>>Date: Wednesday, November 5, 2014 at 8:18 PM
> >>>To: Chris Mattmann <ma...@usc.edu>
> >>>Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
> >>><pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
> >>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >>>parse null Error.
> >>>
> >>>>Please find below tasks.xml file.
> >>>>
> >>>>
> >>>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
> >>>><!--
> >>>> TODO: Add some examples
> >>>>-->
> >>>><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
> >>>>
> >>>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPG
> >>>>E
> >>>>T
> >>>>a
> >>>>sk">
> >>>>
> >>>> <conditions/>
> >>>>
> >>>> <configuration>
> >>>> <property name="PGETask_Name" value="FileConcatenator"/>
> >>>> <property name="PGETask_ConfigFilePath"
> >>>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
> >>>>envReplace="true"/>
> >>>> <property name="PGETask_DumpMetadata" value="true"/>
> >>>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
> >>>>envReplace="true" />
> >>>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
> >>>>envReplace="true"/>
> >>>> <property name="PCS_MetFileExtension" value="met"/>
> >>>> <property name="PCS_ClientTransferServiceFactory"
> >>>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactor
> >>>>y
> >>>>"
> >>>>/
> >>>>>
> >>>> <property name="PCS_ActionRepoFile"
> >>>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml"
> >>>>envReplace="true"/>
> >>>> </configuration>
> >>>>
> >>>> <requiredMetFields>
> >>>> <metfield name="RunID"/>
> >>>> </requiredMetFields>
> >>>>
> >>>></task>
> >>>></cas:tasks>
> >>>>
> >>>>
> >>>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
> >>>><ma...@usc.edu> wrote:
> >>>>
> >>>>Hi Renu,
> >>>>
> >>>>Can you please show a paste.bin of your tasks.xml?
> >>>>
> >>>>Cheers,
> >>>>Chris
> >>>>
> >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>>Chris Mattmann, Ph.D.
> >>>>Adjunct Associate Professor, Computer Science Department
> >>>>University of Southern California
> >>>>Los Angeles, CA 90089 USA
> >>>>Email: mattmann@usc.edu
> >>>>WWW: http://sunset.usc.edu/~mattmann/
> >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>-----Original Message-----
> >>>>From: Renu Kanakamedala <ka...@usc.edu>
> >>>>Date: Wednesday, November 5, 2014 at 7:55 PM
> >>>>To: Chris Mattmann <ma...@usc.edu>
> >>>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >>>>parse
> >>>>null Error.
> >>>>
> >>>>>Hello Professor,
> >>>
> >>>>>
> >>>>>
> >>>>>When i am trying to run file concatenator CAS PGE example using the
> >>>>>following command.
> >>>>>
> >>>>>
> >>>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
> >>>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
> >>>>>
> >>>>>
> >>>>>
> >>>>>The message shown after executing this command is "Sending Event
> >>>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and
> >>>>>batch
> >>>>>stub.
> >>>>>
> >>>>>
> >>>>>Batch Stub:
> >>>>>Caused by:
> >>>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceEx
> >>>>>c
> >>>>>e
> >>>>>p
> >>>>>t
> >>>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
> >>>>>value: null
> >>>>>at
> >>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> >>>>>at
> >>>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> >>>>>... 2 more
> >>>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
> >>>>>parse value: null
> >>>>>at
> >>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
> >>>>>e
> >>>>>C
> >>>>>o
> >>>>>n
> >>>>>figBuilder.java:87)
> >>>>>at
> >>>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc
> >>>>>e
> >>>>>.
> >>>>>j
> >>>>>a
> >>>>>va:230)
> >>>>>at
> >>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> >>>>>... 3 more
> >>>>>Caused by: java.lang.Exception: Failed to parse value: null
> >>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> >>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> >>>>>at
> >>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
> >>>>>e
> >>>>>C
> >>>>>o
> >>>>>n
> >>>>>figBuilder.java:77)
> >>>>>... 5 more
> >>>>>Caused by: java.lang.NullPointerException
> >>>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> >>>>>at java.util.regex.Matcher.reset(Matcher.java:308)
> >>>>>at java.util.regex.Matcher.<init>(Matcher.java:228)
> >>>>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
> >>>>>at
> >>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla
> >>>>>c
> >>>>>e
> >>>>>m
> >>>>>e
> >>>>>nt(PathUtils.java:321)
> >>>>>at
> >>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU
> >>>>>t
> >>>>>i
> >>>>>l
> >>>>>s
> >>>>>.java:96)
> >>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
> >>>>>... 7 more
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>RES mngr:
> >>>>>
> >>>>>
> >>>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
> >>>>>addJob
> >>>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
> >>>>>Nov 05, 2014 5:26:30 PM
> >>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
> >>>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
> >>>>>Scheduling for execution
> >>>>>Nov 05, 2014 5:26:30 PM
> >>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
> >>>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>>plainConnect
> >>>>>FINEST: ProxySelector Request for
> >>>>
> >>>>
> >>>
> >>>
> >>>>>http://localhost:2001/ <http://localhost:2001/>
> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>>plainConnect
> >>>>>FINEST: Proxy used: DIRECT
> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>>writeRequests
> >>>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
> >>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >>>>> q=.2}{Connection: keep-alive}{Content-Length: 105}
> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>>getInputStream
> >>>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
> >>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>>>>text/xml}{Content-Length: 121}
> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>>plainConnect
> >>>>>FINEST: ProxySelector Request for
> >>>>>http://localhost:2001/ <http://localhost:2001/>
> >>>
> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>>plainConnect
> >>>>>FINEST: Proxy used: DIRECT
> >>>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>>writeRequests
> >>>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
> >>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >>>>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
> >>>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>>getInputStream
> >>>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
> >>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>>>>text/xml}{Content-Length: 121}
> >>>>>Nov 05, 2014 5:26:32 PM
> >>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
> >>>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00'
> >>>>>:
> >>>>>batchstub.executeJob returned false
> >>>>>java.lang.Exception: batchstub.executeJob returned false
> >>>>>at
> >>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBa
> >>>>>t
> >>>>>c
> >>>>>h
> >>>>>M
> >>>>>grProxy.java:125)
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>Workflow:
> >>>>>
> >>>>>
> >>>>>Nov 05, 2014 5:26:27 PM
> >>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>>>>checkTaskRequiredMetadata
> >>>>>INFO: All required metadata fields present for task:
> >>>>>[FileConcatenator]
> >>>>>Nov 05, 2014 5:26:28 PM
> >>>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
> >>>>>submitJob
> >>>>>FINEST: [{job.queueName=high,
> >>>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>>>>job.name <http://job.name> <http://job.name> <http://job.name>
> >>>>><http://job.name>
> >>>>><http://job.name>=urn:oodt:FileConcatenator,
> >>
> >>
> >>>>>job.id <http://job.id> <http://job.id> <http://job.id> <http://job.id
> >
> >>>>><http://job.id>=, job.status=,
> >>>>>job.load=2,
> >>>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
> >>>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.Fil
> >>>>>e
> >>>>>C
> >>>>>o
> >>>>>n
> >>>>>catenatorPGETask,
> >>>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs
> >>>>>/
> >>>>>P
> >>>>>G
> >>>>>E
> >>>>>Config.xml,
> >>>>>
> >>>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatrans
> >>>>>f
> >>>>>e
> >>>>>r
> >>>>>.
> >>>>>LocalDataTransferFactory,
> >>>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.
> >>>>>x
> >>>>>m
> >>>>>l
> >>>>>,
> >>>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
> >>>>>PCS_WorkflowManagerUrl=http://localhost:9001,
> >>>>> PCS_FileManagerUrl=http://localhost:9000,
> >>>>>PGETask_Name=FileConcatenator},
> >>>>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
> >>>>>WorkflowManagerUrl=[http://ubuntu:9001],
> >>>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
> >>>>> ProcessingNode=[ubuntu], RunID=[testNumber1],
> >>>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
> >>>>>Nov 05, 2014 5:26:39 PM
> >>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>>>>stop
> >>>>>WARNING: Attempt to kill current resmgr job:
> >>>>>[2014-11-05T17:26:28.785-08:00]: failed
> >>>>>
> >>>>>
> >>>>>When i searched online i found one of your mail archives where you had
> >>>>>mentioned to change tasks.xml
> >>>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
> >>>>>
> >>>>>
> >>>>>
> >>>>>It did not work. Please let us know how to proceed.
> >>>>>
> >>>>>
> >>>>>
> >>>>>Regards,
> >>>>>Renu
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>--
> >>
> >>
> >>>Meghamsh UtkurGraduate Student,University of Southern California
> >>>utkur@usc.edu
> >>>
> >
> >
> >>>(503)881-4791 <tel:%28503%29881-4791> <tel:%28503%29881-4791>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> >
> >
> >
> >
>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Christian Alan Mattmann <ma...@usc.edu>.
Can you add lines:
-Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \
-Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \
Starting at line 92 in your batch_stub script, restart
the batch stub and see if that fixes it?
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Adjunct Associate Professor, Computer Science Department
University of Southern California
Los Angeles, CA 90089 USA
Email: mattmann@usc.edu
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Renu Kanakamedala <ka...@usc.edu>
Date: Wednesday, November 5, 2014 at 10:51 PM
To: Chris Mattmann <ma...@usc.edu>
Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
<de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
Kondakindi <ko...@usc.edu>
Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
parse null Error.
>wmgr script
>
>http://pastebin.com/zE7PWnXL
>
>
>resmgr batch_stub
>
>
>http://pastebin.com/gPUvbE9e
>
>
>- Renu
>
>
>
>
>
>On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann
><ma...@usc.edu> wrote:
>
>Great. One last question.
>
>Can you show me the contents (paste.bin) of your wmgr
>script and your batch_stub script?
>
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>Chris Mattmann, Ph.D.
>Adjunct Associate Professor, Computer Science Department
>University of Southern California
>Los Angeles, CA 90089 USA
>Email: mattmann@usc.edu
>WWW: http://sunset.usc.edu/~mattmann/
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>-----Original Message-----
>From: Renu Kanakamedala <ka...@usc.edu>
>Date: Wednesday, November 5, 2014 at 10:20 PM
>To: Chris Mattmann <ma...@usc.edu>
>Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
>Kondakindi <ko...@usc.edu>
>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>parse null Error.
>
>>renu@ubuntu:~$ echo $WORKFLOW_HOME
>>/home/renu/oodt/workflow/
>>renu@ubuntu:~$ echo $FILEMGR_HOME
>>/home/renu/oodt/filemgr
>>renu@ubuntu:~$ echo $WORKFLOW_URL
>>http://localhost:9001
>>renu@ubuntu:~$ echo $FILEMGR_URL
>>http://localhost:9000
>>renu@ubuntu:~$ echo $PGE_ROOT
>>/home/renu/oodt/pge/
>>
>>renu@ubuntu:~$ echo $CRAWLER_HOME
>>/home/renu/oodt/crawler/
>>
>>
>>
>>
>>
>>
>>
>>On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann
>><ma...@usc.edu> wrote:
>>
>>Right, which is why I asked to specifically echo the value
>>of all environment variables so we would be able to see the
>>values. Can Renu please show the echo values?
>>
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>Chris Mattmann, Ph.D.
>>Adjunct Associate Professor, Computer Science Department
>>University of Southern California
>>Los Angeles, CA 90089 USA
>>Email: mattmann@usc.edu
>>WWW: http://sunset.usc.edu/~mattmann/
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>-----Original Message-----
>>From: Meghamsh Utkur <ut...@usc.edu>
>>Date: Wednesday, November 5, 2014 at 9:35 PM
>>To: Renu Kanakamedala <ka...@usc.edu>
>>Cc: Chris Mattmann <ma...@usc.edu>, "dev@oodt.apache.org"
>><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
>>Kondakindi <ko...@usc.edu>
>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>parse null Error.
>>
>>>Professor,
>>>
>>>
>>>To add to what Renu has said, I went through the source code and I
>>>suspect that the value here in this method is passed as null.
>>>
>>>XmlFilePgeConfigBuilder.java
>>>
>>>1. private String fillIn(String value,Metadata inputMetadata,boolean
>>>envReplaceRecur)
>>>2. throwsException {
>>>3. try{
>>>4. while((value =PathUtils.doDynamicReplacement(value,
>>>inputMetadata)).contains("[")&& envReplaceRecur);
>>>5. if(value.toUpperCase()
>>>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
>>>7. value
>>>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
>>>URL(inputMetadata
>>>8.
>>>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.
>>>p
>>>a
>>>rseSqlQueryMethod(value)));
>>>9. return value;
>>>10. }catch(Exception e){
>>>11. thrownew Exception("Failed to parse value: "+ value, e);
>>>12. }
>>>13. }
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
>>><ka...@usc.edu> wrote:
>>>
>>>Hello Professor,
>>>
>>>
>>>All the environment variables present in tasks.xml i.e.
>>>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
>>>
>>>
>>>Please let us know how to proceed.
>>>
>>>
>>>Regards,
>>>Renu
>>>
>>>
>>>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
>>><ma...@usc.edu> wrote:
>>>
>>>Can you please confirm that all of the bracketed environment
>>>variables below do in fact exist in the shell that you are
>>>running ./oodt start in?
>>>
>>>E.g., please echo $ENV_VAR below and show their values.
>>>
>>>Cheers,
>>>Chris
>>>
>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>Chris Mattmann, Ph.D.
>>>Adjunct Associate Professor, Computer Science Department
>>>University of Southern California
>>>Los Angeles, CA 90089 USA
>>>Email: mattmann@usc.edu
>>>WWW: http://sunset.usc.edu/~mattmann/
>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>>
>>>
>>>
>>>-----Original Message-----
>>>From: Renu Kanakamedala <ka...@usc.edu>
>>>Date: Wednesday, November 5, 2014 at 8:18 PM
>>>To: Chris Mattmann <ma...@usc.edu>
>>>Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
>>><pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
>>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>>parse null Error.
>>>
>>>>Please find below tasks.xml file.
>>>>
>>>>
>>>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
>>>><!--
>>>> TODO: Add some examples
>>>>-->
>>>><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>>>>
>>>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPG
>>>>E
>>>>T
>>>>a
>>>>sk">
>>>>
>>>> <conditions/>
>>>>
>>>> <configuration>
>>>> <property name="PGETask_Name" value="FileConcatenator"/>
>>>> <property name="PGETask_ConfigFilePath"
>>>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>>>>envReplace="true"/>
>>>> <property name="PGETask_DumpMetadata" value="true"/>
>>>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>>>>envReplace="true" />
>>>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
>>>>envReplace="true"/>
>>>> <property name="PCS_MetFileExtension" value="met"/>
>>>> <property name="PCS_ClientTransferServiceFactory"
>>>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactor
>>>>y
>>>>"
>>>>/
>>>>>
>>>> <property name="PCS_ActionRepoFile"
>>>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml"
>>>>envReplace="true"/>
>>>> </configuration>
>>>>
>>>> <requiredMetFields>
>>>> <metfield name="RunID"/>
>>>> </requiredMetFields>
>>>>
>>>></task>
>>>></cas:tasks>
>>>>
>>>>
>>>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
>>>><ma...@usc.edu> wrote:
>>>>
>>>>Hi Renu,
>>>>
>>>>Can you please show a paste.bin of your tasks.xml?
>>>>
>>>>Cheers,
>>>>Chris
>>>>
>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>Chris Mattmann, Ph.D.
>>>>Adjunct Associate Professor, Computer Science Department
>>>>University of Southern California
>>>>Los Angeles, CA 90089 USA
>>>>Email: mattmann@usc.edu
>>>>WWW: http://sunset.usc.edu/~mattmann/
>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>
>>>>
>>>>
>>>>
>>>>-----Original Message-----
>>>>From: Renu Kanakamedala <ka...@usc.edu>
>>>>Date: Wednesday, November 5, 2014 at 7:55 PM
>>>>To: Chris Mattmann <ma...@usc.edu>
>>>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>>>parse
>>>>null Error.
>>>>
>>>>>Hello Professor,
>>>
>>>>>
>>>>>
>>>>>When i am trying to run file concatenator CAS PGE example using the
>>>>>following command.
>>>>>
>>>>>
>>>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>>>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>>>>>
>>>>>
>>>>>
>>>>>The message shown after executing this command is "Sending Event
>>>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and
>>>>>batch
>>>>>stub.
>>>>>
>>>>>
>>>>>Batch Stub:
>>>>>Caused by:
>>>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceEx
>>>>>c
>>>>>e
>>>>>p
>>>>>t
>>>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>>>>>value: null
>>>>>at
>>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>>>>>at
>>>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>>>>>... 2 more
>>>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>>>>>parse value: null
>>>>>at
>>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
>>>>>e
>>>>>C
>>>>>o
>>>>>n
>>>>>figBuilder.java:87)
>>>>>at
>>>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc
>>>>>e
>>>>>.
>>>>>j
>>>>>a
>>>>>va:230)
>>>>>at
>>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>>>>>... 3 more
>>>>>Caused by: java.lang.Exception: Failed to parse value: null
>>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>>>>>at
>>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
>>>>>e
>>>>>C
>>>>>o
>>>>>n
>>>>>figBuilder.java:77)
>>>>>... 5 more
>>>>>Caused by: java.lang.NullPointerException
>>>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>>>>>at java.util.regex.Matcher.reset(Matcher.java:308)
>>>>>at java.util.regex.Matcher.<init>(Matcher.java:228)
>>>>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>>>>>at
>>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla
>>>>>c
>>>>>e
>>>>>m
>>>>>e
>>>>>nt(PathUtils.java:321)
>>>>>at
>>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU
>>>>>t
>>>>>i
>>>>>l
>>>>>s
>>>>>.java:96)
>>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>>>>>... 7 more
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>RES mngr:
>>>>>
>>>>>
>>>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>>>>>addJob
>>>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>>>>>Nov 05, 2014 5:26:30 PM
>>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>>>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>>>>>Scheduling for execution
>>>>>Nov 05, 2014 5:26:30 PM
>>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>>>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>plainConnect
>>>>>FINEST: ProxySelector Request for
>>>>
>>>>
>>>
>>>
>>>>>http://localhost:2001/ <http://localhost:2001/>
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>plainConnect
>>>>>FINEST: Proxy used: DIRECT
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>writeRequests
>>>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>>>> q=.2}{Connection: keep-alive}{Content-Length: 105}
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>getInputStream
>>>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>>>text/xml}{Content-Length: 121}
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>plainConnect
>>>>>FINEST: ProxySelector Request for
>>>>>http://localhost:2001/ <http://localhost:2001/>
>>>
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>plainConnect
>>>>>FINEST: Proxy used: DIRECT
>>>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>writeRequests
>>>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>>>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>>>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>getInputStream
>>>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>>>text/xml}{Content-Length: 121}
>>>>>Nov 05, 2014 5:26:32 PM
>>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>>>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00'
>>>>>:
>>>>>batchstub.executeJob returned false
>>>>>java.lang.Exception: batchstub.executeJob returned false
>>>>>at
>>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBa
>>>>>t
>>>>>c
>>>>>h
>>>>>M
>>>>>grProxy.java:125)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>Workflow:
>>>>>
>>>>>
>>>>>Nov 05, 2014 5:26:27 PM
>>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>>>checkTaskRequiredMetadata
>>>>>INFO: All required metadata fields present for task:
>>>>>[FileConcatenator]
>>>>>Nov 05, 2014 5:26:28 PM
>>>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>>>>>submitJob
>>>>>FINEST: [{job.queueName=high,
>>>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>>>job.name <http://job.name> <http://job.name> <http://job.name>
>>>>><http://job.name>
>>>>><http://job.name>=urn:oodt:FileConcatenator,
>>
>>
>>>>>job.id <http://job.id> <http://job.id> <http://job.id> <http://job.id>
>>>>><http://job.id>=, job.status=,
>>>>>job.load=2,
>>>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>>>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.Fil
>>>>>e
>>>>>C
>>>>>o
>>>>>n
>>>>>catenatorPGETask,
>>>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs
>>>>>/
>>>>>P
>>>>>G
>>>>>E
>>>>>Config.xml,
>>>>>
>>>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatrans
>>>>>f
>>>>>e
>>>>>r
>>>>>.
>>>>>LocalDataTransferFactory,
>>>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.
>>>>>x
>>>>>m
>>>>>l
>>>>>,
>>>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>>>>>PCS_WorkflowManagerUrl=http://localhost:9001,
>>>>> PCS_FileManagerUrl=http://localhost:9000,
>>>>>PGETask_Name=FileConcatenator},
>>>>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>>>>>WorkflowManagerUrl=[http://ubuntu:9001],
>>>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>>>>> ProcessingNode=[ubuntu], RunID=[testNumber1],
>>>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>>>>>Nov 05, 2014 5:26:39 PM
>>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>>>stop
>>>>>WARNING: Attempt to kill current resmgr job:
>>>>>[2014-11-05T17:26:28.785-08:00]: failed
>>>>>
>>>>>
>>>>>When i searched online i found one of your mail archives where you had
>>>>>mentioned to change tasks.xml
>>>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>>>>>
>>>>>
>>>>>
>>>>>It did not work. Please let us know how to proceed.
>>>>>
>>>>>
>>>>>
>>>>>Regards,
>>>>>Renu
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>--
>>
>>
>>>Meghamsh UtkurGraduate Student,University of Southern California
>>>utkur@usc.edu
>>>
>
>
>>>(503)881-4791 <tel:%28503%29881-4791> <tel:%28503%29881-4791>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Renu Kanakamedala <ka...@usc.edu>.
*wmgr script *
http://pastebin.com/zE7PWnXL
*resmgr batch_stub*
http://pastebin.com/gPUvbE9e
- Renu
On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann <ma...@usc.edu>
wrote:
> Great. One last question.
>
> Can you show me the contents (paste.bin) of your wmgr
> script and your batch_stub script?
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Adjunct Associate Professor, Computer Science Department
> University of Southern California
> Los Angeles, CA 90089 USA
> Email: mattmann@usc.edu
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
> -----Original Message-----
> From: Renu Kanakamedala <ka...@usc.edu>
> Date: Wednesday, November 5, 2014 at 10:20 PM
> To: Chris Mattmann <ma...@usc.edu>
> Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
> <de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
> Kondakindi <ko...@usc.edu>
> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> parse null Error.
>
> >renu@ubuntu:~$ echo $WORKFLOW_HOME
> >/home/renu/oodt/workflow/
> >renu@ubuntu:~$ echo $FILEMGR_HOME
> >/home/renu/oodt/filemgr
> >renu@ubuntu:~$ echo $WORKFLOW_URL
> >http://localhost:9001
> >renu@ubuntu:~$ echo $FILEMGR_URL
> >http://localhost:9000
> >renu@ubuntu:~$ echo $PGE_ROOT
> >/home/renu/oodt/pge/
> >
> >renu@ubuntu:~$ echo $CRAWLER_HOME
> >/home/renu/oodt/crawler/
> >
> >
> >
> >
> >
> >
> >
> >On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann
> ><ma...@usc.edu> wrote:
> >
> >Right, which is why I asked to specifically echo the value
> >of all environment variables so we would be able to see the
> >values. Can Renu please show the echo values?
> >
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >Chris Mattmann, Ph.D.
> >Adjunct Associate Professor, Computer Science Department
> >University of Southern California
> >Los Angeles, CA 90089 USA
> >Email: mattmann@usc.edu
> >WWW: http://sunset.usc.edu/~mattmann/
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> >
> >
> >
> >-----Original Message-----
> >From: Meghamsh Utkur <ut...@usc.edu>
> >Date: Wednesday, November 5, 2014 at 9:35 PM
> >To: Renu Kanakamedala <ka...@usc.edu>
> >Cc: Chris Mattmann <ma...@usc.edu>, "dev@oodt.apache.org"
> ><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
> >Kondakindi <ko...@usc.edu>
> >Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >parse null Error.
> >
> >>Professor,
> >>
> >>
> >>To add to what Renu has said, I went through the source code and I
> >>suspect that the value here in this method is passed as null.
> >>
> >>XmlFilePgeConfigBuilder.java
> >>
> >>1. private String fillIn(String value,Metadata inputMetadata,boolean
> >>envReplaceRecur)
> >>2. throwsException {
> >>3. try{
> >>4. while((value =PathUtils.doDynamicReplacement(value,
> >>inputMetadata)).contains("[")&& envReplaceRecur);
> >>5. if(value.toUpperCase()
> >>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
> >>7. value
> >>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
> >>URL(inputMetadata
> >>8.
> >>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.p
> >>a
> >>rseSqlQueryMethod(value)));
> >>9. return value;
> >>10. }catch(Exception e){
> >>11. thrownew Exception("Failed to parse value: "+ value, e);
> >>12. }
> >>13. }
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
> >><ka...@usc.edu> wrote:
> >>
> >>Hello Professor,
> >>
> >>
> >>All the environment variables present in tasks.xml i.e.
> >>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
> >>
> >>
> >>Please let us know how to proceed.
> >>
> >>
> >>Regards,
> >>Renu
> >>
> >>
> >>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
> >><ma...@usc.edu> wrote:
> >>
> >>Can you please confirm that all of the bracketed environment
> >>variables below do in fact exist in the shell that you are
> >>running ./oodt start in?
> >>
> >>E.g., please echo $ENV_VAR below and show their values.
> >>
> >>Cheers,
> >>Chris
> >>
> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>Chris Mattmann, Ph.D.
> >>Adjunct Associate Professor, Computer Science Department
> >>University of Southern California
> >>Los Angeles, CA 90089 USA
> >>Email: mattmann@usc.edu
> >>WWW: http://sunset.usc.edu/~mattmann/
> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>
> >>
> >>
> >>
> >>-----Original Message-----
> >>From: Renu Kanakamedala <ka...@usc.edu>
> >>Date: Wednesday, November 5, 2014 at 8:18 PM
> >>To: Chris Mattmann <ma...@usc.edu>
> >>Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
> >><pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
> >>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >>parse null Error.
> >>
> >>>Please find below tasks.xml file.
> >>>
> >>>
> >>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
> >>><!--
> >>> TODO: Add some examples
> >>>-->
> >>><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
> >>>
> >>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGE
> >>>T
> >>>a
> >>>sk">
> >>>
> >>> <conditions/>
> >>>
> >>> <configuration>
> >>> <property name="PGETask_Name" value="FileConcatenator"/>
> >>> <property name="PGETask_ConfigFilePath"
> >>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
> >>>envReplace="true"/>
> >>> <property name="PGETask_DumpMetadata" value="true"/>
> >>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
> >>>envReplace="true" />
> >>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
> >>>envReplace="true"/>
> >>> <property name="PCS_MetFileExtension" value="met"/>
> >>> <property name="PCS_ClientTransferServiceFactory"
> >>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory
> >>>"
> >>>/
> >>>>
> >>> <property name="PCS_ActionRepoFile"
> >>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml"
> >>>envReplace="true"/>
> >>> </configuration>
> >>>
> >>> <requiredMetFields>
> >>> <metfield name="RunID"/>
> >>> </requiredMetFields>
> >>>
> >>></task>
> >>></cas:tasks>
> >>>
> >>>
> >>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
> >>><ma...@usc.edu> wrote:
> >>>
> >>>Hi Renu,
> >>>
> >>>Can you please show a paste.bin of your tasks.xml?
> >>>
> >>>Cheers,
> >>>Chris
> >>>
> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>Chris Mattmann, Ph.D.
> >>>Adjunct Associate Professor, Computer Science Department
> >>>University of Southern California
> >>>Los Angeles, CA 90089 USA
> >>>Email: mattmann@usc.edu
> >>>WWW: http://sunset.usc.edu/~mattmann/
> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>
> >>>
> >>>
> >>>
> >>>-----Original Message-----
> >>>From: Renu Kanakamedala <ka...@usc.edu>
> >>>Date: Wednesday, November 5, 2014 at 7:55 PM
> >>>To: Chris Mattmann <ma...@usc.edu>
> >>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >>>parse
> >>>null Error.
> >>>
> >>>>Hello Professor,
> >>
> >>>>
> >>>>
> >>>>When i am trying to run file concatenator CAS PGE example using the
> >>>>following command.
> >>>>
> >>>>
> >>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
> >>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
> >>>>
> >>>>
> >>>>
> >>>>The message shown after executing this command is "Sending Event
> >>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
> >>>>stub.
> >>>>
> >>>>
> >>>>Batch Stub:
> >>>>Caused by:
> >>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExc
> >>>>e
> >>>>p
> >>>>t
> >>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
> >>>>value: null
> >>>>at
> >>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> >>>>at
> >>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> >>>>... 2 more
> >>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
> >>>>parse value: null
> >>>>at
> >>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge
> >>>>C
> >>>>o
> >>>>n
> >>>>figBuilder.java:87)
> >>>>at
> >>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance
> >>>>.
> >>>>j
> >>>>a
> >>>>va:230)
> >>>>at
> >>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> >>>>... 3 more
> >>>>Caused by: java.lang.Exception: Failed to parse value: null
> >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> >>>>at
> >>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge
> >>>>C
> >>>>o
> >>>>n
> >>>>figBuilder.java:77)
> >>>>... 5 more
> >>>>Caused by: java.lang.NullPointerException
> >>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> >>>>at java.util.regex.Matcher.reset(Matcher.java:308)
> >>>>at java.util.regex.Matcher.<init>(Matcher.java:228)
> >>>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
> >>>>at
> >>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplac
> >>>>e
> >>>>m
> >>>>e
> >>>>nt(PathUtils.java:321)
> >>>>at
> >>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUt
> >>>>i
> >>>>l
> >>>>s
> >>>>.java:96)
> >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
> >>>>... 7 more
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>RES mngr:
> >>>>
> >>>>
> >>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
> >>>>addJob
> >>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
> >>>>Nov 05, 2014 5:26:30 PM
> >>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
> >>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
> >>>>Scheduling for execution
> >>>>Nov 05, 2014 5:26:30 PM
> >>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
> >>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>plainConnect
> >>>>FINEST: ProxySelector Request for
> >>>
> >>>
> >>
> >>
> >>>>http://localhost:2001/ <http://localhost:2001/>
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>plainConnect
> >>>>FINEST: Proxy used: DIRECT
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>writeRequests
> >>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
> >>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >>>> q=.2}{Connection: keep-alive}{Content-Length: 105}
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>getInputStream
> >>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
> >>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>>>text/xml}{Content-Length: 121}
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>plainConnect
> >>>>FINEST: ProxySelector Request for
> >>>>http://localhost:2001/ <http://localhost:2001/>
> >>
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>plainConnect
> >>>>FINEST: Proxy used: DIRECT
> >>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>writeRequests
> >>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
> >>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >>>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
> >>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>getInputStream
> >>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
> >>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>>>text/xml}{Content-Length: 121}
> >>>>Nov 05, 2014 5:26:32 PM
> >>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
> >>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00'
> >>>>:
> >>>>batchstub.executeJob returned false
> >>>>java.lang.Exception: batchstub.executeJob returned false
> >>>>at
> >>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBat
> >>>>c
> >>>>h
> >>>>M
> >>>>grProxy.java:125)
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>Workflow:
> >>>>
> >>>>
> >>>>Nov 05, 2014 5:26:27 PM
> >>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>>>checkTaskRequiredMetadata
> >>>>INFO: All required metadata fields present for task: [FileConcatenator]
> >>>>Nov 05, 2014 5:26:28 PM
> >>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
> >>>>submitJob
> >>>>FINEST: [{job.queueName=high,
> >>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
> >>>
> >>>
> >>
> >>
> >>
> >>
> >>>>job.name <http://job.name> <http://job.name> <http://job.name>
> >>>><http://job.name>=urn:oodt:FileConcatenator,
> >
> >
> >>>>job.id <http://job.id> <http://job.id> <http://job.id>
> >>>><http://job.id>=, job.status=,
> >>>>job.load=2,
> >>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
> >>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.File
> >>>>C
> >>>>o
> >>>>n
> >>>>catenatorPGETask,
> >>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/
> >>>>P
> >>>>G
> >>>>E
> >>>>Config.xml,
> >>>>
> >>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransf
> >>>>e
> >>>>r
> >>>>.
> >>>>LocalDataTransferFactory,
> >>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.x
> >>>>m
> >>>>l
> >>>>,
> >>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
> >>>>PCS_WorkflowManagerUrl=http://localhost:9001,
> >>>> PCS_FileManagerUrl=http://localhost:9000,
> >>>>PGETask_Name=FileConcatenator},
> >>>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
> >>>>WorkflowManagerUrl=[http://ubuntu:9001],
> >>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
> >>>> ProcessingNode=[ubuntu], RunID=[testNumber1],
> >>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
> >>>>Nov 05, 2014 5:26:39 PM
> >>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>>>stop
> >>>>WARNING: Attempt to kill current resmgr job:
> >>>>[2014-11-05T17:26:28.785-08:00]: failed
> >>>>
> >>>>
> >>>>When i searched online i found one of your mail archives where you had
> >>>>mentioned to change tasks.xml
> >>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
> >>>>
> >>>>
> >>>>
> >>>>It did not work. Please let us know how to proceed.
> >>>>
> >>>>
> >>>>
> >>>>Regards,
> >>>>Renu
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>--
> >
> >
> >>Meghamsh UtkurGraduate Student,University of Southern California
> >>utkur@usc.edu
> >>
> >>(503)881-4791 <tel:%28503%29881-4791>
> >>
> >>
> >>
> >
> >
> >
> >
> >
> >
>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Christian Alan Mattmann <ma...@usc.edu>.
Great. One last question.
Can you show me the contents (paste.bin) of your wmgr
script and your batch_stub script?
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Adjunct Associate Professor, Computer Science Department
University of Southern California
Los Angeles, CA 90089 USA
Email: mattmann@usc.edu
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Renu Kanakamedala <ka...@usc.edu>
Date: Wednesday, November 5, 2014 at 10:20 PM
To: Chris Mattmann <ma...@usc.edu>
Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org"
<de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
Kondakindi <ko...@usc.edu>
Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
parse null Error.
>renu@ubuntu:~$ echo $WORKFLOW_HOME
>/home/renu/oodt/workflow/
>renu@ubuntu:~$ echo $FILEMGR_HOME
>/home/renu/oodt/filemgr
>renu@ubuntu:~$ echo $WORKFLOW_URL
>http://localhost:9001
>renu@ubuntu:~$ echo $FILEMGR_URL
>http://localhost:9000
>renu@ubuntu:~$ echo $PGE_ROOT
>/home/renu/oodt/pge/
>
>renu@ubuntu:~$ echo $CRAWLER_HOME
>/home/renu/oodt/crawler/
>
>
>
>
>
>
>
>On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann
><ma...@usc.edu> wrote:
>
>Right, which is why I asked to specifically echo the value
>of all environment variables so we would be able to see the
>values. Can Renu please show the echo values?
>
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>Chris Mattmann, Ph.D.
>Adjunct Associate Professor, Computer Science Department
>University of Southern California
>Los Angeles, CA 90089 USA
>Email: mattmann@usc.edu
>WWW: http://sunset.usc.edu/~mattmann/
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>-----Original Message-----
>From: Meghamsh Utkur <ut...@usc.edu>
>Date: Wednesday, November 5, 2014 at 9:35 PM
>To: Renu Kanakamedala <ka...@usc.edu>
>Cc: Chris Mattmann <ma...@usc.edu>, "dev@oodt.apache.org"
><de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
>Kondakindi <ko...@usc.edu>
>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>parse null Error.
>
>>Professor,
>>
>>
>>To add to what Renu has said, I went through the source code and I
>>suspect that the value here in this method is passed as null.
>>
>>XmlFilePgeConfigBuilder.java
>>
>>1. private String fillIn(String value,Metadata inputMetadata,boolean
>>envReplaceRecur)
>>2. throwsException {
>>3. try{
>>4. while((value =PathUtils.doDynamicReplacement(value,
>>inputMetadata)).contains("[")&& envReplaceRecur);
>>5. if(value.toUpperCase()
>>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
>>7. value
>>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
>>URL(inputMetadata
>>8.
>>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.p
>>a
>>rseSqlQueryMethod(value)));
>>9. return value;
>>10. }catch(Exception e){
>>11. thrownew Exception("Failed to parse value: "+ value, e);
>>12. }
>>13. }
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
>><ka...@usc.edu> wrote:
>>
>>Hello Professor,
>>
>>
>>All the environment variables present in tasks.xml i.e.
>>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
>>
>>
>>Please let us know how to proceed.
>>
>>
>>Regards,
>>Renu
>>
>>
>>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
>><ma...@usc.edu> wrote:
>>
>>Can you please confirm that all of the bracketed environment
>>variables below do in fact exist in the shell that you are
>>running ./oodt start in?
>>
>>E.g., please echo $ENV_VAR below and show their values.
>>
>>Cheers,
>>Chris
>>
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>Chris Mattmann, Ph.D.
>>Adjunct Associate Professor, Computer Science Department
>>University of Southern California
>>Los Angeles, CA 90089 USA
>>Email: mattmann@usc.edu
>>WWW: http://sunset.usc.edu/~mattmann/
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>-----Original Message-----
>>From: Renu Kanakamedala <ka...@usc.edu>
>>Date: Wednesday, November 5, 2014 at 8:18 PM
>>To: Chris Mattmann <ma...@usc.edu>
>>Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
>><pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>parse null Error.
>>
>>>Please find below tasks.xml file.
>>>
>>>
>>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
>>><!--
>>> TODO: Add some examples
>>>-->
>>><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>>>
>>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGE
>>>T
>>>a
>>>sk">
>>>
>>> <conditions/>
>>>
>>> <configuration>
>>> <property name="PGETask_Name" value="FileConcatenator"/>
>>> <property name="PGETask_ConfigFilePath"
>>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>>>envReplace="true"/>
>>> <property name="PGETask_DumpMetadata" value="true"/>
>>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>>>envReplace="true" />
>>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
>>>envReplace="true"/>
>>> <property name="PCS_MetFileExtension" value="met"/>
>>> <property name="PCS_ClientTransferServiceFactory"
>>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory
>>>"
>>>/
>>>>
>>> <property name="PCS_ActionRepoFile"
>>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml"
>>>envReplace="true"/>
>>> </configuration>
>>>
>>> <requiredMetFields>
>>> <metfield name="RunID"/>
>>> </requiredMetFields>
>>>
>>></task>
>>></cas:tasks>
>>>
>>>
>>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
>>><ma...@usc.edu> wrote:
>>>
>>>Hi Renu,
>>>
>>>Can you please show a paste.bin of your tasks.xml?
>>>
>>>Cheers,
>>>Chris
>>>
>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>Chris Mattmann, Ph.D.
>>>Adjunct Associate Professor, Computer Science Department
>>>University of Southern California
>>>Los Angeles, CA 90089 USA
>>>Email: mattmann@usc.edu
>>>WWW: http://sunset.usc.edu/~mattmann/
>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>>
>>>
>>>
>>>-----Original Message-----
>>>From: Renu Kanakamedala <ka...@usc.edu>
>>>Date: Wednesday, November 5, 2014 at 7:55 PM
>>>To: Chris Mattmann <ma...@usc.edu>
>>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>>parse
>>>null Error.
>>>
>>>>Hello Professor,
>>
>>>>
>>>>
>>>>When i am trying to run file concatenator CAS PGE example using the
>>>>following command.
>>>>
>>>>
>>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>>>>
>>>>
>>>>
>>>>The message shown after executing this command is "Sending Event
>>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
>>>>stub.
>>>>
>>>>
>>>>Batch Stub:
>>>>Caused by:
>>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExc
>>>>e
>>>>p
>>>>t
>>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>>>>value: null
>>>>at
>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>>>>at
>>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>>>>... 2 more
>>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>>>>parse value: null
>>>>at
>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge
>>>>C
>>>>o
>>>>n
>>>>figBuilder.java:87)
>>>>at
>>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance
>>>>.
>>>>j
>>>>a
>>>>va:230)
>>>>at
>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>>>>... 3 more
>>>>Caused by: java.lang.Exception: Failed to parse value: null
>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>>>>at
>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge
>>>>C
>>>>o
>>>>n
>>>>figBuilder.java:77)
>>>>... 5 more
>>>>Caused by: java.lang.NullPointerException
>>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>>>>at java.util.regex.Matcher.reset(Matcher.java:308)
>>>>at java.util.regex.Matcher.<init>(Matcher.java:228)
>>>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>>>>at
>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplac
>>>>e
>>>>m
>>>>e
>>>>nt(PathUtils.java:321)
>>>>at
>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUt
>>>>i
>>>>l
>>>>s
>>>>.java:96)
>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>>>>... 7 more
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>RES mngr:
>>>>
>>>>
>>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>>>>addJob
>>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>>>>Nov 05, 2014 5:26:30 PM
>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>>>>Scheduling for execution
>>>>Nov 05, 2014 5:26:30 PM
>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>plainConnect
>>>>FINEST: ProxySelector Request for
>>>
>>>
>>
>>
>>>>http://localhost:2001/ <http://localhost:2001/>
>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>plainConnect
>>>>FINEST: Proxy used: DIRECT
>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>writeRequests
>>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>>> q=.2}{Connection: keep-alive}{Content-Length: 105}
>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>getInputStream
>>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>>text/xml}{Content-Length: 121}
>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>plainConnect
>>>>FINEST: ProxySelector Request for
>>>>http://localhost:2001/ <http://localhost:2001/>
>>
>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>plainConnect
>>>>FINEST: Proxy used: DIRECT
>>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>>>>writeRequests
>>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>>>>getInputStream
>>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>>text/xml}{Content-Length: 121}
>>>>Nov 05, 2014 5:26:32 PM
>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00'
>>>>:
>>>>batchstub.executeJob returned false
>>>>java.lang.Exception: batchstub.executeJob returned false
>>>>at
>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBat
>>>>c
>>>>h
>>>>M
>>>>grProxy.java:125)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>Workflow:
>>>>
>>>>
>>>>Nov 05, 2014 5:26:27 PM
>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>>checkTaskRequiredMetadata
>>>>INFO: All required metadata fields present for task: [FileConcatenator]
>>>>Nov 05, 2014 5:26:28 PM
>>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>>>>submitJob
>>>>FINEST: [{job.queueName=high,
>>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>>>
>>>
>>
>>
>>
>>
>>>>job.name <http://job.name> <http://job.name> <http://job.name>
>>>><http://job.name>=urn:oodt:FileConcatenator,
>
>
>>>>job.id <http://job.id> <http://job.id> <http://job.id>
>>>><http://job.id>=, job.status=,
>>>>job.load=2,
>>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.File
>>>>C
>>>>o
>>>>n
>>>>catenatorPGETask,
>>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/
>>>>P
>>>>G
>>>>E
>>>>Config.xml,
>>>>
>>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransf
>>>>e
>>>>r
>>>>.
>>>>LocalDataTransferFactory,
>>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.x
>>>>m
>>>>l
>>>>,
>>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>>>>PCS_WorkflowManagerUrl=http://localhost:9001,
>>>> PCS_FileManagerUrl=http://localhost:9000,
>>>>PGETask_Name=FileConcatenator},
>>>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>>>>WorkflowManagerUrl=[http://ubuntu:9001],
>>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>>>> ProcessingNode=[ubuntu], RunID=[testNumber1],
>>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>>>>Nov 05, 2014 5:26:39 PM
>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>>stop
>>>>WARNING: Attempt to kill current resmgr job:
>>>>[2014-11-05T17:26:28.785-08:00]: failed
>>>>
>>>>
>>>>When i searched online i found one of your mail archives where you had
>>>>mentioned to change tasks.xml
>>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>>>>
>>>>
>>>>
>>>>It did not work. Please let us know how to proceed.
>>>>
>>>>
>>>>
>>>>Regards,
>>>>Renu
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>--
>
>
>>Meghamsh UtkurGraduate Student,University of Southern California
>>utkur@usc.edu
>>
>>(503)881-4791 <tel:%28503%29881-4791>
>>
>>
>>
>
>
>
>
>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Renu Kanakamedala <ka...@usc.edu>.
renu@ubuntu:~$ echo $WORKFLOW_HOME
/home/renu/oodt/workflow/
renu@ubuntu:~$ echo $FILEMGR_HOME
/home/renu/oodt/filemgr
renu@ubuntu:~$ echo $WORKFLOW_URL
http://localhost:9001
renu@ubuntu:~$ echo $FILEMGR_URL
http://localhost:9000
renu@ubuntu:~$ echo $PGE_ROOT
/home/renu/oodt/pge/
renu@ubuntu:~$ echo $CRAWLER_HOME
/home/renu/oodt/crawler/
On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann <ma...@usc.edu>
wrote:
> Right, which is why I asked to specifically echo the value
> of all environment variables so we would be able to see the
> values. Can Renu please show the echo values?
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Adjunct Associate Professor, Computer Science Department
> University of Southern California
> Los Angeles, CA 90089 USA
> Email: mattmann@usc.edu
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
> -----Original Message-----
> From: Meghamsh Utkur <ut...@usc.edu>
> Date: Wednesday, November 5, 2014 at 9:35 PM
> To: Renu Kanakamedala <ka...@usc.edu>
> Cc: Chris Mattmann <ma...@usc.edu>, "dev@oodt.apache.org"
> <de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
> Kondakindi <ko...@usc.edu>
> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> parse null Error.
>
> >Professor,
> >
> >
> >To add to what Renu has said, I went through the source code and I
> >suspect that the value here in this method is passed as null.
> >
> >XmlFilePgeConfigBuilder.java
> >
> >1. private String fillIn(String value,Metadata inputMetadata,boolean
> >envReplaceRecur)
> >2. throwsException {
> >3. try{
> >4. while((value =PathUtils.doDynamicReplacement(value,
> >inputMetadata)).contains("[")&& envReplaceRecur);
> >5. if(value.toUpperCase()
> >6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
> >7. value
> >=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
> >URL(inputMetadata
> >8.
> >.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.pa
> >rseSqlQueryMethod(value)));
> >9. return value;
> >10. }catch(Exception e){
> >11. thrownew Exception("Failed to parse value: "+ value, e);
> >12. }
> >13. }
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
> ><ka...@usc.edu> wrote:
> >
> >Hello Professor,
> >
> >
> >All the environment variables present in tasks.xml i.e.
> >$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
> >
> >
> >Please let us know how to proceed.
> >
> >
> >Regards,
> >Renu
> >
> >
> >On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
> ><ma...@usc.edu> wrote:
> >
> >Can you please confirm that all of the bracketed environment
> >variables below do in fact exist in the shell that you are
> >running ./oodt start in?
> >
> >E.g., please echo $ENV_VAR below and show their values.
> >
> >Cheers,
> >Chris
> >
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >Chris Mattmann, Ph.D.
> >Adjunct Associate Professor, Computer Science Department
> >University of Southern California
> >Los Angeles, CA 90089 USA
> >Email: mattmann@usc.edu
> >WWW: http://sunset.usc.edu/~mattmann/
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> >
> >
> >
> >-----Original Message-----
> >From: Renu Kanakamedala <ka...@usc.edu>
> >Date: Wednesday, November 5, 2014 at 8:18 PM
> >To: Chris Mattmann <ma...@usc.edu>
> >Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
> ><pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
> >Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >parse null Error.
> >
> >>Please find below tasks.xml file.
> >>
> >>
> >><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
> >><!--
> >> TODO: Add some examples
> >>-->
> >><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
> >>
> >>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGET
> >>a
> >>sk">
> >>
> >> <conditions/>
> >>
> >> <configuration>
> >> <property name="PGETask_Name" value="FileConcatenator"/>
> >> <property name="PGETask_ConfigFilePath"
> >>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
> >>envReplace="true"/>
> >> <property name="PGETask_DumpMetadata" value="true"/>
> >> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
> >>envReplace="true" />
> >> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
> >>envReplace="true"/>
> >> <property name="PCS_MetFileExtension" value="met"/>
> >> <property name="PCS_ClientTransferServiceFactory"
> >>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory"
> >>/
> >>>
> >> <property name="PCS_ActionRepoFile"
> >>value="file:[CRAWLER_HOME]/policy/crawler-config.xml" envReplace="true"/>
> >> </configuration>
> >>
> >> <requiredMetFields>
> >> <metfield name="RunID"/>
> >> </requiredMetFields>
> >>
> >></task>
> >></cas:tasks>
> >>
> >>
> >>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
> >><ma...@usc.edu> wrote:
> >>
> >>Hi Renu,
> >>
> >>Can you please show a paste.bin of your tasks.xml?
> >>
> >>Cheers,
> >>Chris
> >>
> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>Chris Mattmann, Ph.D.
> >>Adjunct Associate Professor, Computer Science Department
> >>University of Southern California
> >>Los Angeles, CA 90089 USA
> >>Email: mattmann@usc.edu
> >>WWW: http://sunset.usc.edu/~mattmann/
> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>
> >>
> >>
> >>
> >>-----Original Message-----
> >>From: Renu Kanakamedala <ka...@usc.edu>
> >>Date: Wednesday, November 5, 2014 at 7:55 PM
> >>To: Chris Mattmann <ma...@usc.edu>
> >>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
> >>null Error.
> >>
> >>>Hello Professor,
> >
> >>>
> >>>
> >>>When i am trying to run file concatenator CAS PGE example using the
> >>>following command.
> >>>
> >>>
> >>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
> >>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
> >>>
> >>>
> >>>
> >>>The message shown after executing this command is "Sending Event
> >>>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
> >>>stub.
> >>>
> >>>
> >>>Batch Stub:
> >>>Caused by:
> >>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExce
> >>>p
> >>>t
> >>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
> >>>value: null
> >>>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> >>>at
> >>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> >>>... 2 more
> >>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
> >>>parse value: null
> >>>at
> >>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
> >>>o
> >>>n
> >>>figBuilder.java:87)
> >>>at
> >>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.
> >>>j
> >>>a
> >>>va:230)
> >>>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> >>>... 3 more
> >>>Caused by: java.lang.Exception: Failed to parse value: null
> >>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> >>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> >>>at
> >>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
> >>>o
> >>>n
> >>>figBuilder.java:77)
> >>>... 5 more
> >>>Caused by: java.lang.NullPointerException
> >>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> >>>at java.util.regex.Matcher.reset(Matcher.java:308)
> >>>at java.util.regex.Matcher.<init>(Matcher.java:228)
> >>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
> >>>at
> >>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplace
> >>>m
> >>>e
> >>>nt(PathUtils.java:321)
> >>>at
> >>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUti
> >>>l
> >>>s
> >>>.java:96)
> >>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
> >>>... 7 more
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>RES mngr:
> >>>
> >>>
> >>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
> >>>addJob
> >>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
> >>>Nov 05, 2014 5:26:30 PM
> >>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
> >>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
> >>>Scheduling for execution
> >>>Nov 05, 2014 5:26:30 PM
> >>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
> >>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>plainConnect
> >>>FINEST: ProxySelector Request for
> >>
> >>
> >
> >
> >>>http://localhost:2001/ <http://localhost:2001/>
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>plainConnect
> >>>FINEST: Proxy used: DIRECT
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>writeRequests
> >>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
> >>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >>> q=.2}{Connection: keep-alive}{Content-Length: 105}
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>getInputStream
> >>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
> >>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>>text/xml}{Content-Length: 121}
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>plainConnect
> >>>FINEST: ProxySelector Request for
> >>>http://localhost:2001/ <http://localhost:2001/>
> >
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>plainConnect
> >>>FINEST: Proxy used: DIRECT
> >>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
> >>>writeRequests
> >>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
> >>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
> >>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
> >>>getInputStream
> >>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
> >>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>>text/xml}{Content-Length: 121}
> >>>Nov 05, 2014 5:26:32 PM
> >>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
> >>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
> >>>batchstub.executeJob returned false
> >>>java.lang.Exception: batchstub.executeJob returned false
> >>>at
> >>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatc
> >>>h
> >>>M
> >>>grProxy.java:125)
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>Workflow:
> >>>
> >>>
> >>>Nov 05, 2014 5:26:27 PM
> >>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>>checkTaskRequiredMetadata
> >>>INFO: All required metadata fields present for task: [FileConcatenator]
> >>>Nov 05, 2014 5:26:28 PM
> >>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
> >>>submitJob
> >>>FINEST: [{job.queueName=high,
> >>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
> >>
> >>
> >
> >
> >
> >
> >>>job.name <http://job.name> <http://job.name>
> >>><http://job.name>=urn:oodt:FileConcatenator,
> >>>job.id <http://job.id> <http://job.id> <http://job.id>=, job.status=,
> >>>job.load=2,
> >>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
> >>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileC
> >>>o
> >>>n
> >>>catenatorPGETask,
> >>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/P
> >>>G
> >>>E
> >>>Config.xml,
> >>>
> >>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfe
> >>>r
> >>>.
> >>>LocalDataTransferFactory,
> >>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xm
> >>>l
> >>>,
> >>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
> >>>PCS_WorkflowManagerUrl=http://localhost:9001,
> >>> PCS_FileManagerUrl=http://localhost:9000,
> >>>PGETask_Name=FileConcatenator},
> >>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
> >>>WorkflowManagerUrl=[http://ubuntu:9001],
> >>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
> >>> ProcessingNode=[ubuntu], RunID=[testNumber1],
> >>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
> >>>Nov 05, 2014 5:26:39 PM
> >>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>>stop
> >>>WARNING: Attempt to kill current resmgr job:
> >>>[2014-11-05T17:26:28.785-08:00]: failed
> >>>
> >>>
> >>>When i searched online i found one of your mail archives where you had
> >>>mentioned to change tasks.xml
> >>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
> >>>
> >>>
> >>>
> >>>It did not work. Please let us know how to proceed.
> >>>
> >>>
> >>>
> >>>Regards,
> >>>Renu
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >--
> >Meghamsh UtkurGraduate Student,University of Southern California
> >utkur@usc.edu
> >
> >(503)881-4791
> >
> >
> >
>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Christian Alan Mattmann <ma...@usc.edu>.
Right, which is why I asked to specifically echo the value
of all environment variables so we would be able to see the
values. Can Renu please show the echo values?
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Adjunct Associate Professor, Computer Science Department
University of Southern California
Los Angeles, CA 90089 USA
Email: mattmann@usc.edu
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Meghamsh Utkur <ut...@usc.edu>
Date: Wednesday, November 5, 2014 at 9:35 PM
To: Renu Kanakamedala <ka...@usc.edu>
Cc: Chris Mattmann <ma...@usc.edu>, "dev@oodt.apache.org"
<de...@oodt.apache.org>, Preethi Ramesh <pr...@usc.edu>, Gouthami
Kondakindi <ko...@usc.edu>
Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
parse null Error.
>Professor,
>
>
>To add to what Renu has said, I went through the source code and I
>suspect that the value here in this method is passed as null.
>
>XmlFilePgeConfigBuilder.java
>
>1. private String fillIn(String value,Metadata inputMetadata,boolean
>envReplaceRecur)
>2. throwsException {
>3. try{
>4. while((value =PathUtils.doDynamicReplacement(value,
>inputMetadata)).contains("[")&& envReplaceRecur);
>5. if(value.toUpperCase()
>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
>7. value
>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
>URL(inputMetadata
>8.
>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.pa
>rseSqlQueryMethod(value)));
>9. return value;
>10. }catch(Exception e){
>11. thrownew Exception("Failed to parse value: "+ value, e);
>12. }
>13. }
>
>
>
>
>
>
>
>
>
>
>
>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
><ka...@usc.edu> wrote:
>
>Hello Professor,
>
>
>All the environment variables present in tasks.xml i.e.
>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
>
>
>Please let us know how to proceed.
>
>
>Regards,
>Renu
>
>
>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
><ma...@usc.edu> wrote:
>
>Can you please confirm that all of the bracketed environment
>variables below do in fact exist in the shell that you are
>running ./oodt start in?
>
>E.g., please echo $ENV_VAR below and show their values.
>
>Cheers,
>Chris
>
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>Chris Mattmann, Ph.D.
>Adjunct Associate Professor, Computer Science Department
>University of Southern California
>Los Angeles, CA 90089 USA
>Email: mattmann@usc.edu
>WWW: http://sunset.usc.edu/~mattmann/
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>-----Original Message-----
>From: Renu Kanakamedala <ka...@usc.edu>
>Date: Wednesday, November 5, 2014 at 8:18 PM
>To: Chris Mattmann <ma...@usc.edu>
>Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
><pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>parse null Error.
>
>>Please find below tasks.xml file.
>>
>>
>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
>><!--
>> TODO: Add some examples
>>-->
>><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>>
>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGET
>>a
>>sk">
>>
>> <conditions/>
>>
>> <configuration>
>> <property name="PGETask_Name" value="FileConcatenator"/>
>> <property name="PGETask_ConfigFilePath"
>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>>envReplace="true"/>
>> <property name="PGETask_DumpMetadata" value="true"/>
>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>>envReplace="true" />
>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
>>envReplace="true"/>
>> <property name="PCS_MetFileExtension" value="met"/>
>> <property name="PCS_ClientTransferServiceFactory"
>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory"
>>/
>>>
>> <property name="PCS_ActionRepoFile"
>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml" envReplace="true"/>
>> </configuration>
>>
>> <requiredMetFields>
>> <metfield name="RunID"/>
>> </requiredMetFields>
>>
>></task>
>></cas:tasks>
>>
>>
>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
>><ma...@usc.edu> wrote:
>>
>>Hi Renu,
>>
>>Can you please show a paste.bin of your tasks.xml?
>>
>>Cheers,
>>Chris
>>
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>Chris Mattmann, Ph.D.
>>Adjunct Associate Professor, Computer Science Department
>>University of Southern California
>>Los Angeles, CA 90089 USA
>>Email: mattmann@usc.edu
>>WWW: http://sunset.usc.edu/~mattmann/
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>-----Original Message-----
>>From: Renu Kanakamedala <ka...@usc.edu>
>>Date: Wednesday, November 5, 2014 at 7:55 PM
>>To: Chris Mattmann <ma...@usc.edu>
>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
>>null Error.
>>
>>>Hello Professor,
>
>>>
>>>
>>>When i am trying to run file concatenator CAS PGE example using the
>>>following command.
>>>
>>>
>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>>>
>>>
>>>
>>>The message shown after executing this command is "Sending Event
>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
>>>stub.
>>>
>>>
>>>Batch Stub:
>>>Caused by:
>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExce
>>>p
>>>t
>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>>>value: null
>>>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>>>at
>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>>>... 2 more
>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>>>parse value: null
>>>at
>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
>>>o
>>>n
>>>figBuilder.java:87)
>>>at
>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.
>>>j
>>>a
>>>va:230)
>>>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>>>... 3 more
>>>Caused by: java.lang.Exception: Failed to parse value: null
>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>>>at
>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
>>>o
>>>n
>>>figBuilder.java:77)
>>>... 5 more
>>>Caused by: java.lang.NullPointerException
>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>>>at java.util.regex.Matcher.reset(Matcher.java:308)
>>>at java.util.regex.Matcher.<init>(Matcher.java:228)
>>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>>>at
>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplace
>>>m
>>>e
>>>nt(PathUtils.java:321)
>>>at
>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUti
>>>l
>>>s
>>>.java:96)
>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>>>... 7 more
>>>
>>>
>>>
>>>
>>>
>>>RES mngr:
>>>
>>>
>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>>>addJob
>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>>>Nov 05, 2014 5:26:30 PM
>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>>>Scheduling for execution
>>>Nov 05, 2014 5:26:30 PM
>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>plainConnect
>>>FINEST: ProxySelector Request for
>>
>>
>
>
>>>http://localhost:2001/ <http://localhost:2001/>
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>plainConnect
>>>FINEST: Proxy used: DIRECT
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>writeRequests
>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>> q=.2}{Connection: keep-alive}{Content-Length: 105}
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>getInputStream
>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>text/xml}{Content-Length: 121}
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>plainConnect
>>>FINEST: ProxySelector Request for
>>>http://localhost:2001/ <http://localhost:2001/>
>
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>plainConnect
>>>FINEST: Proxy used: DIRECT
>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>>>writeRequests
>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>>>getInputStream
>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>text/xml}{Content-Length: 121}
>>>Nov 05, 2014 5:26:32 PM
>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
>>>batchstub.executeJob returned false
>>>java.lang.Exception: batchstub.executeJob returned false
>>>at
>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatc
>>>h
>>>M
>>>grProxy.java:125)
>>>
>>>
>>>
>>>
>>>
>>>Workflow:
>>>
>>>
>>>Nov 05, 2014 5:26:27 PM
>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>checkTaskRequiredMetadata
>>>INFO: All required metadata fields present for task: [FileConcatenator]
>>>Nov 05, 2014 5:26:28 PM
>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>>>submitJob
>>>FINEST: [{job.queueName=high,
>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>>
>>
>
>
>
>
>>>job.name <http://job.name> <http://job.name>
>>><http://job.name>=urn:oodt:FileConcatenator,
>>>job.id <http://job.id> <http://job.id> <http://job.id>=, job.status=,
>>>job.load=2,
>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileC
>>>o
>>>n
>>>catenatorPGETask,
>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/P
>>>G
>>>E
>>>Config.xml,
>>>
>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfe
>>>r
>>>.
>>>LocalDataTransferFactory,
>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xm
>>>l
>>>,
>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>>>PCS_WorkflowManagerUrl=http://localhost:9001,
>>> PCS_FileManagerUrl=http://localhost:9000,
>>>PGETask_Name=FileConcatenator},
>>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>>>WorkflowManagerUrl=[http://ubuntu:9001],
>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>>> ProcessingNode=[ubuntu], RunID=[testNumber1],
>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>>>Nov 05, 2014 5:26:39 PM
>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>stop
>>>WARNING: Attempt to kill current resmgr job:
>>>[2014-11-05T17:26:28.785-08:00]: failed
>>>
>>>
>>>When i searched online i found one of your mail archives where you had
>>>mentioned to change tasks.xml
>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>>>
>>>
>>>
>>>It did not work. Please let us know how to proceed.
>>>
>>>
>>>
>>>Regards,
>>>Renu
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>--
>Meghamsh UtkurGraduate Student,University of Southern California
>utkur@usc.edu
>
>(503)881-4791
>
>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Meghamsh Utkur <ut...@usc.edu>.
Professor,
To add to what Renu has said, I went through the source code and I suspect
that the value here in this method is passed as null.
XmlFilePgeConfigBuilder.java
1. private String fillIn(String value, Metadata inputMetadata, boolean
envReplaceRecur)
2. throws Exception {
3. try {
4. while ((value = PathUtils.doDynamicReplacement(value, inputMetadata)).
contains("[") && envReplaceRecur);
5. if (value.toUpperCase()
6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
7. value = QueryUtils.getQueryResultsAsString(new XmlRpcFileManagerClient
(new URL(inputMetadata
8. .getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(
SqlParser.parseSqlQueryMethod(value)));
9. return value;
10. }catch (Exception e) {
11. throw new Exception("Failed to parse value: " + value, e);
12. }
13. }
On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala <ka...@usc.edu> wrote:
> Hello Professor,
>
> All the environment variables present in tasks.xml i.e.
> $WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
>
> Please let us know how to proceed.
>
> Regards,
> Renu
>
> On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann <ma...@usc.edu>
> wrote:
>
>> Can you please confirm that all of the bracketed environment
>> variables below do in fact exist in the shell that you are
>> running ./oodt start in?
>>
>> E.g., please echo $ENV_VAR below and show their values.
>>
>> Cheers,
>> Chris
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Adjunct Associate Professor, Computer Science Department
>> University of Southern California
>> Los Angeles, CA 90089 USA
>> Email: mattmann@usc.edu
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>> -----Original Message-----
>> From: Renu Kanakamedala <ka...@usc.edu>
>> Date: Wednesday, November 5, 2014 at 8:18 PM
>> To: Chris Mattmann <ma...@usc.edu>
>> Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
>> <pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
>> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>> parse null Error.
>>
>> >Please find below tasks.xml file.
>> >
>> >
>> ><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
>> ><!--
>> > TODO: Add some examples
>> >-->
>> ><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>> >
>>
>> >class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGETa
>> >sk">
>> >
>> > <conditions/>
>> >
>> > <configuration>
>> > <property name="PGETask_Name" value="FileConcatenator"/>
>> > <property name="PGETask_ConfigFilePath"
>> >value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>> >envReplace="true"/>
>> > <property name="PGETask_DumpMetadata" value="true"/>
>> > <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>> >envReplace="true" />
>> > <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
>> >envReplace="true"/>
>> > <property name="PCS_MetFileExtension" value="met"/>
>> > <property name="PCS_ClientTransferServiceFactory"
>>
>> >value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory"/
>> >>
>> > <property name="PCS_ActionRepoFile"
>> >value="file:[CRAWLER_HOME]/policy/crawler-config.xml" envReplace="true"/>
>> > </configuration>
>> >
>> > <requiredMetFields>
>> > <metfield name="RunID"/>
>> > </requiredMetFields>
>> >
>> ></task>
>> ></cas:tasks>
>> >
>> >
>> >On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
>> ><ma...@usc.edu> wrote:
>> >
>> >Hi Renu,
>> >
>> >Can you please show a paste.bin of your tasks.xml?
>> >
>> >Cheers,
>> >Chris
>> >
>> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >Chris Mattmann, Ph.D.
>> >Adjunct Associate Professor, Computer Science Department
>> >University of Southern California
>> >Los Angeles, CA 90089 USA
>> >Email: mattmann@usc.edu
>> >WWW: http://sunset.usc.edu/~mattmann/
>> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >
>> >
>> >
>> >
>> >-----Original Message-----
>> >From: Renu Kanakamedala <ka...@usc.edu>
>> >Date: Wednesday, November 5, 2014 at 7:55 PM
>> >To: Chris Mattmann <ma...@usc.edu>
>> >Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
>> >null Error.
>> >
>> >>Hello Professor,
>>
>> >>
>> >>
>> >>When i am trying to run file concatenator CAS PGE example using the
>> >>following command.
>> >>
>> >>
>> >>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>> >>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>> >>
>> >>
>> >>
>> >>The message shown after executing this command is "Sending Event
>> >>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
>> >>stub.
>> >>
>> >>
>> >>Batch Stub:
>> >>Caused by:
>>
>> >>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcep
>> >>t
>> >>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>> >>value: null
>> >>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>> >>at
>> org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>> >>... 2 more
>> >>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>> >>parse value: null
>> >>at
>>
>> >>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>> >>n
>> >>figBuilder.java:87)
>> >>at
>>
>> >>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.j
>> >>a
>> >>va:230)
>> >>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>> >>... 3 more
>> >>Caused by: java.lang.Exception: Failed to parse value: null
>> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> >>at
>>
>> >>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>> >>n
>> >>figBuilder.java:77)
>> >>... 5 more
>> >>Caused by: java.lang.NullPointerException
>> >>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>> >>at java.util.regex.Matcher.reset(Matcher.java:308)
>> >>at java.util.regex.Matcher.<init>(Matcher.java:228)
>> >>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>> >>at
>>
>> >>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacem
>> >>e
>> >>nt(PathUtils.java:321)
>> >>at
>>
>> >>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtil
>> >>s
>> >>.java:96)
>> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>> >>... 7 more
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>RES mngr:
>> >>
>> >>
>> >>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>> >>addJob
>> >>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>> >>Nov 05, 2014 5:26:30 PM
>> >>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>> >>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>> >>Scheduling for execution
>> >>Nov 05, 2014 5:26:30 PM
>> >>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>> >>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: ProxySelector Request for
>> >
>> >
>> >>http://localhost:2001/ <http://localhost:2001/>
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: Proxy used: DIRECT
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>writeRequests
>> >>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>> >>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>> >>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>> >>text/html, image/gif, image/jpeg, *; q=.2, */*;
>> >> q=.2}{Connection: keep-alive}{Content-Length: 105}
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>getInputStream
>> >>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>> >>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>> >>text/xml}{Content-Length: 121}
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: ProxySelector Request for
>> >>http://localhost:2001/ <http://localhost:2001/>
>>
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: Proxy used: DIRECT
>> >>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>> >>writeRequests
>> >>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>> >>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>> >>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>> >>text/html, image/gif, image/jpeg, *; q=.2, */*;
>> >> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>> >>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>> >>getInputStream
>> >>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>> >>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>> >>text/xml}{Content-Length: 121}
>> >>Nov 05, 2014 5:26:32 PM
>> >>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>> >>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
>> >>batchstub.executeJob returned false
>> >>java.lang.Exception: batchstub.executeJob returned false
>> >>at
>>
>> >>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatch
>> >>M
>> >>grProxy.java:125)
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>Workflow:
>> >>
>> >>
>> >>Nov 05, 2014 5:26:27 PM
>> >>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> >>checkTaskRequiredMetadata
>> >>INFO: All required metadata fields present for task: [FileConcatenator]
>> >>Nov 05, 2014 5:26:28 PM
>> >>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>> submitJob
>> >>FINEST: [{job.queueName=high,
>> >>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>> >
>> >
>> >>job.name <http://job.name> <http://job.name>=urn:oodt:FileConcatenator,
>> >>job.id <http://job.id> <http://job.id>=, job.status=, job.load=2,
>> >>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>>
>> >>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileCo
>> >>n
>> >>catenatorPGETask,
>>
>> >>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/PG
>> >>E
>> >>Config.xml,
>> >>
>>
>> >>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer
>> >>.
>> >>LocalDataTransferFactory,
>>
>> >>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xml
>> >>,
>> >> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>> >>PCS_WorkflowManagerUrl=http://localhost:9001,
>> >> PCS_FileManagerUrl=http://localhost:9000,
>> >>PGETask_Name=FileConcatenator},
>> >>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>> >>WorkflowManagerUrl=[http://ubuntu:9001],
>> >>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>> >> ProcessingNode=[ubuntu], RunID=[testNumber1],
>> >>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>> >>Nov 05, 2014 5:26:39 PM
>> >>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> stop
>> >>WARNING: Attempt to kill current resmgr job:
>> >>[2014-11-05T17:26:28.785-08:00]: failed
>> >>
>> >>
>> >>When i searched online i found one of your mail archives where you had
>> >>mentioned to change tasks.xml
>> >>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>> >>
>> >>
>> >>
>> >>It did not work. Please let us know how to proceed.
>> >>
>> >>
>> >>
>> >>Regards,
>> >>Renu
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>
--
*Meghamsh Utkur*
*Graduate Student,University of Southern California*
*utkur@usc.edu <ut...@usc.edu>*
*(503)881-4791*
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
parse null Error.
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
Can you please confirm via paste.bin the contents of your wmgr and
batch_stub scripts?
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Renu Kanakamedala <ka...@usc.edu>
Reply-To: "dev@oodt.apache.org" <de...@oodt.apache.org>
Date: Wednesday, November 5, 2014 at 9:30 PM
To: Chris Mattmann <ma...@usc.edu>
Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
<pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>, "utkur@usc.edu"
<ut...@usc.edu>
Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
parse null Error.
>Hello Professor,
>
>All the environment variables present in tasks.xml i.e.
>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
>
>Please let us know how to proceed.
>
>Regards,
>Renu
>
>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann <ma...@usc.edu>
>wrote:
>
>> Can you please confirm that all of the bracketed environment
>> variables below do in fact exist in the shell that you are
>> running ./oodt start in?
>>
>> E.g., please echo $ENV_VAR below and show their values.
>>
>> Cheers,
>> Chris
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Adjunct Associate Professor, Computer Science Department
>> University of Southern California
>> Los Angeles, CA 90089 USA
>> Email: mattmann@usc.edu
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>> -----Original Message-----
>> From: Renu Kanakamedala <ka...@usc.edu>
>> Date: Wednesday, November 5, 2014 at 8:18 PM
>> To: Chris Mattmann <ma...@usc.edu>
>> Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
>> <pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
>> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>> parse null Error.
>>
>> >Please find below tasks.xml file.
>> >
>> >
>> ><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
>> ><!--
>> > TODO: Add some examples
>> >-->
>> ><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>> >
>>
>>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGE
>>>Ta
>> >sk">
>> >
>> > <conditions/>
>> >
>> > <configuration>
>> > <property name="PGETask_Name" value="FileConcatenator"/>
>> > <property name="PGETask_ConfigFilePath"
>> >value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>> >envReplace="true"/>
>> > <property name="PGETask_DumpMetadata" value="true"/>
>> > <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>> >envReplace="true" />
>> > <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
>> >envReplace="true"/>
>> > <property name="PCS_MetFileExtension" value="met"/>
>> > <property name="PCS_ClientTransferServiceFactory"
>>
>>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory
>>>"/
>> >>
>> > <property name="PCS_ActionRepoFile"
>> >value="file:[CRAWLER_HOME]/policy/crawler-config.xml"
>>envReplace="true"/>
>> > </configuration>
>> >
>> > <requiredMetFields>
>> > <metfield name="RunID"/>
>> > </requiredMetFields>
>> >
>> ></task>
>> ></cas:tasks>
>> >
>> >
>> >On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
>> ><ma...@usc.edu> wrote:
>> >
>> >Hi Renu,
>> >
>> >Can you please show a paste.bin of your tasks.xml?
>> >
>> >Cheers,
>> >Chris
>> >
>> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >Chris Mattmann, Ph.D.
>> >Adjunct Associate Professor, Computer Science Department
>> >University of Southern California
>> >Los Angeles, CA 90089 USA
>> >Email: mattmann@usc.edu
>> >WWW: http://sunset.usc.edu/~mattmann/
>> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >
>> >
>> >
>> >
>> >-----Original Message-----
>> >From: Renu Kanakamedala <ka...@usc.edu>
>> >Date: Wednesday, November 5, 2014 at 7:55 PM
>> >To: Chris Mattmann <ma...@usc.edu>
>> >Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>parse
>> >null Error.
>> >
>> >>Hello Professor,
>> >>
>> >>
>> >>When i am trying to run file concatenator CAS PGE example using the
>> >>following command.
>> >>
>> >>
>> >>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>> >>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>> >>
>> >>
>> >>
>> >>The message shown after executing this command is "Sending Event
>> >>SUCCESS". Below are the log messages in workflow mgr, res mgr and
>>batch
>> >>stub.
>> >>
>> >>
>> >>Batch Stub:
>> >>Caused by:
>>
>>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExc
>>>>ep
>> >>t
>> >>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>> >>value: null
>> >>at
>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>> >>at
>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>> >>... 2 more
>> >>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>> >>parse value: null
>> >>at
>>
>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge
>>>>Co
>> >>n
>> >>figBuilder.java:87)
>> >>at
>>
>>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance
>>>>.j
>> >>a
>> >>va:230)
>> >>at
>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>> >>... 3 more
>> >>Caused by: java.lang.Exception: Failed to parse value: null
>> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> >>at
>>
>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge
>>>>Co
>> >>n
>> >>figBuilder.java:77)
>> >>... 5 more
>> >>Caused by: java.lang.NullPointerException
>> >>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>> >>at java.util.regex.Matcher.reset(Matcher.java:308)
>> >>at java.util.regex.Matcher.<init>(Matcher.java:228)
>> >>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>> >>at
>>
>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplac
>>>>em
>> >>e
>> >>nt(PathUtils.java:321)
>> >>at
>>
>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUt
>>>>il
>> >>s
>> >>.java:96)
>> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>> >>... 7 more
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>RES mngr:
>> >>
>> >>
>> >>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>> >>addJob
>> >>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>> >>Nov 05, 2014 5:26:30 PM
>> >>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>> >>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>> >>Scheduling for execution
>> >>Nov 05, 2014 5:26:30 PM
>> >>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>> >>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: ProxySelector Request for
>> >
>> >
>> >>http://localhost:2001/ <http://localhost:2001/>
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: Proxy used: DIRECT
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>writeRequests
>> >>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>> >>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>> >>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>> >>text/html, image/gif, image/jpeg, *; q=.2, */*;
>> >> q=.2}{Connection: keep-alive}{Content-Length: 105}
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>getInputStream
>> >>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>> >>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>> >>text/xml}{Content-Length: 121}
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: ProxySelector Request for
>> >>http://localhost:2001/ <http://localhost:2001/>
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: Proxy used: DIRECT
>> >>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>> >>writeRequests
>> >>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>> >>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>> >>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>> >>text/html, image/gif, image/jpeg, *; q=.2, */*;
>> >> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>> >>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>> >>getInputStream
>> >>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>> >>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>> >>text/xml}{Content-Length: 121}
>> >>Nov 05, 2014 5:26:32 PM
>> >>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>> >>SEVERE: Job execution failed for jobId
>>'2014-11-05T17:26:28.785-08:00' :
>> >>batchstub.executeJob returned false
>> >>java.lang.Exception: batchstub.executeJob returned false
>> >>at
>>
>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBat
>>>>ch
>> >>M
>> >>grProxy.java:125)
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>Workflow:
>> >>
>> >>
>> >>Nov 05, 2014 5:26:27 PM
>> >>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> >>checkTaskRequiredMetadata
>> >>INFO: All required metadata fields present for task:
>>[FileConcatenator]
>> >>Nov 05, 2014 5:26:28 PM
>> >>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>>submitJob
>> >>FINEST: [{job.queueName=high,
>> >>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>> >
>> >
>> >>job.name <http://job.name>
>><http://job.name>=urn:oodt:FileConcatenator,
>> >>job.id <http://job.id> <http://job.id>=, job.status=, job.load=2,
>> >>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>>
>>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.File
>>>>Co
>> >>n
>> >>catenatorPGETask,
>>
>>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/
>>>>PG
>> >>E
>> >>Config.xml,
>> >>
>>
>>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransf
>>>>er
>> >>.
>> >>LocalDataTransferFactory,
>>
>>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.x
>>>>ml
>> >>,
>> >> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>> >>PCS_WorkflowManagerUrl=http://localhost:9001,
>> >> PCS_FileManagerUrl=http://localhost:9000,
>> >>PGETask_Name=FileConcatenator},
>> >>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>> >>WorkflowManagerUrl=[http://ubuntu:9001],
>> >>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>> >> ProcessingNode=[ubuntu], RunID=[testNumber1],
>> >>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>> >>Nov 05, 2014 5:26:39 PM
>> >>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>stop
>> >>WARNING: Attempt to kill current resmgr job:
>> >>[2014-11-05T17:26:28.785-08:00]: failed
>> >>
>> >>
>> >>When i searched online i found one of your mail archives where you had
>> >>mentioned to change tasks.xml
>> >>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>> >>
>> >>
>> >>
>> >>It did not work. Please let us know how to proceed.
>> >>
>> >>
>> >>
>> >>Regards,
>> >>Renu
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Renu Kanakamedala <ka...@usc.edu>.
Hello Professor,
All the environment variables present in tasks.xml i.e.
$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
Please let us know how to proceed.
Regards,
Renu
On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann <ma...@usc.edu>
wrote:
> Can you please confirm that all of the bracketed environment
> variables below do in fact exist in the shell that you are
> running ./oodt start in?
>
> E.g., please echo $ENV_VAR below and show their values.
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Adjunct Associate Professor, Computer Science Department
> University of Southern California
> Los Angeles, CA 90089 USA
> Email: mattmann@usc.edu
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
> -----Original Message-----
> From: Renu Kanakamedala <ka...@usc.edu>
> Date: Wednesday, November 5, 2014 at 8:18 PM
> To: Chris Mattmann <ma...@usc.edu>
> Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
> <pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> parse null Error.
>
> >Please find below tasks.xml file.
> >
> >
> ><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
> ><!--
> > TODO: Add some examples
> >-->
> ><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
> >
> >class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGETa
> >sk">
> >
> > <conditions/>
> >
> > <configuration>
> > <property name="PGETask_Name" value="FileConcatenator"/>
> > <property name="PGETask_ConfigFilePath"
> >value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
> >envReplace="true"/>
> > <property name="PGETask_DumpMetadata" value="true"/>
> > <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
> >envReplace="true" />
> > <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
> >envReplace="true"/>
> > <property name="PCS_MetFileExtension" value="met"/>
> > <property name="PCS_ClientTransferServiceFactory"
> >value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory"/
> >>
> > <property name="PCS_ActionRepoFile"
> >value="file:[CRAWLER_HOME]/policy/crawler-config.xml" envReplace="true"/>
> > </configuration>
> >
> > <requiredMetFields>
> > <metfield name="RunID"/>
> > </requiredMetFields>
> >
> ></task>
> ></cas:tasks>
> >
> >
> >On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
> ><ma...@usc.edu> wrote:
> >
> >Hi Renu,
> >
> >Can you please show a paste.bin of your tasks.xml?
> >
> >Cheers,
> >Chris
> >
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >Chris Mattmann, Ph.D.
> >Adjunct Associate Professor, Computer Science Department
> >University of Southern California
> >Los Angeles, CA 90089 USA
> >Email: mattmann@usc.edu
> >WWW: http://sunset.usc.edu/~mattmann/
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> >
> >
> >
> >-----Original Message-----
> >From: Renu Kanakamedala <ka...@usc.edu>
> >Date: Wednesday, November 5, 2014 at 7:55 PM
> >To: Chris Mattmann <ma...@usc.edu>
> >Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
> >null Error.
> >
> >>Hello Professor,
> >>
> >>
> >>When i am trying to run file concatenator CAS PGE example using the
> >>following command.
> >>
> >>
> >>./wmgr-client --url http://localhost:9001 --operation --sendEvent
> >>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
> >>
> >>
> >>
> >>The message shown after executing this command is "Sending Event
> >>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
> >>stub.
> >>
> >>
> >>Batch Stub:
> >>Caused by:
> >>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcep
> >>t
> >>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
> >>value: null
> >>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> >>at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> >>... 2 more
> >>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
> >>parse value: null
> >>at
> >>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
> >>n
> >>figBuilder.java:87)
> >>at
> >>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.j
> >>a
> >>va:230)
> >>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> >>... 3 more
> >>Caused by: java.lang.Exception: Failed to parse value: null
> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> >>at
> >>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
> >>n
> >>figBuilder.java:77)
> >>... 5 more
> >>Caused by: java.lang.NullPointerException
> >>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> >>at java.util.regex.Matcher.reset(Matcher.java:308)
> >>at java.util.regex.Matcher.<init>(Matcher.java:228)
> >>at java.util.regex.Pattern.matcher(Pattern.java:1088)
> >>at
> >>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacem
> >>e
> >>nt(PathUtils.java:321)
> >>at
> >>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtil
> >>s
> >>.java:96)
> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
> >>... 7 more
> >>
> >>
> >>
> >>
> >>
> >>RES mngr:
> >>
> >>
> >>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
> >>addJob
> >>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
> >>Nov 05, 2014 5:26:30 PM
> >>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
> >>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
> >>Scheduling for execution
> >>Nov 05, 2014 5:26:30 PM
> >>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
> >>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>plainConnect
> >>FINEST: ProxySelector Request for
> >
> >
> >>http://localhost:2001/ <http://localhost:2001/>
> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>plainConnect
> >>FINEST: Proxy used: DIRECT
> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>writeRequests
> >>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
> >>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >> q=.2}{Connection: keep-alive}{Content-Length: 105}
> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>getInputStream
> >>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
> >>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>text/xml}{Content-Length: 121}
> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>plainConnect
> >>FINEST: ProxySelector Request for
> >>http://localhost:2001/ <http://localhost:2001/>
> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>plainConnect
> >>FINEST: Proxy used: DIRECT
> >>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
> >>writeRequests
> >>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
> >>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >> q=.2}{Connection: keep-alive}{Content-Length: 2536}
> >>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
> >>getInputStream
> >>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
> >>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>text/xml}{Content-Length: 121}
> >>Nov 05, 2014 5:26:32 PM
> >>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
> >>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
> >>batchstub.executeJob returned false
> >>java.lang.Exception: batchstub.executeJob returned false
> >>at
> >>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatch
> >>M
> >>grProxy.java:125)
> >>
> >>
> >>
> >>
> >>
> >>Workflow:
> >>
> >>
> >>Nov 05, 2014 5:26:27 PM
> >>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>checkTaskRequiredMetadata
> >>INFO: All required metadata fields present for task: [FileConcatenator]
> >>Nov 05, 2014 5:26:28 PM
> >>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient submitJob
> >>FINEST: [{job.queueName=high,
> >>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
> >
> >
> >>job.name <http://job.name> <http://job.name>=urn:oodt:FileConcatenator,
> >>job.id <http://job.id> <http://job.id>=, job.status=, job.load=2,
> >>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
> >>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileCo
> >>n
> >>catenatorPGETask,
> >>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/PG
> >>E
> >>Config.xml,
> >>
> >>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer
> >>.
> >>LocalDataTransferFactory,
> >>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xml
> >>,
> >> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
> >>PCS_WorkflowManagerUrl=http://localhost:9001,
> >> PCS_FileManagerUrl=http://localhost:9000,
> >>PGETask_Name=FileConcatenator},
> >>task.metadata={TaskId=[urn:oodt:FileConcatenator],
> >>WorkflowManagerUrl=[http://ubuntu:9001],
> >>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
> >> ProcessingNode=[ubuntu], RunID=[testNumber1],
> >>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
> >>Nov 05, 2014 5:26:39 PM
> >>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread stop
> >>WARNING: Attempt to kill current resmgr job:
> >>[2014-11-05T17:26:28.785-08:00]: failed
> >>
> >>
> >>When i searched online i found one of your mail archives where you had
> >>mentioned to change tasks.xml
> >>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
> >>
> >>
> >>
> >>It did not work. Please let us know how to proceed.
> >>
> >>
> >>
> >>Regards,
> >>Renu
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Christian Alan Mattmann <ma...@usc.edu>.
Can you please confirm that all of the bracketed environment
variables below do in fact exist in the shell that you are
running ./oodt start in?
E.g., please echo $ENV_VAR below and show their values.
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Adjunct Associate Professor, Computer Science Department
University of Southern California
Los Angeles, CA 90089 USA
Email: mattmann@usc.edu
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Renu Kanakamedala <ka...@usc.edu>
Date: Wednesday, November 5, 2014 at 8:18 PM
To: Chris Mattmann <ma...@usc.edu>
Cc: "dev@oodt.apache.org" <de...@oodt.apache.org>, Preethi Ramesh
<pr...@usc.edu>, Gouthami Kondakindi <ko...@usc.edu>
Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
parse null Error.
>Please find below tasks.xml file.
>
>
><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
><!--
> TODO: Add some examples
>-->
><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>
>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGETa
>sk">
>
> <conditions/>
>
> <configuration>
> <property name="PGETask_Name" value="FileConcatenator"/>
> <property name="PGETask_ConfigFilePath"
>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>envReplace="true"/>
> <property name="PGETask_DumpMetadata" value="true"/>
> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>envReplace="true" />
> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
>envReplace="true"/>
> <property name="PCS_MetFileExtension" value="met"/>
> <property name="PCS_ClientTransferServiceFactory"
>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory"/
>>
> <property name="PCS_ActionRepoFile"
>value="file:[CRAWLER_HOME]/policy/crawler-config.xml" envReplace="true"/>
> </configuration>
>
> <requiredMetFields>
> <metfield name="RunID"/>
> </requiredMetFields>
>
></task>
></cas:tasks>
>
>
>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
><ma...@usc.edu> wrote:
>
>Hi Renu,
>
>Can you please show a paste.bin of your tasks.xml?
>
>Cheers,
>Chris
>
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>Chris Mattmann, Ph.D.
>Adjunct Associate Professor, Computer Science Department
>University of Southern California
>Los Angeles, CA 90089 USA
>Email: mattmann@usc.edu
>WWW: http://sunset.usc.edu/~mattmann/
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>-----Original Message-----
>From: Renu Kanakamedala <ka...@usc.edu>
>Date: Wednesday, November 5, 2014 at 7:55 PM
>To: Chris Mattmann <ma...@usc.edu>
>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
>null Error.
>
>>Hello Professor,
>>
>>
>>When i am trying to run file concatenator CAS PGE example using the
>>following command.
>>
>>
>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>>
>>
>>
>>The message shown after executing this command is "Sending Event
>>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
>>stub.
>>
>>
>>Batch Stub:
>>Caused by:
>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcep
>>t
>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>>value: null
>>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>>at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>>... 2 more
>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>>parse value: null
>>at
>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>>n
>>figBuilder.java:87)
>>at
>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.j
>>a
>>va:230)
>>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>>... 3 more
>>Caused by: java.lang.Exception: Failed to parse value: null
>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>>at
>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>>n
>>figBuilder.java:77)
>>... 5 more
>>Caused by: java.lang.NullPointerException
>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>>at java.util.regex.Matcher.reset(Matcher.java:308)
>>at java.util.regex.Matcher.<init>(Matcher.java:228)
>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>>at
>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacem
>>e
>>nt(PathUtils.java:321)
>>at
>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtil
>>s
>>.java:96)
>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>>... 7 more
>>
>>
>>
>>
>>
>>RES mngr:
>>
>>
>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>>addJob
>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>>Nov 05, 2014 5:26:30 PM
>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>>Scheduling for execution
>>Nov 05, 2014 5:26:30 PM
>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>plainConnect
>>FINEST: ProxySelector Request for
>
>
>>http://localhost:2001/ <http://localhost:2001/>
>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>plainConnect
>>FINEST: Proxy used: DIRECT
>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>writeRequests
>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>> q=.2}{Connection: keep-alive}{Content-Length: 105}
>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>getInputStream
>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>text/xml}{Content-Length: 121}
>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>plainConnect
>>FINEST: ProxySelector Request for
>>http://localhost:2001/ <http://localhost:2001/>
>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>plainConnect
>>FINEST: Proxy used: DIRECT
>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>>writeRequests
>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>>getInputStream
>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>text/xml}{Content-Length: 121}
>>Nov 05, 2014 5:26:32 PM
>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
>>batchstub.executeJob returned false
>>java.lang.Exception: batchstub.executeJob returned false
>>at
>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatch
>>M
>>grProxy.java:125)
>>
>>
>>
>>
>>
>>Workflow:
>>
>>
>>Nov 05, 2014 5:26:27 PM
>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>checkTaskRequiredMetadata
>>INFO: All required metadata fields present for task: [FileConcatenator]
>>Nov 05, 2014 5:26:28 PM
>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient submitJob
>>FINEST: [{job.queueName=high,
>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>
>
>>job.name <http://job.name> <http://job.name>=urn:oodt:FileConcatenator,
>>job.id <http://job.id> <http://job.id>=, job.status=, job.load=2,
>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileCo
>>n
>>catenatorPGETask,
>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/PG
>>E
>>Config.xml,
>>
>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer
>>.
>>LocalDataTransferFactory,
>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xml
>>,
>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>>PCS_WorkflowManagerUrl=http://localhost:9001,
>> PCS_FileManagerUrl=http://localhost:9000,
>>PGETask_Name=FileConcatenator},
>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>>WorkflowManagerUrl=[http://ubuntu:9001],
>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>> ProcessingNode=[ubuntu], RunID=[testNumber1],
>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>>Nov 05, 2014 5:26:39 PM
>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread stop
>>WARNING: Attempt to kill current resmgr job:
>>[2014-11-05T17:26:28.785-08:00]: failed
>>
>>
>>When i searched online i found one of your mail archives where you had
>>mentioned to change tasks.xml
>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>>
>>
>>
>>It did not work. Please let us know how to proceed.
>>
>>
>>
>>Regards,
>>Renu
>>
>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>
>
>
>
>
Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
null Error.
Posted by Renu Kanakamedala <ka...@usc.edu>.
Please find below tasks.xml file.
<cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
<!--
TODO: Add some examples
-->
<task id="urn:oodt:FileConcatenator" name="FileConcatenator"
class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGETask">
<conditions/>
<configuration>
<property name="PGETask_Name" value="FileConcatenator"/>
<property name="PGETask_ConfigFilePath"
value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
envReplace="true"/>
<property name="PGETask_DumpMetadata" value="true"/>
<property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
envReplace="true" />
<property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
envReplace="true"/>
<property name="PCS_MetFileExtension" value="met"/>
<property name="PCS_ClientTransferServiceFactory"
value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory"/>
<property name="PCS_ActionRepoFile"
value="file:[CRAWLER_HOME]/policy/crawler-config.xml" envReplace="true"/>
</configuration>
<requiredMetFields>
<metfield name="RunID"/>
</requiredMetFields>
</task>
</cas:tasks>
On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann <ma...@usc.edu>
wrote:
> Hi Renu,
>
> Can you please show a paste.bin of your tasks.xml?
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Adjunct Associate Professor, Computer Science Department
> University of Southern California
> Los Angeles, CA 90089 USA
> Email: mattmann@usc.edu
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
> -----Original Message-----
> From: Renu Kanakamedala <ka...@usc.edu>
> Date: Wednesday, November 5, 2014 at 7:55 PM
> To: Chris Mattmann <ma...@usc.edu>
> Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
> null Error.
>
> >Hello Professor,
> >
> >
> >When i am trying to run file concatenator CAS PGE example using the
> >following command.
> >
> >
> >./wmgr-client --url http://localhost:9001 --operation --sendEvent
> >--eventName fileconcatenator-pge --metaData --key RunID testNumber1
> >
> >
> >
> >The message shown after executing this command is "Sending Event
> >SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
> >stub.
> >
> >
> >Batch Stub:
> >Caused by:
> >org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcept
> >ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
> >value: null
> >at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> >at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> >... 2 more
> >Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
> >parse value: null
> >at
> >org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCon
> >figBuilder.java:87)
> >at
> >org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.ja
> >va:230)
> >at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> >... 3 more
> >Caused by: java.lang.Exception: Failed to parse value: null
> >at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> >at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> >at
> >org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCon
> >figBuilder.java:77)
> >... 5 more
> >Caused by: java.lang.NullPointerException
> >at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> >at java.util.regex.Matcher.reset(Matcher.java:308)
> >at java.util.regex.Matcher.<init>(Matcher.java:228)
> >at java.util.regex.Pattern.matcher(Pattern.java:1088)
> >at
> >org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplaceme
> >nt(PathUtils.java:321)
> >at
> >org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils
> >.java:96)
> >at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
> >... 7 more
> >
> >
> >
> >
> >
> >RES mngr:
> >
> >
> >Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
> >addJob
> >INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
> >Nov 05, 2014 5:26:30 PM
> >org.apache.oodt.cas.resource.scheduler.LRUScheduler run
> >INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
> >Scheduling for execution
> >Nov 05, 2014 5:26:30 PM
> >org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
> >INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >plainConnect
> >FINEST: ProxySelector Request for
> >http://localhost:2001/ <http://localhost:2001/>
> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >plainConnect
> >FINEST: Proxy used: DIRECT
> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >writeRequests
> >FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
> >null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >text/html, image/gif, image/jpeg, *; q=.2, */*;
> > q=.2}{Connection: keep-alive}{Content-Length: 105}
> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >getInputStream
> >FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
> >OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >text/xml}{Content-Length: 121}
> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >plainConnect
> >FINEST: ProxySelector Request for
> >http://localhost:2001/ <http://localhost:2001/>
> >Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >plainConnect
> >FINEST: Proxy used: DIRECT
> >Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
> >writeRequests
> >FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
> >null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >text/html, image/gif, image/jpeg, *; q=.2, */*;
> > q=.2}{Connection: keep-alive}{Content-Length: 2536}
> >Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
> >getInputStream
> >FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
> >OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >text/xml}{Content-Length: 121}
> >Nov 05, 2014 5:26:32 PM
> >org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
> >SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
> >batchstub.executeJob returned false
> >java.lang.Exception: batchstub.executeJob returned false
> >at
> >org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatchM
> >grProxy.java:125)
> >
> >
> >
> >
> >
> >Workflow:
> >
> >
> >Nov 05, 2014 5:26:27 PM
> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >checkTaskRequiredMetadata
> >INFO: All required metadata fields present for task: [FileConcatenator]
> >Nov 05, 2014 5:26:28 PM
> >org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient submitJob
> >FINEST: [{job.queueName=high,
> >job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
> >job.name <http://job.name>=urn:oodt:FileConcatenator,
> >job.id <http://job.id>=, job.status=, job.load=2,
> >job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
> >{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileCon
> >catenatorPGETask,
> >task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/PGE
> >Config.xml,
> >
> >PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer.
> >LocalDataTransferFactory,
> >PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xml,
> > PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
> >PCS_WorkflowManagerUrl=http://localhost:9001,
> > PCS_FileManagerUrl=http://localhost:9000,
> >PGETask_Name=FileConcatenator},
> >task.metadata={TaskId=[urn:oodt:FileConcatenator],
> >WorkflowManagerUrl=[http://ubuntu:9001],
> >JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
> > ProcessingNode=[ubuntu], RunID=[testNumber1],
> >WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
> >Nov 05, 2014 5:26:39 PM
> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread stop
> >WARNING: Attempt to kill current resmgr job:
> >[2014-11-05T17:26:28.785-08:00]: failed
> >
> >
> >When i searched online i found one of your mail archives where you had
> >mentioned to change tasks.xml
> >https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
> >
> >
> >
> >It did not work. Please let us know how to proceed.
> >
> >
> >
> >Regards,
> >Renu
> >
> >
> >
> >
> >
> >
> >
> >
>
>