You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@oodt.apache.org by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov> on 2015/06/05 16:04:51 UTC

Re: OODT Documentation

Hi Andrea,

Thanks for checking in. One of the issues at the UI level with
0.8.1 is that the 0.8.1 release introduced some odd jar level
issues with log4j having to do with the streaming components
of OODT that were in development.

OODT 0.9 should be released hopefully in < 24 hours, I would
recommend upgrading to that and I think the below issues should
go away.

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Andrea DeMarco <dr...@gmail.com>
Date: Friday, June 5, 2015 at 2:58 AM
To: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>
Cc: "Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Andrew
Faulkner <aj...@mrao.cam.ac.uk>, Kristian ZARB ADAMI <kz...@astro.ox.ac.uk>,
jpluser <ch...@jpl.nasa.gov>, "Ramirez, Paul M (398M)"
<pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>
Subject: Re: OODT Documentation

>Hi all,
>
>The Radix 0.9-SNAPSHOT version does not appear to be available. We've had
>some minor success installing 0.8.1, we even managed to start the File
>manager/Workflow Manager/Resource Manager, and the Ops UI loads on
>Tomcat. However various tabs display Wicket errors/stacktraces.
> I'm attaching a few screens of the errors in this email.
>
>Checking the following seems to be OK:
>http://localhost:9000/      (for FileManager)
>http://localhost:9001/      (for WorkflowManager)
>http://localhost:9002/      (for ResourceManager)
>
>And in the console:
>
>andrea@issa:~/oodt/bin$ ./oodt start
>Using OODT_BASE: /home/andrea/oodt
>Using OODT_HOME: /home/andrea/oodt
>Using OODT_TMPDIR: /home/andrea/oodt/temp
>Using JRE_HOME: /usr/lib/jvm/java-7-openjdk-amd64
>Using CLASSPATH:
>Starting OODT File Manager [ Successful ]
>Starting OODT Resource Manager [ Successful ]
>Starting OODT Workflow Manager [ Successful ]
>
>
>
>I am guessing the system is operational, but perhaps something's up at
>the UI level. Any thoughts?
>
>
>Thanks,
>Andrea
>

RE: OODT Documentation

Posted by "Mallder, Valerie" <Va...@jhuapl.edu>.

I'm actually suspicious that this might be a missing environment variable setting.  Doesn't the XmlHelper resolve the environment variable names to their values in an xml file?  I can't remember.

Would be good to check the environment settings and make sure none of those are null as well.



Valerie A. Mallder
New Horizons Deputy Mission System Engineer
Johns Hopkins University/Applied Physics Laboratory


> -----Original Message-----
> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
> Sent: Tuesday, June 09, 2015 11:44 AM
> To: dev@oodt.apache.org
> Subject: Re: OODT Documentation
> 
> Good advice, Val. I think I may know the issue here, though.
> I believe the issue is *definitely* associated with the legacy keys issue, as you
> stated. If you roll down to the NPE that Andrea is experience, it’s because one of
> the PGE config key values is null, one of the required ones that OODT expects.
> Pro tip: filing a JIRA ticket for actually stating which one of these is null would be
> IMMENSELY helpful and a relatively simple fix. Andrea, or Val, if one of you could
> file that JIRA ticket I will commit to updating PGE to print out which key is actually
> null. This one has been a long time coming.
> 
> Now: why is that key null? It’s null b/c I’m almost positive that the legancyMode for
> PGE is turned on by default in RADIX.
> 
> Andrea: what happens if you grab this file: http://s.apache.org/kDs and replace
> your $OODT_HOME/resmgr/bin/batch_stub file with it. Stop the batch stub, turn it
> back on, submit the job again, and cross your fingers!
> 
> Cheers,
> Chris
> 
> 
> ----
> Chris Mattmann
> chris.mattmann@gmail.com
> 
> 
> 
> 
> 
> 
> -----Original Message-----
> From: "Mallder, Valerie" <Va...@jhuapl.edu>
> Reply-To: <de...@oodt.apache.org>
> Date: Tuesday, June 9, 2015 at 8:05 AM
> To: "dev@oodt.apache.org" <de...@oodt.apache.org>
> Subject: RE: OODT Documentation
> 
> >Hi Andrea,
> >
> >I have found it helpful when debugging, or just investigating how
> >something works, to remove the resource manager from the equation and
> >run without it at first.  If you can get your tasks to run without the
> >resource manager, then you should be able to get them to run WITH the
> >resource manager. And if you can't, then at least you've narrowed down
> >the area where the problem is located.
> >
> >Go into oodt/workflow/etc and edit the file workflow.properties and
> >comment out line 36.  (Line 39 if you are using the trunk version) This
> >should be the line that specifies the resource manager url. Just
> >comment that out.  And then try running again.  You will of course get
> >a different set of log messages, but they might not look so cryptic
> >this time.
> >
> >Let me know how it goes.
> >Val
> >
> >
> >
> >
> >Valerie A. Mallder
> >New Horizons Deputy Mission System Engineer Johns Hopkins
> >University/Applied Physics Laboratory
> >
> >
> >> -----Original Message-----
> >> From: Andrea DeMarco [mailto:dr.andrea.demarco@gmail.com]
> >> Sent: Tuesday, June 09, 2015 10:37 AM
> >> To: Mattmann, Chris A (3980)
> >> Cc: Wyngaard, Jane R (3980-Affiliate); Starch, Michael D (398M);
> >>Kristian ZARB  ADAMI; Ramirez, Paul M (398M); Alessio Magro;
> >>dev@oodt.apache.org
> >> Subject: Re: OODT Documentation
> >>
> >> Hi Chris,
> >>
> >> Some additional detail:
> >>
> >> When stopping/starting the workflow manager before kicking the PGE,
> >>the terminal  (post PGE submission) also prints out the following:
> >>
> >> andrea@issa:~/oodt/workflow/bin$ ./wmgr stop
> >>andrea@issa:~/oodt/workflow/bin$
> >> ./wmgr start Workflow Manager started PID file
> >>(/home/andrea/oodt/workflow/run/cas.workflow.pid).
> >> andrea@issa:~/oodt/workflow/bin$ Jun 09, 2015 4:34:24 PM
> >>org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager
> >>loadProperties
> >> INFO: Loading Workflow Manager Configuration Properties from:
> >> [/home/andrea/oodt/workflow/etc/workflow.properties]
> >> Jun 09, 2015 4:34:25 PM
> >> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
> >> INFO: Workflow Manager started by andrea Jun 09, 2015 4:34:29 PM
> >>org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
> >> INFO: WorkflowManager: Received event: fileconcatenator-pge Jun 09,
> >>2015
> >> 4:34:29 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager
> >> handleEvent
> >> INFO: WorkflowManager: Workflow FileConcatenatorWorkflow retrieved
> >>for event  fileconcatenator-pge Jun 09, 2015 4:34:29 PM
> >>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >> setRClient
> >> INFO: Resource Manager Job Submission enabled to:
> >>[http://localhost:9002] Jun
> >> 09, 2015 4:34:29 PM
> >> org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >> checkTaskRequiredMetadata
> >> INFO: All required metadata fields present for task:
> >>[FileConcatenator] Jun 09,
> >> 2015 4:34:29 PM
> >> org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
> >>submitJob
> >> FINEST: [{job.queueName=high,
> >> job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
> >> job.name=urn:oodt:FileConcatenator,
> >> job.id=, job.status=, job.load=2,
> >>
> >>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
> >>
> >>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.Fil
> >>eCo
> >>ncaten
> >> atorPGETask,
> >>
> >>task.config={PGETask_ConfigFilePath=/home/andrea/oodt/pge/file_concate
> >>nat
> >>or/
> >> pge-configs/PGEConfig.xml,
> >>
> >>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatrans
> >>fer
> >>.Local
> >> DataTransferFactory,
> >>
> >>PCS_ActionRepoFile=file:/home/andrea/oodt/crawler/policy/crawler-confi
> >>g.x
> >>ml,
> >> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
> >>PCS_WorkflowManagerUrl= http://localhost:9001,
> >>PCS_FileManagerUrl=http://localhost:9000,
> >> PGETask_Name=FileConcatenator},
> >> task.metadata={TaskId=[urn:oodt:FileConcatenator],
> >>WorkflowManagerUrl=[  http://issa:9001],
> >>JobId=[a2eecfc3-0eb4-11e5-ac1b-fff04e088d71],
> >> ProcessingNode=[issa], RunID=[testNumber1],
> >>WorkflowInstId=[a2eecfc3-0eb4-
> >> 11e5-ac1b-fff04e088d71]}}]
> >>
> >> Not sure what it all means, but it may help :)
> >>
> >> Cheers,
> >> Andrea
> >>
> >> On 9 June 2015 at 16:03, Andrea DeMarco <dr...@gmail.com>
> >> wrote:
> >>
> >> > Hi Chris,
> >> >
> >> > No problem - I can imagine that checklist to be quite extensive :)
> >> >
> >> > I'll try and help as best I can:
> >> >
> >> > 1) The output when starting the batch_stub is:
> >> > andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001 [1] 2535
> >> > andrea@issa:~/oodt/resmgr/bin$ Jun 09, 2015 3:53:04 PM
> >> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
> >> > INFO: XmlRpc Batch Stub started by andrea
> >> >
> >> > 2) The latest oodt.out log file (after running the below) is
> >> > attached
> >>.
> >> >
> >> > 3) Kicking of the PGE with the following command (and terminal
> >>output):
> >> >
> >> > andrea@issa:~/oodt/workflow/bin$ ./wmgr-client --url
> >> > http://localhost:9001 --operation --sendEvent --eventName
> >> > fileconcatenator-pge --metaData --key RunID testNumber1 Sending
> >> > event 'fileconcatenator-pge'... SUCCESS
> >> >
> >> > I have now shut down my VM and restarted everything. When kicking
> >> > off the PGE, this is the terminal output from the same terminal
> >> > that started the
> >> > batch_stub:
> >> >
> >> > Jun 09, 2015 3:57:26 PM
> >> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub
> >> > genericExecuteJob
> >> > INFO: stub attempting to execute class:
> >> > [org.apache.oodt.cas.workflow.structs.TaskJob]
> >> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> >> > createPgeMetadata
> >> > INFO: Converting workflow configuration to static metadata...
> >> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> >> > createPgeMetadata
> >> > INFO: Loading workflow context metadata...
> >> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> >> > createPgeConfig
> >> > INFO: Create PgeConfig...
> >> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> >> > createPgeConfig
> >> > INFO: Using default PgeConfigBuilder:
> >> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
> >> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance run
> >> > SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to
> >> > parse
> >> > value: null
> >> > java.io.IOException: Failed to build PgeConfig : Failed to parse
> >>value:
> >> > null
> >> > at
> >> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFil
> >> > ePg
> >> > eConfigBuilder.java:87)
> >> > at
> >> > org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInst
> >> > anc
> >> > e.java:231) at
> >> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:12
> >> > 3)
> >> > at
> >> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:1
> >> > 02)
> >> > at
> >> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$Runnable
> >> > Job
> >> > .run(XmlRpcBatchStub.java:229) at
> >> > java.lang.Thread.run(Thread.java:745)
> >> > Caused by: java.lang.Exception: Failed to parse value: null at
> >> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> >> > at
> >> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> >> > at
> >> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFil
> >> > ePg
> >> > eConfigBuilder.java:77)
> >> > ... 5 more
> >> > Caused by: java.lang.NullPointerException at
> >> > java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> >> > at java.util.regex.Matcher.reset(Matcher.java:308)
> >> > at java.util.regex.Matcher.<init>(Matcher.java:228)
> >> > at java.util.regex.Pattern.matcher(Pattern.java:1088)
> >> > at
> >> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRe
> >> > pla
> >> > cement(PathUtils.java:321)
> >> > at
> >> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(Pa
> >> > thU
> >> > tils.java:96) at
> >> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
> >> > ... 7 more
> >> >
> >> > org.apache.oodt.cas.resource.structs.exceptions.JobInputException:
> >> > Failed to run task at
> >> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:1
> >> > 04)
> >> > at
> >> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$Runnable
> >> > Job
> >> > .run(XmlRpcBatchStub.java:229) at
> >> > java.lang.Thread.run(Thread.java:745)
> >> > Caused by:
> >> >
> >>
> >>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceEx
> >>cep
> >>tion
> >> :
> >> > PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value:
> >> > null at
> >> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:15
> >> > 0)
> >> > at
> >> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:1
> >> > 02)
> >> > ... 2 more
> >> > Caused by: java.io.IOException: Failed to build PgeConfig : Failed
> >> > to parse value: null at
> >> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFil
> >> > ePg
> >> > eConfigBuilder.java:87)
> >> > at
> >> > org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInst
> >> > anc
> >> > e.java:231) at
> >> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:12
> >> > 3)
> >> > ... 3 more
> >> > Caused by: java.lang.Exception: Failed to parse value: null at
> >> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> >> > at
> >> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> >> > at
> >> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFil
> >> > ePg
> >> > eConfigBuilder.java:77)
> >> > ... 5 more
> >> > Caused by: java.lang.NullPointerException at
> >> > java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> >> > at java.util.regex.Matcher.reset(Matcher.java:308)
> >> > at java.util.regex.Matcher.<init>(Matcher.java:228)
> >> > at java.util.regex.Pattern.matcher(Pattern.java:1088)
> >> > at
> >> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRe
> >> > pla
> >> > cement(PathUtils.java:321)
> >> > at
> >> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(Pa
> >> > thU
> >> > tils.java:96) at
> >> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
> >> > ... 7 more
> >> >
> >> >
> >> >
> >> > Cheers,
> >> > Andrea
> >> >
> >> >
> >> > On 9 June 2015 at 15:47, Mattmann, Chris A (3980) <
> >> > chris.a.mattmann@jpl.nasa.gov> wrote:
> >> >
> >> >> Thanks Andrea, OK bear with me as we go through our checklist :)
> >> >>
> >> >> Does anything show in $OODT_HOME/log/oodt.log?
> >> >>
> >> >> How are you kicking off the PGE? Does the terminal in which you
> >> >> started the batch stub print anything?
> >> >>
> >> >> Cheers,
> >> >> Chris
> >> >>
> >> >>
> >>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> +++++
> >> >> Chris Mattmann, Ph.D.
> >> >> Chief Architect
> >> >> Instrument Software and Science Data Systems Section (398) NASA
> >> >> Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >> >> Office: 168-519, Mailstop: 168-527
> >> >> Email: chris.a.mattmann@nasa.gov
> >> >> WWW:  http://sunset.usc.edu/~mattmann/
> >> >>
> >>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> +++++
> >> >> Adjunct Associate Professor, Computer Science Department
> >> >> University of Southern California, Los Angeles, CA 90089 USA
> >> >>
> >>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> +++++
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> -----Original Message-----
> >> >> From: Andrea DeMarco <dr...@gmail.com>
> >> >> Date: Tuesday, June 9, 2015 at 6:44 AM
> >> >> To: jpluser <ch...@jpl.nasa.gov>
> >> >> Cc: "Wyngaard, Jane R (3980-Affiliate)"
> >> >> <Ja...@jpl.nasa.gov>, "Starch, Michael D (398M)"
> >> >> <Mi...@jpl.nasa.gov>, Kristian ZARB ADAMI
> >> <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
> >> >> <pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
> >> >> "dev@oodt.apache.org" <de...@oodt.apache.org>
> >> >> Subject: Re: OODT Documentation
> >> >>
> >> >> >Hi Chris,
> >> >> >
> >> >> >
> >> >> >Yup, the jar file is present in both resmgr/lib and workflow/lib.
> >> >> >You can have a look at my JAR contents at:
> >> >> >
> >> >> >
> >> >> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0
> >> >> .ja
> >> >> r?dl=
> >> >> >0
> >> >> ><
> >> >> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0
> >> >> .ja
> >> >> r?dl
> >> >> >=0>  - perhaps something's off there?
> >> >> >
> >> >> >
> >> >> >Cheers,
> >> >> >Andrea
> >> >> >
> >> >>
> >> >>
> >> >
> >> >
> >>
> >>
> >> --
> >> Best Regards,
> >> Andrea DeMarco
> >> Institute of Space Sciences and Astronomy University of Malta
>

Re: OODT Documentation

Posted by Chris Mattmann <ch...@gmail.com>.

Good advice, Val. I think I may know the issue here, though.
I believe the issue is *definitely* associated with the legacy
keys issue, as you stated. If you roll down to the NPE that Andrea
is experience, it’s because one of the PGE config key values is
null, one of the required ones that OODT expects. Pro tip: filing
a JIRA ticket for actually stating which one of these is null would
be IMMENSELY helpful and a relatively simple fix. Andrea, or Val,
if one of you could file that JIRA ticket I will commit to updating
PGE to print out which key is actually null. This one has been a long
time coming.

Now: why is that key null? It’s null b/c I’m almost positive that
the legancyMode for PGE is turned on by default in RADIX.

Andrea: what happens if you grab this file: http://s.apache.org/kDs
and replace your $OODT_HOME/resmgr/bin/batch_stub file with it. Stop
the batch stub, turn it back on, submit the job again, and cross
your fingers!

Cheers,
Chris


----
Chris Mattmann
chris.mattmann@gmail.com






-----Original Message-----
From: "Mallder, Valerie" <Va...@jhuapl.edu>
Reply-To: <de...@oodt.apache.org>
Date: Tuesday, June 9, 2015 at 8:05 AM
To: "dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: RE: OODT Documentation

>Hi Andrea,
>
>I have found it helpful when debugging, or just investigating how
>something works, to remove the resource manager from the equation and run
>without it at first.  If you can get your tasks to run without the
>resource manager, then you should be able to get them to run WITH the
>resource manager. And if you can't, then at least you've narrowed down
>the area where the problem is located.
>
>Go into oodt/workflow/etc and edit the file workflow.properties and
>comment out line 36.  (Line 39 if you are using the trunk version) This
>should be the line that specifies the resource manager url. Just comment
>that out.  And then try running again.  You will of course get a
>different set of log messages, but they might not look so cryptic this
>time.
>
>Let me know how it goes.
>Val
>
>
>
>
>Valerie A. Mallder
>New Horizons Deputy Mission System Engineer
>Johns Hopkins University/Applied Physics Laboratory
>
>
>> -----Original Message-----
>> From: Andrea DeMarco [mailto:dr.andrea.demarco@gmail.com]
>> Sent: Tuesday, June 09, 2015 10:37 AM
>> To: Mattmann, Chris A (3980)
>> Cc: Wyngaard, Jane R (3980-Affiliate); Starch, Michael D (398M);
>>Kristian ZARB
>> ADAMI; Ramirez, Paul M (398M); Alessio Magro; dev@oodt.apache.org
>> Subject: Re: OODT Documentation
>> 
>> Hi Chris,
>> 
>> Some additional detail:
>> 
>> When stopping/starting the workflow manager before kicking the PGE, the
>>terminal
>> (post PGE submission) also prints out the following:
>> 
>> andrea@issa:~/oodt/workflow/bin$ ./wmgr stop
>>andrea@issa:~/oodt/workflow/bin$
>> ./wmgr start Workflow Manager started PID file
>> (/home/andrea/oodt/workflow/run/cas.workflow.pid).
>> andrea@issa:~/oodt/workflow/bin$ Jun 09, 2015 4:34:24 PM
>> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
>> INFO: Loading Workflow Manager Configuration Properties from:
>> [/home/andrea/oodt/workflow/etc/workflow.properties]
>> Jun 09, 2015 4:34:25 PM
>> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
>> INFO: Workflow Manager started by andrea Jun 09, 2015 4:34:29 PM
>> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
>> INFO: WorkflowManager: Received event: fileconcatenator-pge Jun 09, 2015
>> 4:34:29 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager
>> handleEvent
>> INFO: WorkflowManager: Workflow FileConcatenatorWorkflow retrieved for
>>event
>> fileconcatenator-pge Jun 09, 2015 4:34:29 PM
>> org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> setRClient
>> INFO: Resource Manager Job Submission enabled to:
>>[http://localhost:9002] Jun
>> 09, 2015 4:34:29 PM
>> org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> checkTaskRequiredMetadata
>> INFO: All required metadata fields present for task: [FileConcatenator]
>>Jun 09,
>> 2015 4:34:29 PM
>> org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>>submitJob
>> FINEST: [{job.queueName=high,
>> job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>> job.name=urn:oodt:FileConcatenator,
>> job.id=, job.status=, job.load=2,
>> job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>> 
>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileCo
>>ncaten
>> atorPGETask,
>> 
>>task.config={PGETask_ConfigFilePath=/home/andrea/oodt/pge/file_concatenat
>>or/
>> pge-configs/PGEConfig.xml,
>> 
>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer
>>.Local
>> DataTransferFactory,
>> 
>>PCS_ActionRepoFile=file:/home/andrea/oodt/crawler/policy/crawler-config.x
>>ml,
>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>> PCS_WorkflowManagerUrl= http://localhost:9001,
>> PCS_FileManagerUrl=http://localhost:9000,
>> PGETask_Name=FileConcatenator},
>> task.metadata={TaskId=[urn:oodt:FileConcatenator], WorkflowManagerUrl=[
>> http://issa:9001], JobId=[a2eecfc3-0eb4-11e5-ac1b-fff04e088d71],
>> ProcessingNode=[issa], RunID=[testNumber1],
>>WorkflowInstId=[a2eecfc3-0eb4-
>> 11e5-ac1b-fff04e088d71]}}]
>> 
>> Not sure what it all means, but it may help :)
>> 
>> Cheers,
>> Andrea
>> 
>> On 9 June 2015 at 16:03, Andrea DeMarco <dr...@gmail.com>
>> wrote:
>> 
>> > Hi Chris,
>> >
>> > No problem - I can imagine that checklist to be quite extensive :)
>> >
>> > I'll try and help as best I can:
>> >
>> > 1) The output when starting the batch_stub is:
>> > andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001 [1] 2535
>> > andrea@issa:~/oodt/resmgr/bin$ Jun 09, 2015 3:53:04 PM
>> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
>> > INFO: XmlRpc Batch Stub started by andrea
>> >
>> > 2) The latest oodt.out log file (after running the below) is attached
>>.
>> >
>> > 3) Kicking of the PGE with the following command (and terminal
>>output):
>> >
>> > andrea@issa:~/oodt/workflow/bin$ ./wmgr-client --url
>> > http://localhost:9001 --operation --sendEvent --eventName
>> > fileconcatenator-pge --metaData --key RunID testNumber1 Sending event
>> > 'fileconcatenator-pge'... SUCCESS
>> >
>> > I have now shut down my VM and restarted everything. When kicking off
>> > the PGE, this is the terminal output from the same terminal that
>> > started the
>> > batch_stub:
>> >
>> > Jun 09, 2015 3:57:26 PM
>> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub
>> > genericExecuteJob
>> > INFO: stub attempting to execute class:
>> > [org.apache.oodt.cas.workflow.structs.TaskJob]
>> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
>> > createPgeMetadata
>> > INFO: Converting workflow configuration to static metadata...
>> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
>> > createPgeMetadata
>> > INFO: Loading workflow context metadata...
>> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
>> > createPgeConfig
>> > INFO: Create PgeConfig...
>> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
>> > createPgeConfig
>> > INFO: Using default PgeConfigBuilder:
>> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
>> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance run
>> > SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to
>> > parse
>> > value: null
>> > java.io.IOException: Failed to build PgeConfig : Failed to parse
>>value:
>> > null
>> > at
>> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
>> > eConfigBuilder.java:87)
>> > at
>> > org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc
>> > e.java:231) at
>> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>> > at
>> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>> > at
>> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob
>> > .run(XmlRpcBatchStub.java:229) at
>> > java.lang.Thread.run(Thread.java:745)
>> > Caused by: java.lang.Exception: Failed to parse value: null at
>> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> > at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> > at
>> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
>> > eConfigBuilder.java:77)
>> > ... 5 more
>> > Caused by: java.lang.NullPointerException at
>> > java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>> > at java.util.regex.Matcher.reset(Matcher.java:308)
>> > at java.util.regex.Matcher.<init>(Matcher.java:228)
>> > at java.util.regex.Pattern.matcher(Pattern.java:1088)
>> > at
>> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla
>> > cement(PathUtils.java:321)
>> > at
>> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU
>> > tils.java:96) at
>> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
>> > ... 7 more
>> >
>> > org.apache.oodt.cas.resource.structs.exceptions.JobInputException:
>> > Failed to run task at
>> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104)
>> > at
>> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob
>> > .run(XmlRpcBatchStub.java:229) at
>> > java.lang.Thread.run(Thread.java:745)
>> > Caused by:
>> >
>> 
>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcep
>>tion
>> :
>> > PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value:
>> > null at
>> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>> > at
>> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>> > ... 2 more
>> > Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>> > parse value: null at
>> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
>> > eConfigBuilder.java:87)
>> > at
>> > org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc
>> > e.java:231) at
>> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>> > ... 3 more
>> > Caused by: java.lang.Exception: Failed to parse value: null at
>> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> > at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> > at
>> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
>> > eConfigBuilder.java:77)
>> > ... 5 more
>> > Caused by: java.lang.NullPointerException at
>> > java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>> > at java.util.regex.Matcher.reset(Matcher.java:308)
>> > at java.util.regex.Matcher.<init>(Matcher.java:228)
>> > at java.util.regex.Pattern.matcher(Pattern.java:1088)
>> > at
>> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla
>> > cement(PathUtils.java:321)
>> > at
>> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU
>> > tils.java:96) at
>> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
>> > ... 7 more
>> >
>> >
>> >
>> > Cheers,
>> > Andrea
>> >
>> >
>> > On 9 June 2015 at 15:47, Mattmann, Chris A (3980) <
>> > chris.a.mattmann@jpl.nasa.gov> wrote:
>> >
>> >> Thanks Andrea, OK bear with me as we go through our checklist :)
>> >>
>> >> Does anything show in $OODT_HOME/log/oodt.log?
>> >>
>> >> How are you kicking off the PGE? Does the terminal in which you
>> >> started the batch stub print anything?
>> >>
>> >> Cheers,
>> >> Chris
>> >>
>> >>
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>> >> Chris Mattmann, Ph.D.
>> >> Chief Architect
>> >> Instrument Software and Science Data Systems Section (398) NASA Jet
>> >> Propulsion Laboratory Pasadena, CA 91109 USA
>> >> Office: 168-519, Mailstop: 168-527
>> >> Email: chris.a.mattmann@nasa.gov
>> >> WWW:  http://sunset.usc.edu/~mattmann/
>> >>
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>> >> Adjunct Associate Professor, Computer Science Department University
>> >> of Southern California, Los Angeles, CA 90089 USA
>> >>
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>> >>
>> >>
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Andrea DeMarco <dr...@gmail.com>
>> >> Date: Tuesday, June 9, 2015 at 6:44 AM
>> >> To: jpluser <ch...@jpl.nasa.gov>
>> >> Cc: "Wyngaard, Jane R (3980-Affiliate)"
>> >> <Ja...@jpl.nasa.gov>, "Starch, Michael D (398M)"
>> >> <Mi...@jpl.nasa.gov>, Kristian ZARB ADAMI
>> <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
>> >> <pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
>> >> "dev@oodt.apache.org" <de...@oodt.apache.org>
>> >> Subject: Re: OODT Documentation
>> >>
>> >> >Hi Chris,
>> >> >
>> >> >
>> >> >Yup, the jar file is present in both resmgr/lib and workflow/lib.
>> >> >You can have a look at my JAR contents at:
>> >> >
>> >> >
>> >> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.ja
>> >> r?dl=
>> >> >0
>> >> ><
>> >> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.ja
>> >> r?dl
>> >> >=0>  - perhaps something's off there?
>> >> >
>> >> >
>> >> >Cheers,
>> >> >Andrea
>> >> >
>> >>
>> >>
>> >
>> >
>> 
>> 
>> --
>> Best Regards,
>> Andrea DeMarco
>> Institute of Space Sciences and Astronomy University of Malta

RE: OODT Documentation

Posted by "Mallder, Valerie" <Va...@jhuapl.edu>.

Hi Andrea,

I have found it helpful when debugging, or just investigating how something works, to remove the resource manager from the equation and run without it at first.  If you can get your tasks to run without the resource manager, then you should be able to get them to run WITH the resource manager. And if you can't, then at least you've narrowed down the area where the problem is located. 

Go into oodt/workflow/etc and edit the file workflow.properties and comment out line 36.  (Line 39 if you are using the trunk version) This should be the line that specifies the resource manager url. Just comment that out.  And then try running again.  You will of course get a different set of log messages, but they might not look so cryptic this time.

Let me know how it goes.
Val




Valerie A. Mallder
New Horizons Deputy Mission System Engineer
Johns Hopkins University/Applied Physics Laboratory


> -----Original Message-----
> From: Andrea DeMarco [mailto:dr.andrea.demarco@gmail.com]
> Sent: Tuesday, June 09, 2015 10:37 AM
> To: Mattmann, Chris A (3980)
> Cc: Wyngaard, Jane R (3980-Affiliate); Starch, Michael D (398M); Kristian ZARB
> ADAMI; Ramirez, Paul M (398M); Alessio Magro; dev@oodt.apache.org
> Subject: Re: OODT Documentation
> 
> Hi Chris,
> 
> Some additional detail:
> 
> When stopping/starting the workflow manager before kicking the PGE, the terminal
> (post PGE submission) also prints out the following:
> 
> andrea@issa:~/oodt/workflow/bin$ ./wmgr stop andrea@issa:~/oodt/workflow/bin$
> ./wmgr start Workflow Manager started PID file
> (/home/andrea/oodt/workflow/run/cas.workflow.pid).
> andrea@issa:~/oodt/workflow/bin$ Jun 09, 2015 4:34:24 PM
> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
> INFO: Loading Workflow Manager Configuration Properties from:
> [/home/andrea/oodt/workflow/etc/workflow.properties]
> Jun 09, 2015 4:34:25 PM
> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
> INFO: Workflow Manager started by andrea Jun 09, 2015 4:34:29 PM
> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
> INFO: WorkflowManager: Received event: fileconcatenator-pge Jun 09, 2015
> 4:34:29 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager
> handleEvent
> INFO: WorkflowManager: Workflow FileConcatenatorWorkflow retrieved for event
> fileconcatenator-pge Jun 09, 2015 4:34:29 PM
> org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> setRClient
> INFO: Resource Manager Job Submission enabled to: [http://localhost:9002] Jun
> 09, 2015 4:34:29 PM
> org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> checkTaskRequiredMetadata
> INFO: All required metadata fields present for task: [FileConcatenator] Jun 09,
> 2015 4:34:29 PM
> org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient submitJob
> FINEST: [{job.queueName=high,
> job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
> job.name=urn:oodt:FileConcatenator,
> job.id=, job.status=, job.load=2,
> job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
> {task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileConcaten
> atorPGETask,
> task.config={PGETask_ConfigFilePath=/home/andrea/oodt/pge/file_concatenator/
> pge-configs/PGEConfig.xml,
> PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer.Local
> DataTransferFactory,
> PCS_ActionRepoFile=file:/home/andrea/oodt/crawler/policy/crawler-config.xml,
> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
> PCS_WorkflowManagerUrl= http://localhost:9001,
> PCS_FileManagerUrl=http://localhost:9000,
> PGETask_Name=FileConcatenator},
> task.metadata={TaskId=[urn:oodt:FileConcatenator], WorkflowManagerUrl=[
> http://issa:9001], JobId=[a2eecfc3-0eb4-11e5-ac1b-fff04e088d71],
> ProcessingNode=[issa], RunID=[testNumber1], WorkflowInstId=[a2eecfc3-0eb4-
> 11e5-ac1b-fff04e088d71]}}]
> 
> Not sure what it all means, but it may help :)
> 
> Cheers,
> Andrea
> 
> On 9 June 2015 at 16:03, Andrea DeMarco <dr...@gmail.com>
> wrote:
> 
> > Hi Chris,
> >
> > No problem - I can imagine that checklist to be quite extensive :)
> >
> > I'll try and help as best I can:
> >
> > 1) The output when starting the batch_stub is:
> > andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001 [1] 2535
> > andrea@issa:~/oodt/resmgr/bin$ Jun 09, 2015 3:53:04 PM
> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
> > INFO: XmlRpc Batch Stub started by andrea
> >
> > 2) The latest oodt.out log file (after running the below) is attached .
> >
> > 3) Kicking of the PGE with the following command (and terminal output):
> >
> > andrea@issa:~/oodt/workflow/bin$ ./wmgr-client --url
> > http://localhost:9001 --operation --sendEvent --eventName
> > fileconcatenator-pge --metaData --key RunID testNumber1 Sending event
> > 'fileconcatenator-pge'... SUCCESS
> >
> > I have now shut down my VM and restarted everything. When kicking off
> > the PGE, this is the terminal output from the same terminal that
> > started the
> > batch_stub:
> >
> > Jun 09, 2015 3:57:26 PM
> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub
> > genericExecuteJob
> > INFO: stub attempting to execute class:
> > [org.apache.oodt.cas.workflow.structs.TaskJob]
> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> > createPgeMetadata
> > INFO: Converting workflow configuration to static metadata...
> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> > createPgeMetadata
> > INFO: Loading workflow context metadata...
> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> > createPgeConfig
> > INFO: Create PgeConfig...
> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> > createPgeConfig
> > INFO: Using default PgeConfigBuilder:
> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance run
> > SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to
> > parse
> > value: null
> > java.io.IOException: Failed to build PgeConfig : Failed to parse value:
> > null
> > at
> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
> > eConfigBuilder.java:87)
> > at
> > org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc
> > e.java:231) at
> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> > at
> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> > at
> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob
> > .run(XmlRpcBatchStub.java:229) at
> > java.lang.Thread.run(Thread.java:745)
> > Caused by: java.lang.Exception: Failed to parse value: null at
> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> > at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> > at
> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
> > eConfigBuilder.java:77)
> > ... 5 more
> > Caused by: java.lang.NullPointerException at
> > java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> > at java.util.regex.Matcher.reset(Matcher.java:308)
> > at java.util.regex.Matcher.<init>(Matcher.java:228)
> > at java.util.regex.Pattern.matcher(Pattern.java:1088)
> > at
> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla
> > cement(PathUtils.java:321)
> > at
> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU
> > tils.java:96) at
> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
> > ... 7 more
> >
> > org.apache.oodt.cas.resource.structs.exceptions.JobInputException:
> > Failed to run task at
> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104)
> > at
> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob
> > .run(XmlRpcBatchStub.java:229) at
> > java.lang.Thread.run(Thread.java:745)
> > Caused by:
> >
> org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceException
> :
> > PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value:
> > null at
> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> > at
> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> > ... 2 more
> > Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
> > parse value: null at
> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
> > eConfigBuilder.java:87)
> > at
> > org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc
> > e.java:231) at
> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> > ... 3 more
> > Caused by: java.lang.Exception: Failed to parse value: null at
> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> > at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> > at
> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
> > eConfigBuilder.java:77)
> > ... 5 more
> > Caused by: java.lang.NullPointerException at
> > java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> > at java.util.regex.Matcher.reset(Matcher.java:308)
> > at java.util.regex.Matcher.<init>(Matcher.java:228)
> > at java.util.regex.Pattern.matcher(Pattern.java:1088)
> > at
> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla
> > cement(PathUtils.java:321)
> > at
> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU
> > tils.java:96) at
> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
> > ... 7 more
> >
> >
> >
> > Cheers,
> > Andrea
> >
> >
> > On 9 June 2015 at 15:47, Mattmann, Chris A (3980) <
> > chris.a.mattmann@jpl.nasa.gov> wrote:
> >
> >> Thanks Andrea, OK bear with me as we go through our checklist :)
> >>
> >> Does anything show in $OODT_HOME/log/oodt.log?
> >>
> >> How are you kicking off the PGE? Does the terminal in which you
> >> started the batch stub print anything?
> >>
> >> Cheers,
> >> Chris
> >>
> >>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++
> >> Chris Mattmann, Ph.D.
> >> Chief Architect
> >> Instrument Software and Science Data Systems Section (398) NASA Jet
> >> Propulsion Laboratory Pasadena, CA 91109 USA
> >> Office: 168-519, Mailstop: 168-527
> >> Email: chris.a.mattmann@nasa.gov
> >> WWW:  http://sunset.usc.edu/~mattmann/
> >>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++
> >> Adjunct Associate Professor, Computer Science Department University
> >> of Southern California, Los Angeles, CA 90089 USA
> >>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: Andrea DeMarco <dr...@gmail.com>
> >> Date: Tuesday, June 9, 2015 at 6:44 AM
> >> To: jpluser <ch...@jpl.nasa.gov>
> >> Cc: "Wyngaard, Jane R (3980-Affiliate)"
> >> <Ja...@jpl.nasa.gov>, "Starch, Michael D (398M)"
> >> <Mi...@jpl.nasa.gov>, Kristian ZARB ADAMI
> <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
> >> <pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
> >> "dev@oodt.apache.org" <de...@oodt.apache.org>
> >> Subject: Re: OODT Documentation
> >>
> >> >Hi Chris,
> >> >
> >> >
> >> >Yup, the jar file is present in both resmgr/lib and workflow/lib.
> >> >You can have a look at my JAR contents at:
> >> >
> >> >
> >> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.ja
> >> r?dl=
> >> >0
> >> ><
> >> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.ja
> >> r?dl
> >> >=0>  - perhaps something's off there?
> >> >
> >> >
> >> >Cheers,
> >> >Andrea
> >> >
> >>
> >>
> >
> >
> 
> 
> --
> Best Regards,
> Andrea DeMarco
> Institute of Space Sciences and Astronomy University of Malta

Re: OODT Documentation

Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.

Thanks that helps to debug. Can you file a ticket in JIRA as well
for this Andrea? You shouldn’t get getting those Workflow Exceptions
on the Resource Manager OPSUI page though - weird - did you mean to
see the Workflow page was busted?

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Andrea DeMarco <dr...@gmail.com>
Date: Thursday, June 11, 2015 at 8:19 AM
To: jpluser <ch...@jpl.nasa.gov>
Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
"Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
<pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
"dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: Re: OODT Documentation

>Hi Chris,
>
>One other detail: is it perhaps also indicative of an issue in some
>way that the Resource Monitor of OPSUI is the only tab/page that still
>gives exceptions (after the 0.9 upgrade)?
>
>WicketMessage: Can't instantiate page using constructor public
>org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage(org.apache.wicket.Pag
>eParameters)
>and argument status = "ALL" pageNum = "null"
>
>Root cause:
>
>java.lang.NumberFormatException: For input string: "null"
>     at 
>java.lang.NumberFormatException.forInputString(NumberFormatException.java:
>65)
>     at java.lang.Integer.parseInt(Integer.java:580)
>     at java.lang.Integer.parseInt(Integer.java:615)
>     at 
>org.apache.wicket.util.string.StringValue.toInt(StringValue.java:501)
>     at 
>org.apache.wicket.util.string.StringValue.toInt(StringValue.java:520)
>     at org.apache.wicket.util.value.ValueMap.getInt(ValueMap.java:275)
>     at 
>org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage.<init>(WorkflowInstan
>ceViewerPage.java:43)
>     at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>     at 
>org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory
>.java:188)
>     at 
>org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.ja
>va:89)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.n
>ewPage(BookmarkablePageRequestTarget.java:305)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.g
>etPage(BookmarkablePageRequestTarget.java:320)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.p
>rocessEvents(BookmarkablePageRequestTarget.java:234)
>     at 
>org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abst
>ractRequestCycleProcessor.java:92)
>     at 
>org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1
>250)
>     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
>     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
>     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
>     at 
>org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)
>     at 
>org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:31
>9)
>     at 
>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicati
>onFilterChain.java:215)
>     at 
>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilter
>Chain.java:188)
>     at 
>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.
>java:210)
>     at 
>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.
>java:174)
>     at 
>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:1
>27)
>     at 
>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:1
>17)
>     at 
>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.ja
>va:108)
>     at 
>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151
>)
>     at 
>org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
>     at 
>org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.proces
>sConnection(Http11BaseProtocol.java:665)
>     at 
>org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.j
>ava:528)
>     at 
>org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollower
>WorkerThread.java:81)
>     at 
>org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.j
>ava:685)
>     at java.lang.Thread.run(Thread.java:745)
>
>Complete stack:
>
>org.apache.wicket.WicketRuntimeException: Can't instantiate page using
>constructor public
>org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage(org.apache.wicket.Pag
>eParameters)
>and argument status = "ALL" pageNum = "null"
>     at 
>org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory
>.java:212)
>     at 
>org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.ja
>va:89)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.n
>ewPage(BookmarkablePageRequestTarget.java:305)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.g
>etPage(BookmarkablePageRequestTarget.java:320)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.p
>rocessEvents(BookmarkablePageRequestTarget.java:234)
>     at 
>org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abst
>ractRequestCycleProcessor.java:92)
>     at 
>org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1
>250)
>     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
>     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
>     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
>     at 
>org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)
>
>java.lang.reflect.InvocationTargetException
>     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>Method)
>     at 
>sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAcc
>essorImpl.java:62)
>     at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>     at 
>org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory
>.java:188)
>     at 
>org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.ja
>va:89)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.n
>ewPage(BookmarkablePageRequestTarget.java:305)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.g
>etPage(BookmarkablePageRequestTarget.java:320)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.p
>rocessEvents(BookmarkablePageRequestTarget.java:234)
>     at 
>org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abst
>ractRequestCycleProcessor.java:92)
>     at 
>org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1
>250)
>     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
>     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
>     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
>     at 
>org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)
>
>org.apache.wicket.util.string.StringValueConversionException: Unable
>to convert 'null' to an int value
>     at 
>org.apache.wicket.util.string.StringValue.toInt(StringValue.java:505)
>     at 
>org.apache.wicket.util.string.StringValue.toInt(StringValue.java:520)
>     at org.apache.wicket.util.value.ValueMap.getInt(ValueMap.java:275)
>     at 
>org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage.<init>(WorkflowInstan
>ceViewerPage.java:43)
>     at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>     at 
>org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory
>.java:188)
>     at 
>org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.ja
>va:89)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.n
>ewPage(BookmarkablePageRequestTarget.java:305)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.g
>etPage(BookmarkablePageRequestTarget.java:320)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.p
>rocessEvents(BookmarkablePageRequestTarget.java:234)
>     at 
>org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abst
>ractRequestCycleProcessor.java:92)
>     at 
>org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1
>250)
>     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
>     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
>     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
>     at 
>org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)
>
>Sorry for the massive exception trails - just wanted to give you all
>the info I have on my end.
>
>Thanks,
>Andrea
>
>On 11 June 2015 at 17:13, Andrea DeMarco <dr...@gmail.com>
>wrote:
>> Hi Chris,
>>
>> I have tried with the new batch_stub - same result. Attached is the
>> oodt.out log file.
>>
>> Moreover this is the exception from the batch_stub terminal:
>>
>> andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001
>> [1] 2777
>> andrea@issa:~/oodt/resmgr/bin$ Jun 11, 2015 4:56:28 PM
>> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
>> INFO: XmlRpc Batch Stub started by andrea
>> Jun 11, 2015 4:58:38 PM
>> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub
>> genericExecuteJob
>> INFO: stub attempting to execute class:
>> [org.apache.oodt.cas.workflow.structs.TaskJob]
>> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>> createPgeMetadata
>> INFO: Converting workflow configuration to static metadata...
>> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>> createPgeMetadata
>> INFO: Loading workflow context metadata...
>> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>>createPgeConfig
>> INFO: Create PgeConfig...
>> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>>createPgeConfig
>> INFO: Using default PgeConfigBuilder:
>> org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
>> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance run
>> SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to
>> parse value: null
>> java.io.IOException: Failed to build PgeConfig : Failed to parse value:
>>null
>> at 
>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>>nfigBuilder.java:87)
>> at 
>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.j
>>ava:231)
>> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>> at 
>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>> at 
>>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.ru
>>n(XmlRpcBatchStub.java:229)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.Exception: Failed to parse value: null
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> at 
>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>>nfigBuilder.java:77)
>> ... 5 more
>> Caused by: java.lang.NullPointerException
>> at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
>> at java.util.regex.Matcher.reset(Matcher.java:309)
>> at java.util.regex.Matcher.<init>(Matcher.java:229)
>> at java.util.regex.Pattern.matcher(Pattern.java:1093)
>> at 
>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacem
>>ent(PathUtils.java:321)
>> at 
>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtil
>>s.java:96)
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
>> ... 7 more
>>
>> org.apache.oodt.cas.resource.structs.exceptions.JobInputException:
>> Failed to run task
>> at 
>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104)
>> at 
>>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.ru
>>n(XmlRpcBatchStub.java:229)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: 
>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcep
>>tion:
>> PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value:
>> null
>> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>> at 
>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>> ... 2 more
>> Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>> parse value: null
>> at 
>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>>nfigBuilder.java:87)
>> at 
>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.j
>>ava:231)
>> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>> ... 3 more
>> Caused by: java.lang.Exception: Failed to parse value: null
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> at 
>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>>nfigBuilder.java:77)
>> ... 5 more
>> Caused by: java.lang.NullPointerException
>> at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
>> at java.util.regex.Matcher.reset(Matcher.java:309)
>> at java.util.regex.Matcher.<init>(Matcher.java:229)
>> at java.util.regex.Pattern.matcher(Pattern.java:1093)
>> at 
>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacem
>>ent(PathUtils.java:321)
>> at 
>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtil
>>s.java:96)
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
>> ... 7 more
>>
>>
>>
>> I have then turned off the resource manager as Valerie suggested,
>> turned off OODT entirely and restarted the job, and lo and behold, it
>> worked, and files were ingested successfully! I ran the job a second
>> time and it worked again...attached also find a log file for this bit
>> (oodt_2.out)
>>
>> Minor progress :) Now to get that resource manager working...
>> Also: what can I do to file the JIRA ticket you need?
>>
>> Cheers,
>> Andrea
>
>
>
>-- 
>Best Regards,
>Andrea DeMarco
>Institute of Space Sciences and Astronomy
>University of Malta

Re: OODT Documentation

Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.

Also one suggestion I have so I don’t have to keep moderating your
emails through to the list would be for you and whomever else
you think on your team would be relevant to subscribe to Apache OODT
email list by sending a blank email to dev-sbuscribe@oodt.apache.org
from your preferred address, then following the instructions from there.
Then you send email to dev@oodt.apache.org and you are subscribed
and I don’t have to moderate it through and you will be situationally
aware of the replies.

Thanks!

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Andrea DeMarco <dr...@gmail.com>
Date: Thursday, June 11, 2015 at 8:19 AM
To: jpluser <ch...@jpl.nasa.gov>
Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
"Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
<pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
"dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: Re: OODT Documentation

>Hi Chris,
>
>One other detail: is it perhaps also indicative of an issue in some
>way that the Resource Monitor of OPSUI is the only tab/page that still
>gives exceptions (after the 0.9 upgrade)?
>
>WicketMessage: Can't instantiate page using constructor public
>org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage(org.apache.wicket.Pag
>eParameters)
>and argument status = "ALL" pageNum = "null"
>
>Root cause:
>
>java.lang.NumberFormatException: For input string: "null"
>     at 
>java.lang.NumberFormatException.forInputString(NumberFormatException.java:
>65)
>     at java.lang.Integer.parseInt(Integer.java:580)
>     at java.lang.Integer.parseInt(Integer.java:615)
>     at 
>org.apache.wicket.util.string.StringValue.toInt(StringValue.java:501)
>     at 
>org.apache.wicket.util.string.StringValue.toInt(StringValue.java:520)
>     at org.apache.wicket.util.value.ValueMap.getInt(ValueMap.java:275)
>     at 
>org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage.<init>(WorkflowInstan
>ceViewerPage.java:43)
>     at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>     at 
>org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory
>.java:188)
>     at 
>org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.ja
>va:89)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.n
>ewPage(BookmarkablePageRequestTarget.java:305)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.g
>etPage(BookmarkablePageRequestTarget.java:320)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.p
>rocessEvents(BookmarkablePageRequestTarget.java:234)
>     at 
>org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abst
>ractRequestCycleProcessor.java:92)
>     at 
>org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1
>250)
>     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
>     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
>     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
>     at 
>org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)
>     at 
>org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:31
>9)
>     at 
>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicati
>onFilterChain.java:215)
>     at 
>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilter
>Chain.java:188)
>     at 
>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.
>java:210)
>     at 
>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.
>java:174)
>     at 
>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:1
>27)
>     at 
>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:1
>17)
>     at 
>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.ja
>va:108)
>     at 
>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151
>)
>     at 
>org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
>     at 
>org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.proces
>sConnection(Http11BaseProtocol.java:665)
>     at 
>org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.j
>ava:528)
>     at 
>org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollower
>WorkerThread.java:81)
>     at 
>org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.j
>ava:685)
>     at java.lang.Thread.run(Thread.java:745)
>
>Complete stack:
>
>org.apache.wicket.WicketRuntimeException: Can't instantiate page using
>constructor public
>org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage(org.apache.wicket.Pag
>eParameters)
>and argument status = "ALL" pageNum = "null"
>     at 
>org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory
>.java:212)
>     at 
>org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.ja
>va:89)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.n
>ewPage(BookmarkablePageRequestTarget.java:305)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.g
>etPage(BookmarkablePageRequestTarget.java:320)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.p
>rocessEvents(BookmarkablePageRequestTarget.java:234)
>     at 
>org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abst
>ractRequestCycleProcessor.java:92)
>     at 
>org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1
>250)
>     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
>     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
>     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
>     at 
>org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)
>
>java.lang.reflect.InvocationTargetException
>     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>Method)
>     at 
>sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAcc
>essorImpl.java:62)
>     at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>     at 
>org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory
>.java:188)
>     at 
>org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.ja
>va:89)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.n
>ewPage(BookmarkablePageRequestTarget.java:305)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.g
>etPage(BookmarkablePageRequestTarget.java:320)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.p
>rocessEvents(BookmarkablePageRequestTarget.java:234)
>     at 
>org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abst
>ractRequestCycleProcessor.java:92)
>     at 
>org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1
>250)
>     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
>     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
>     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
>     at 
>org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)
>
>org.apache.wicket.util.string.StringValueConversionException: Unable
>to convert 'null' to an int value
>     at 
>org.apache.wicket.util.string.StringValue.toInt(StringValue.java:505)
>     at 
>org.apache.wicket.util.string.StringValue.toInt(StringValue.java:520)
>     at org.apache.wicket.util.value.ValueMap.getInt(ValueMap.java:275)
>     at 
>org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage.<init>(WorkflowInstan
>ceViewerPage.java:43)
>     at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>     at 
>org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory
>.java:188)
>     at 
>org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.ja
>va:89)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.n
>ewPage(BookmarkablePageRequestTarget.java:305)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.g
>etPage(BookmarkablePageRequestTarget.java:320)
>     at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.p
>rocessEvents(BookmarkablePageRequestTarget.java:234)
>     at 
>org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abst
>ractRequestCycleProcessor.java:92)
>     at 
>org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1
>250)
>     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
>     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
>     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
>     at 
>org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)
>
>Sorry for the massive exception trails - just wanted to give you all
>the info I have on my end.
>
>Thanks,
>Andrea
>
>On 11 June 2015 at 17:13, Andrea DeMarco <dr...@gmail.com>
>wrote:
>> Hi Chris,
>>
>> I have tried with the new batch_stub - same result. Attached is the
>> oodt.out log file.
>>
>> Moreover this is the exception from the batch_stub terminal:
>>
>> andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001
>> [1] 2777
>> andrea@issa:~/oodt/resmgr/bin$ Jun 11, 2015 4:56:28 PM
>> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
>> INFO: XmlRpc Batch Stub started by andrea
>> Jun 11, 2015 4:58:38 PM
>> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub
>> genericExecuteJob
>> INFO: stub attempting to execute class:
>> [org.apache.oodt.cas.workflow.structs.TaskJob]
>> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>> createPgeMetadata
>> INFO: Converting workflow configuration to static metadata...
>> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>> createPgeMetadata
>> INFO: Loading workflow context metadata...
>> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>>createPgeConfig
>> INFO: Create PgeConfig...
>> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>>createPgeConfig
>> INFO: Using default PgeConfigBuilder:
>> org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
>> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance run
>> SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to
>> parse value: null
>> java.io.IOException: Failed to build PgeConfig : Failed to parse value:
>>null
>> at 
>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>>nfigBuilder.java:87)
>> at 
>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.j
>>ava:231)
>> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>> at 
>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>> at 
>>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.ru
>>n(XmlRpcBatchStub.java:229)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.Exception: Failed to parse value: null
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> at 
>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>>nfigBuilder.java:77)
>> ... 5 more
>> Caused by: java.lang.NullPointerException
>> at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
>> at java.util.regex.Matcher.reset(Matcher.java:309)
>> at java.util.regex.Matcher.<init>(Matcher.java:229)
>> at java.util.regex.Pattern.matcher(Pattern.java:1093)
>> at 
>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacem
>>ent(PathUtils.java:321)
>> at 
>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtil
>>s.java:96)
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
>> ... 7 more
>>
>> org.apache.oodt.cas.resource.structs.exceptions.JobInputException:
>> Failed to run task
>> at 
>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104)
>> at 
>>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.ru
>>n(XmlRpcBatchStub.java:229)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: 
>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcep
>>tion:
>> PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value:
>> null
>> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>> at 
>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>> ... 2 more
>> Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>> parse value: null
>> at 
>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>>nfigBuilder.java:87)
>> at 
>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.j
>>ava:231)
>> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>> ... 3 more
>> Caused by: java.lang.Exception: Failed to parse value: null
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> at 
>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>>nfigBuilder.java:77)
>> ... 5 more
>> Caused by: java.lang.NullPointerException
>> at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
>> at java.util.regex.Matcher.reset(Matcher.java:309)
>> at java.util.regex.Matcher.<init>(Matcher.java:229)
>> at java.util.regex.Pattern.matcher(Pattern.java:1093)
>> at 
>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacem
>>ent(PathUtils.java:321)
>> at 
>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtil
>>s.java:96)
>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
>> ... 7 more
>>
>>
>>
>> I have then turned off the resource manager as Valerie suggested,
>> turned off OODT entirely and restarted the job, and lo and behold, it
>> worked, and files were ingested successfully! I ran the job a second
>> time and it worked again...attached also find a log file for this bit
>> (oodt_2.out)
>>
>> Minor progress :) Now to get that resource manager working...
>> Also: what can I do to file the JIRA ticket you need?
>>
>> Cheers,
>> Andrea
>
>
>
>-- 
>Best Regards,
>Andrea DeMarco
>Institute of Space Sciences and Astronomy
>University of Malta

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi Chris,

One other detail: is it perhaps also indicative of an issue in some
way that the Resource Monitor of OPSUI is the only tab/page that still
gives exceptions (after the 0.9 upgrade)?

WicketMessage: Can't instantiate page using constructor public
org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage(org.apache.wicket.PageParameters)
and argument status = "ALL" pageNum = "null"

Root cause:

java.lang.NumberFormatException: For input string: "null"
     at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
     at java.lang.Integer.parseInt(Integer.java:580)
     at java.lang.Integer.parseInt(Integer.java:615)
     at org.apache.wicket.util.string.StringValue.toInt(StringValue.java:501)
     at org.apache.wicket.util.string.StringValue.toInt(StringValue.java:520)
     at org.apache.wicket.util.value.ValueMap.getInt(ValueMap.java:275)
     at org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage.<init>(WorkflowInstanceViewerPage.java:43)
     at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
     at org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory.java:188)
     at org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java:89)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.newPage(BookmarkablePageRequestTarget.java:305)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.getPage(BookmarkablePageRequestTarget.java:320)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.processEvents(BookmarkablePageRequestTarget.java:234)
     at org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(AbstractRequestCycleProcessor.java:92)
     at org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1250)
     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
     at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)
     at org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:319)
     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
     at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
     at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
     at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
     at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
     at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
     at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
     at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
     at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
     at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
     at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
     at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
     at java.lang.Thread.run(Thread.java:745)

Complete stack:

org.apache.wicket.WicketRuntimeException: Can't instantiate page using
constructor public
org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage(org.apache.wicket.PageParameters)
and argument status = "ALL" pageNum = "null"
     at org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory.java:212)
     at org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java:89)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.newPage(BookmarkablePageRequestTarget.java:305)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.getPage(BookmarkablePageRequestTarget.java:320)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.processEvents(BookmarkablePageRequestTarget.java:234)
     at org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(AbstractRequestCycleProcessor.java:92)
     at org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1250)
     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
     at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)

java.lang.reflect.InvocationTargetException
     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
     at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
     at org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory.java:188)
     at org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java:89)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.newPage(BookmarkablePageRequestTarget.java:305)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.getPage(BookmarkablePageRequestTarget.java:320)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.processEvents(BookmarkablePageRequestTarget.java:234)
     at org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(AbstractRequestCycleProcessor.java:92)
     at org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1250)
     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
     at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)

org.apache.wicket.util.string.StringValueConversionException: Unable
to convert 'null' to an int value
     at org.apache.wicket.util.string.StringValue.toInt(StringValue.java:505)
     at org.apache.wicket.util.string.StringValue.toInt(StringValue.java:520)
     at org.apache.wicket.util.value.ValueMap.getInt(ValueMap.java:275)
     at org.apache.oodt.pcs.opsui.WorkflowInstanceViewerPage.<init>(WorkflowInstanceViewerPage.java:43)
     at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
     at org.apache.wicket.session.DefaultPageFactory.createPage(DefaultPageFactory.java:188)
     at org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java:89)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.newPage(BookmarkablePageRequestTarget.java:305)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.getPage(BookmarkablePageRequestTarget.java:320)
     at org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.processEvents(BookmarkablePageRequestTarget.java:234)
     at org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(AbstractRequestCycleProcessor.java:92)
     at org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1250)
     at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
     at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
     at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
     at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)

Sorry for the massive exception trails - just wanted to give you all
the info I have on my end.

Thanks,
Andrea

On 11 June 2015 at 17:13, Andrea DeMarco <dr...@gmail.com> wrote:
> Hi Chris,
>
> I have tried with the new batch_stub - same result. Attached is the
> oodt.out log file.
>
> Moreover this is the exception from the batch_stub terminal:
>
> andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001
> [1] 2777
> andrea@issa:~/oodt/resmgr/bin$ Jun 11, 2015 4:56:28 PM
> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
> INFO: XmlRpc Batch Stub started by andrea
> Jun 11, 2015 4:58:38 PM
> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub
> genericExecuteJob
> INFO: stub attempting to execute class:
> [org.apache.oodt.cas.workflow.structs.TaskJob]
> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
> createPgeMetadata
> INFO: Converting workflow configuration to static metadata...
> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
> createPgeMetadata
> INFO: Loading workflow context metadata...
> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance createPgeConfig
> INFO: Create PgeConfig...
> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance createPgeConfig
> INFO: Using default PgeConfigBuilder:
> org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
> Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance run
> SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to
> parse value: null
> java.io.IOException: Failed to build PgeConfig : Failed to parse value: null
> at org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:87)
> at org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:231)
> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> at org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to parse value: null
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> at org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:77)
> ... 5 more
> Caused by: java.lang.NullPointerException
> at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
> at java.util.regex.Matcher.reset(Matcher.java:309)
> at java.util.regex.Matcher.<init>(Matcher.java:229)
> at java.util.regex.Pattern.matcher(Pattern.java:1093)
> at org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacement(PathUtils.java:321)
> at org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils.java:96)
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
> ... 7 more
>
> org.apache.oodt.cas.resource.structs.exceptions.JobInputException:
> Failed to run task
> at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104)
> at org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceException:
> PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value:
> null
> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> ... 2 more
> Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
> parse value: null
> at org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:87)
> at org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:231)
> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> ... 3 more
> Caused by: java.lang.Exception: Failed to parse value: null
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> at org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:77)
> ... 5 more
> Caused by: java.lang.NullPointerException
> at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
> at java.util.regex.Matcher.reset(Matcher.java:309)
> at java.util.regex.Matcher.<init>(Matcher.java:229)
> at java.util.regex.Pattern.matcher(Pattern.java:1093)
> at org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacement(PathUtils.java:321)
> at org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils.java:96)
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
> ... 7 more
>
>
>
> I have then turned off the resource manager as Valerie suggested,
> turned off OODT entirely and restarted the job, and lo and behold, it
> worked, and files were ingested successfully! I ran the job a second
> time and it worked again...attached also find a log file for this bit
> (oodt_2.out)
>
> Minor progress :) Now to get that resource manager working...
> Also: what can I do to file the JIRA ticket you need?
>
> Cheers,
> Andrea



-- 
Best Regards,
Andrea DeMarco
Institute of Space Sciences and Astronomy
University of Malta

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi Chris,

Here goes: http://pastebin.com/5c7aRBFQ

Cheers,
Andrea

On 14 June 2015 at 19:19, Chris Mattmann <ch...@gmail.com> wrote:
> Thanks Andrea, can you show me your tasks.xml from $WORKFLOW_HOME/policy?
>
> On Thu, Jun 11, 2015 at 8:42 AM, Andrea DeMarco
> <dr...@gmail.com> wrote:
>> Hi Chris,
>>
>> JIRA Ticket filed - though bear with me as I'm very new to this - and
>> I probably don't know how to properly fill in those tickets.
>>
>> Here are some pastebins:
>> 1) batch_stub http://pastebin.com/17gb1dBK
>> 2) pgeconfig http://pastebin.com/iwqWHc1y
>>
>> About the OPSUI - the workflow page was fine, showing progress bars
>> and latest ingestions etc. It's the resource monitor page that was
>> busted. Everything else was ok. I'll file a ticket for that as well
>> momentarily.
>>
>> Will also subscribe to your mailing list :)
>>
>> Thanks,
>> Andrea



-- 
Best Regards,
Andrea DeMarco
Institute of Space Sciences and Astronomy
University of Malta

Re: OODT Documentation

Posted by Chris Mattmann <ch...@gmail.com>.

Thanks Andrea, can you show me your tasks.xml from $WORKFLOW_HOME/policy?

On Thu, Jun 11, 2015 at 8:42 AM, Andrea DeMarco
<dr...@gmail.com> wrote:
> Hi Chris,
>
> JIRA Ticket filed - though bear with me as I'm very new to this - and
> I probably don't know how to properly fill in those tickets.
>
> Here are some pastebins:
> 1) batch_stub http://pastebin.com/17gb1dBK
> 2) pgeconfig http://pastebin.com/iwqWHc1y
>
> About the OPSUI - the workflow page was fine, showing progress bars
> and latest ingestions etc. It's the resource monitor page that was
> busted. Everything else was ok. I'll file a ticket for that as well
> momentarily.
>
> Will also subscribe to your mailing list :)
>
> Thanks,
> Andrea

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi Chris,

JIRA Ticket filed - though bear with me as I'm very new to this - and
I probably don't know how to properly fill in those tickets.

Here are some pastebins:
1) batch_stub http://pastebin.com/17gb1dBK
2) pgeconfig http://pastebin.com/iwqWHc1y

About the OPSUI - the workflow page was fine, showing progress bars
and latest ingestions etc. It's the resource monitor page that was
busted. Everything else was ok. I'll file a ticket for that as well
momentarily.

Will also subscribe to your mailing list :)

Thanks,
Andrea

Re: OODT Documentation

Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.

Thanks Andrea.

To file the JIRA ticket, head over to
http://issues.apache.org/jira/browse/OODT,
register for an account, and then file a new ticket stating that the
OODT PGE should print out what keys are actually null before throwing
an exception which would be make it easier to debug.

Glad the WM worked!

So, now I’m wondering what key is actually null. Can you pass along your
batch_stub file that you are running; can you also pass along your
PGEConfig.xml
file? using something like paste.bin on both would be great b/c I think the
M/L strips attachments.

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Andrea DeMarco <dr...@gmail.com>
Date: Thursday, June 11, 2015 at 8:13 AM
To: jpluser <ch...@jpl.nasa.gov>
Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
"Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
<pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
"dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: Re: OODT Documentation

>Hi Chris,
>
>I have tried with the new batch_stub - same result. Attached is the
>oodt.out log file.
>
>Moreover this is the exception from the batch_stub terminal:
>
>andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001
>[1] 2777
>andrea@issa:~/oodt/resmgr/bin$ Jun 11, 2015 4:56:28 PM
>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
>INFO: XmlRpc Batch Stub started by andrea
>Jun 11, 2015 4:58:38 PM
>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub
>genericExecuteJob
>INFO: stub attempting to execute class:
>[org.apache.oodt.cas.workflow.structs.TaskJob]
>Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>createPgeMetadata
>INFO: Converting workflow configuration to static metadata...
>Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>createPgeMetadata
>INFO: Loading workflow context metadata...
>Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>createPgeConfig
>INFO: Create PgeConfig...
>Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
>createPgeConfig
>INFO: Using default PgeConfigBuilder:
>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
>Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance run
>SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to
>parse value: null
>java.io.IOException: Failed to build PgeConfig : Failed to parse value:
>null
>at 
>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCon
>figBuilder.java:87)
>at 
>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.ja
>va:231)
>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>at 
>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run
>(XmlRpcBatchStub.java:229)
>at java.lang.Thread.run(Thread.java:745)
>Caused by: java.lang.Exception: Failed to parse value: null
>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>at 
>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCon
>figBuilder.java:77)
>... 5 more
>Caused by: java.lang.NullPointerException
>at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
>at java.util.regex.Matcher.reset(Matcher.java:309)
>at java.util.regex.Matcher.<init>(Matcher.java:229)
>at java.util.regex.Pattern.matcher(Pattern.java:1093)
>at 
>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplaceme
>nt(PathUtils.java:321)
>at 
>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils
>.java:96)
>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
>... 7 more
>
>org.apache.oodt.cas.resource.structs.exceptions.JobInputException:
>Failed to run task
>at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104)
>at 
>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run
>(XmlRpcBatchStub.java:229)
>at java.lang.Thread.run(Thread.java:745)
>Caused by: 
>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcept
>ion:
>PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value:
>null
>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>... 2 more
>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>parse value: null
>at 
>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCon
>figBuilder.java:87)
>at 
>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.ja
>va:231)
>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>... 3 more
>Caused by: java.lang.Exception: Failed to parse value: null
>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>at 
>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCon
>figBuilder.java:77)
>... 5 more
>Caused by: java.lang.NullPointerException
>at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
>at java.util.regex.Matcher.reset(Matcher.java:309)
>at java.util.regex.Matcher.<init>(Matcher.java:229)
>at java.util.regex.Pattern.matcher(Pattern.java:1093)
>at 
>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplaceme
>nt(PathUtils.java:321)
>at 
>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils
>.java:96)
>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
>... 7 more
>
>
>
>I have then turned off the resource manager as Valerie suggested,
>turned off OODT entirely and restarted the job, and lo and behold, it
>worked, and files were ingested successfully! I ran the job a second
>time and it worked again...attached also find a log file for this bit
>(oodt_2.out)
>
>Minor progress :) Now to get that resource manager working...
>Also: what can I do to file the JIRA ticket you need?
>
>Cheers,
>Andrea

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi Chris,

I have tried with the new batch_stub - same result. Attached is the
oodt.out log file.

Moreover this is the exception from the batch_stub terminal:

andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001
[1] 2777
andrea@issa:~/oodt/resmgr/bin$ Jun 11, 2015 4:56:28 PM
org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
INFO: XmlRpc Batch Stub started by andrea
Jun 11, 2015 4:58:38 PM
org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub
genericExecuteJob
INFO: stub attempting to execute class:
[org.apache.oodt.cas.workflow.structs.TaskJob]
Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
createPgeMetadata
INFO: Converting workflow configuration to static metadata...
Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance
createPgeMetadata
INFO: Loading workflow context metadata...
Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance createPgeConfig
INFO: Create PgeConfig...
Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance createPgeConfig
INFO: Using default PgeConfigBuilder:
org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
Jun 11, 2015 4:58:38 PM org.apache.oodt.cas.pge.PGETaskInstance run
SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to
parse value: null
java.io.IOException: Failed to build PgeConfig : Failed to parse value: null
at org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:87)
at org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:231)
at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
at org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: Failed to parse value: null
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
at org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:77)
... 5 more
Caused by: java.lang.NullPointerException
at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
at java.util.regex.Matcher.reset(Matcher.java:309)
at java.util.regex.Matcher.<init>(Matcher.java:229)
at java.util.regex.Pattern.matcher(Pattern.java:1093)
at org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacement(PathUtils.java:321)
at org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils.java:96)
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
... 7 more

org.apache.oodt.cas.resource.structs.exceptions.JobInputException:
Failed to run task
at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104)
at org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceException:
PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value:
null
at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
... 2 more
Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
parse value: null
at org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:87)
at org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:231)
at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
... 3 more
Caused by: java.lang.Exception: Failed to parse value: null
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
at org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:77)
... 5 more
Caused by: java.lang.NullPointerException
at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
at java.util.regex.Matcher.reset(Matcher.java:309)
at java.util.regex.Matcher.<init>(Matcher.java:229)
at java.util.regex.Pattern.matcher(Pattern.java:1093)
at org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacement(PathUtils.java:321)
at org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils.java:96)
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
... 7 more



I have then turned off the resource manager as Valerie suggested,
turned off OODT entirely and restarted the job, and lo and behold, it
worked, and files were ingested successfully! I ran the job a second
time and it worked again...attached also find a log file for this bit
(oodt_2.out)

Minor progress :) Now to get that resource manager working...
Also: what can I do to file the JIRA ticket you need?

Cheers,
Andrea

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi Chris,

I definitely did not get the rest of that conversation :D
I'll give this a go and let you know, as well as give it a go without
the resource manager.

Cheers,
Andrea

Re: OODT Documentation

Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.

Hi Andrea,

I sent an email - did you get it? My suggestion would be to make sure
that you update your batch stub script per my instructions here:

http://s.apache.org/YDI

Let me know thanks.

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Andrea DeMarco <dr...@gmail.com>
Date: Thursday, June 11, 2015 at 7:30 AM
To: jpluser <ch...@jpl.nasa.gov>
Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
"Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
<pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
"dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: Re: OODT Documentation

>Hi Chris,
>
>Was wondering if there are any updates on your end, or perhaps
>anything we can try on our installation?
>
>Thanks,
>Andrea
>
>On 9 June 2015 at 16:36, Andrea DeMarco <dr...@gmail.com>
>wrote:
>> Hi Chris,
>>
>> Some additional detail:
>>
>> When stopping/starting the workflow manager before kicking the PGE, the
>> terminal (post PGE submission) also prints out the following:
>>
>> andrea@issa:~/oodt/workflow/bin$ ./wmgr stop
>> andrea@issa:~/oodt/workflow/bin$ ./wmgr start
>> Workflow Manager started PID file
>> (/home/andrea/oodt/workflow/run/cas.workflow.pid).
>> andrea@issa:~/oodt/workflow/bin$ Jun 09, 2015 4:34:24 PM
>> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
>> INFO: Loading Workflow Manager Configuration Properties from:
>> [/home/andrea/oodt/workflow/etc/workflow.properties]
>> Jun 09, 2015 4:34:25 PM
>> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
>> INFO: Workflow Manager started by andrea
>> Jun 09, 2015 4:34:29 PM
>> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
>> INFO: WorkflowManager: Received event: fileconcatenator-pge
>> Jun 09, 2015 4:34:29 PM
>> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
>> INFO: WorkflowManager: Workflow FileConcatenatorWorkflow retrieved for
>>event
>> fileconcatenator-pge
>> Jun 09, 2015 4:34:29 PM
>> org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> setRClient
>> INFO: Resource Manager Job Submission enabled to:
>>[http://localhost:9002]
>> Jun 09, 2015 4:34:29 PM
>> org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> checkTaskRequiredMetadata
>> INFO: All required metadata fields present for task: [FileConcatenator]
>> Jun 09, 2015 4:34:29 PM
>> org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>>submitJob
>> FINEST: [{job.queueName=high,
>> job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>> job.name=urn:oodt:FileConcatenator, job.id=, job.status=, job.load=2,
>> job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>> 
>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileCo
>>ncatenatorPGETask,
>> 
>>task.config={PGETask_ConfigFilePath=/home/andrea/oodt/pge/file_concatenat
>>or/pge-configs/PGEConfig.xml,
>> 
>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer
>>.LocalDataTransferFactory,
>> 
>>PCS_ActionRepoFile=file:/home/andrea/oodt/crawler/policy/crawler-config.x
>>ml,
>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>> PCS_WorkflowManagerUrl=http://localhost:9001,
>> PCS_FileManagerUrl=http://localhost:9000,
>>PGETask_Name=FileConcatenator},
>> task.metadata={TaskId=[urn:oodt:FileConcatenator],
>> WorkflowManagerUrl=[http://issa:9001],
>> JobId=[a2eecfc3-0eb4-11e5-ac1b-fff04e088d71], ProcessingNode=[issa],
>> RunID=[testNumber1],
>> WorkflowInstId=[a2eecfc3-0eb4-11e5-ac1b-fff04e088d71]}}]
>>
>> Not sure what it all means, but it may help :)
>>
>> Cheers,
>> Andrea
>>
>> On 9 June 2015 at 16:03, Andrea DeMarco <dr...@gmail.com>
>>wrote:
>>>
>>> Hi Chris,
>>>
>>> No problem - I can imagine that checklist to be quite extensive :)
>>>
>>> I'll try and help as best I can:
>>>
>>> 1) The output when starting the batch_stub is:
>>> andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001
>>> [1] 2535
>>> andrea@issa:~/oodt/resmgr/bin$ Jun 09, 2015 3:53:04 PM
>>> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
>>> INFO: XmlRpc Batch Stub started by andrea
>>>
>>> 2) The latest oodt.out log file (after running the below) is attached .
>>>
>>> 3) Kicking of the PGE with the following command (and terminal output):
>>>
>>> andrea@issa:~/oodt/workflow/bin$ ./wmgr-client --url
>>>http://localhost:9001
>>> --operation --sendEvent --eventName fileconcatenator-pge --metaData
>>>--key
>>> RunID testNumber1
>>> Sending event 'fileconcatenator-pge'... SUCCESS
>>>
>>> I have now shut down my VM and restarted everything. When kicking off
>>>the
>>> PGE, this is the terminal output from the same terminal that started
>>>the
>>> batch_stub:
>>>
>>> Jun 09, 2015 3:57:26 PM
>>> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub
>>>genericExecuteJob
>>> INFO: stub attempting to execute class:
>>> [org.apache.oodt.cas.workflow.structs.TaskJob]
>>> Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
>>> createPgeMetadata
>>> INFO: Converting workflow configuration to static metadata...
>>> Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
>>> createPgeMetadata
>>> INFO: Loading workflow context metadata...
>>> Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
>>> createPgeConfig
>>> INFO: Create PgeConfig...
>>> Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
>>> createPgeConfig
>>> INFO: Using default PgeConfigBuilder:
>>> org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
>>> Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance run
>>> SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>>> value: null
>>> java.io.IOException: Failed to build PgeConfig : Failed to parse value:
>>> null
>>> at
>>> 
>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
>>>onfigBuilder.java:87)
>>> at
>>> 
>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.
>>>java:231)
>>> at 
>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>>> at 
>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>>> at
>>> 
>>>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.r
>>>un(XmlRpcBatchStub.java:229)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: java.lang.Exception: Failed to parse value: null
>>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>>> at
>>> 
>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
>>>onfigBuilder.java:77)
>>> ... 5 more
>>> Caused by: java.lang.NullPointerException
>>> at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>>> at java.util.regex.Matcher.reset(Matcher.java:308)
>>> at java.util.regex.Matcher.<init>(Matcher.java:228)
>>> at java.util.regex.Pattern.matcher(Pattern.java:1088)
>>> at
>>> 
>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplace
>>>ment(PathUtils.java:321)
>>> at
>>> 
>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUti
>>>ls.java:96)
>>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
>>> ... 7 more
>>>
>>> org.apache.oodt.cas.resource.structs.exceptions.JobInputException:
>>>Failed
>>> to run task
>>> at 
>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104)
>>> at
>>> 
>>>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.r
>>>un(XmlRpcBatchStub.java:229)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by:
>>> 
>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExce
>>>ption:
>>> PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value:
>>>null
>>> at 
>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>>> at 
>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>>> ... 2 more
>>> Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>>> parse value: null
>>> at
>>> 
>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
>>>onfigBuilder.java:87)
>>> at
>>> 
>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.
>>>java:231)
>>> at 
>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>>> ... 3 more
>>> Caused by: java.lang.Exception: Failed to parse value: null
>>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>>> at
>>> 
>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
>>>onfigBuilder.java:77)
>>> ... 5 more
>>> Caused by: java.lang.NullPointerException
>>> at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>>> at java.util.regex.Matcher.reset(Matcher.java:308)
>>> at java.util.regex.Matcher.<init>(Matcher.java:228)
>>> at java.util.regex.Pattern.matcher(Pattern.java:1088)
>>> at
>>> 
>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplace
>>>ment(PathUtils.java:321)
>>> at
>>> 
>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUti
>>>ls.java:96)
>>> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
>>> ... 7 more
>>>
>>>
>>>
>>> Cheers,
>>> Andrea
>>>
>>>
>>> On 9 June 2015 at 15:47, Mattmann, Chris A (3980)
>>> <ch...@jpl.nasa.gov> wrote:
>>>>
>>>> Thanks Andrea, OK bear with me as we go through our checklist :)
>>>>
>>>> Does anything show in $OODT_HOME/log/oodt.log?
>>>>
>>>> How are you kicking off the PGE? Does the terminal in which you
>>>> started the batch stub print anything?
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> Chris Mattmann, Ph.D.
>>>> Chief Architect
>>>> Instrument Software and Science Data Systems Section (398)
>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>> Office: 168-519, Mailstop: 168-527
>>>> Email: chris.a.mattmann@nasa.gov
>>>> WWW:  http://sunset.usc.edu/~mattmann/
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> Adjunct Associate Professor, Computer Science Department
>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Andrea DeMarco <dr...@gmail.com>
>>>> Date: Tuesday, June 9, 2015 at 6:44 AM
>>>> To: jpluser <ch...@jpl.nasa.gov>
>>>> Cc: "Wyngaard, Jane R (3980-Affiliate)"
>>>><Ja...@jpl.nasa.gov>,
>>>> "Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian
>>>>ZARB
>>>> ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
>>>> <pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
>>>> "dev@oodt.apache.org" <de...@oodt.apache.org>
>>>> Subject: Re: OODT Documentation
>>>>
>>>> >Hi Chris,
>>>> >
>>>> >
>>>> >Yup, the jar file is present in both resmgr/lib and workflow/lib. You
>>>> > can
>>>> >have a look at my JAR contents at:
>>>> >
>>>>
>>>> > 
>>>>>https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar
>>>>>?dl=
>>>> >0
>>>>
>>>> > 
>>>>><https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.ja
>>>>>r?dl
>>>> >=0>  - perhaps something's off there?
>>>> >
>>>> >
>>>> >Cheers,
>>>> >Andrea
>>>> >
>>>>
>>>
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Andrea DeMarco
>> Institute of Space Sciences and Astronomy
>> University of Malta
>
>
>
>-- 
>Best Regards,
>Andrea DeMarco
>Institute of Space Sciences and Astronomy
>University of Malta

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi Chris,

Some additional detail:

When stopping/starting the workflow manager before kicking the PGE, the
terminal (post PGE submission) also prints out the following:

andrea@issa:~/oodt/workflow/bin$ ./wmgr stop
andrea@issa:~/oodt/workflow/bin$ ./wmgr start
Workflow Manager started PID file
(/home/andrea/oodt/workflow/run/cas.workflow.pid).
andrea@issa:~/oodt/workflow/bin$ Jun 09, 2015 4:34:24 PM
org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties
INFO: Loading Workflow Manager Configuration Properties from:
[/home/andrea/oodt/workflow/etc/workflow.properties]
Jun 09, 2015 4:34:25 PM
org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by andrea
Jun 09, 2015 4:34:29 PM
org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: fileconcatenator-pge
Jun 09, 2015 4:34:29 PM
org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow FileConcatenatorWorkflow retrieved for
event fileconcatenator-pge
Jun 09, 2015 4:34:29 PM
org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
setRClient
INFO: Resource Manager Job Submission enabled to: [http://localhost:9002]
Jun 09, 2015 4:34:29 PM
org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
checkTaskRequiredMetadata
INFO: All required metadata fields present for task: [FileConcatenator]
Jun 09, 2015 4:34:29 PM
org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient submitJob
FINEST: [{job.queueName=high,
job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
job.name=urn:oodt:FileConcatenator,
job.id=, job.status=, job.load=2,
job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGETask,
task.config={PGETask_ConfigFilePath=/home/andrea/oodt/pge/file_concatenator/pge-configs/PGEConfig.xml,
PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory,
PCS_ActionRepoFile=file:/home/andrea/oodt/crawler/policy/crawler-config.xml,
PCS_MetFileExtension=met, PGETask_DumpMetadata=true, PCS_WorkflowManagerUrl=
http://localhost:9001, PCS_FileManagerUrl=http://localhost:9000,
PGETask_Name=FileConcatenator},
task.metadata={TaskId=[urn:oodt:FileConcatenator], WorkflowManagerUrl=[
http://issa:9001], JobId=[a2eecfc3-0eb4-11e5-ac1b-fff04e088d71],
ProcessingNode=[issa], RunID=[testNumber1],
WorkflowInstId=[a2eecfc3-0eb4-11e5-ac1b-fff04e088d71]}}]

Not sure what it all means, but it may help :)

Cheers,
Andrea

On 9 June 2015 at 16:03, Andrea DeMarco <dr...@gmail.com> wrote:

> Hi Chris,
>
> No problem - I can imagine that checklist to be quite extensive :)
>
> I'll try and help as best I can:
>
> 1) The output when starting the batch_stub is:
> andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001
> [1] 2535
> andrea@issa:~/oodt/resmgr/bin$ Jun 09, 2015 3:53:04 PM
> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
> INFO: XmlRpc Batch Stub started by andrea
>
> 2) The latest oodt.out log file (after running the below) is attached .
>
> 3) Kicking of the PGE with the following command (and terminal output):
>
> andrea@issa:~/oodt/workflow/bin$ ./wmgr-client --url http://localhost:9001
> --operation --sendEvent --eventName fileconcatenator-pge --metaData --key
> RunID testNumber1
> Sending event 'fileconcatenator-pge'... SUCCESS
>
> I have now shut down my VM and restarted everything. When kicking off the
> PGE, this is the terminal output from the same terminal that started the
> batch_stub:
>
> Jun 09, 2015 3:57:26 PM
> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub genericExecuteJob
> INFO: stub attempting to execute class:
> [org.apache.oodt.cas.workflow.structs.TaskJob]
> Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> createPgeMetadata
> INFO: Converting workflow configuration to static metadata...
> Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> createPgeMetadata
> INFO: Loading workflow context metadata...
> Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> createPgeConfig
> INFO: Create PgeConfig...
> Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
> createPgeConfig
> INFO: Using default PgeConfigBuilder:
> org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
> Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance run
> SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
> value: null
> java.io.IOException: Failed to build PgeConfig : Failed to parse value:
> null
> at
> org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:87)
> at
> org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:231)
> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> at
> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Failed to parse value: null
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> at
> org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:77)
> ... 5 more
> Caused by: java.lang.NullPointerException
> at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> at java.util.regex.Matcher.reset(Matcher.java:308)
> at java.util.regex.Matcher.<init>(Matcher.java:228)
> at java.util.regex.Pattern.matcher(Pattern.java:1088)
> at
> org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacement(PathUtils.java:321)
> at
> org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils.java:96)
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
> ... 7 more
>
> org.apache.oodt.cas.resource.structs.exceptions.JobInputException: Failed
> to run task
> at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104)
> at
> org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceException:
> PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value: null
> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> ... 2 more
> Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
> parse value: null
> at
> org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:87)
> at
> org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:231)
> at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> ... 3 more
> Caused by: java.lang.Exception: Failed to parse value: null
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> at
> org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:77)
> ... 5 more
> Caused by: java.lang.NullPointerException
> at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> at java.util.regex.Matcher.reset(Matcher.java:308)
> at java.util.regex.Matcher.<init>(Matcher.java:228)
> at java.util.regex.Pattern.matcher(Pattern.java:1088)
> at
> org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacement(PathUtils.java:321)
> at
> org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils.java:96)
> at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
> ... 7 more
>
>
>
> Cheers,
> Andrea
>
>
> On 9 June 2015 at 15:47, Mattmann, Chris A (3980) <
> chris.a.mattmann@jpl.nasa.gov> wrote:
>
>> Thanks Andrea, OK bear with me as we go through our checklist :)
>>
>> Does anything show in $OODT_HOME/log/oodt.log?
>>
>> How are you kicking off the PGE? Does the terminal in which you
>> started the batch stub print anything?
>>
>> Cheers,
>> Chris
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Chief Architect
>> Instrument Software and Science Data Systems Section (398)
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 168-519, Mailstop: 168-527
>> Email: chris.a.mattmann@nasa.gov
>> WWW:  http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Associate Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>> -----Original Message-----
>> From: Andrea DeMarco <dr...@gmail.com>
>> Date: Tuesday, June 9, 2015 at 6:44 AM
>> To: jpluser <ch...@jpl.nasa.gov>
>> Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
>> "Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
>> ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
>> <pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
>> "dev@oodt.apache.org" <de...@oodt.apache.org>
>> Subject: Re: OODT Documentation
>>
>> >Hi Chris,
>> >
>> >
>> >Yup, the jar file is present in both resmgr/lib and workflow/lib. You can
>> >have a look at my JAR contents at:
>> >
>> >
>> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar?dl=
>> >0
>> ><
>> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar?dl
>> >=0>  - perhaps something's off there?
>> >
>> >
>> >Cheers,
>> >Andrea
>> >
>>
>>
>
>


-- 
Best Regards,
Andrea DeMarco
Institute of Space Sciences and Astronomy
University of Malta

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi Chris,

No problem - I can imagine that checklist to be quite extensive :)

I'll try and help as best I can:

1) The output when starting the batch_stub is:
andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001
[1] 2535
andrea@issa:~/oodt/resmgr/bin$ Jun 09, 2015 3:53:04 PM
org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init>
INFO: XmlRpc Batch Stub started by andrea

2) The latest oodt.out log file (after running the below) is attached .

3) Kicking of the PGE with the following command (and terminal output):

andrea@issa:~/oodt/workflow/bin$ ./wmgr-client --url http://localhost:9001
--operation --sendEvent --eventName fileconcatenator-pge --metaData --key
RunID testNumber1
Sending event 'fileconcatenator-pge'... SUCCESS

I have now shut down my VM and restarted everything. When kicking off the
PGE, this is the terminal output from the same terminal that started the
batch_stub:

Jun 09, 2015 3:57:26 PM
org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub genericExecuteJob
INFO: stub attempting to execute class:
[org.apache.oodt.cas.workflow.structs.TaskJob]
Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
createPgeMetadata
INFO: Converting workflow configuration to static metadata...
Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
createPgeMetadata
INFO: Loading workflow context metadata...
Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
createPgeConfig
INFO: Create PgeConfig...
Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance
createPgeConfig
INFO: Using default PgeConfigBuilder:
org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder
Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance run
SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
value: null
java.io.IOException: Failed to build PgeConfig : Failed to parse value: null
at
org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:87)
at
org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:231)
at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
at
org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: Failed to parse value: null
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
at
org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:77)
... 5 more
Caused by: java.lang.NullPointerException
at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
at java.util.regex.Matcher.reset(Matcher.java:308)
at java.util.regex.Matcher.<init>(Matcher.java:228)
at java.util.regex.Pattern.matcher(Pattern.java:1088)
at
org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacement(PathUtils.java:321)
at
org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils.java:96)
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
... 7 more

org.apache.oodt.cas.resource.structs.exceptions.JobInputException: Failed
to run task
at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104)
at
org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229)
at java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceException:
PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value: null
at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
... 2 more
Caused by: java.io.IOException: Failed to build PgeConfig : Failed to parse
value: null
at
org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:87)
at
org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:231)
at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
... 3 more
Caused by: java.lang.Exception: Failed to parse value: null
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
at
org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:77)
... 5 more
Caused by: java.lang.NullPointerException
at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
at java.util.regex.Matcher.reset(Matcher.java:308)
at java.util.regex.Matcher.<init>(Matcher.java:228)
at java.util.regex.Pattern.matcher(Pattern.java:1088)
at
org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacement(PathUtils.java:321)
at
org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtils.java:96)
at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486)
... 7 more

Cheers,
Andrea

On 9 June 2015 at 15:47, Mattmann, Chris A (3980) <
chris.a.mattmann@jpl.nasa.gov> wrote:

> Thanks Andrea, OK bear with me as we go through our checklist :)
>
> Does anything show in $OODT_HOME/log/oodt.log?
>
> How are you kicking off the PGE? Does the terminal in which you
> started the batch stub print anything?
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW:  http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
> -----Original Message-----
> From: Andrea DeMarco <dr...@gmail.com>
> Date: Tuesday, June 9, 2015 at 6:44 AM
> To: jpluser <ch...@jpl.nasa.gov>
> Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
> "Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
> ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
> <pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
> "dev@oodt.apache.org" <de...@oodt.apache.org>
> Subject: Re: OODT Documentation
>
> >Hi Chris,
> >
> >
> >Yup, the jar file is present in both resmgr/lib and workflow/lib. You can
> >have a look at my JAR contents at:
> >
> >
> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar?dl=
> >0
> ><
> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar?dl
> >=0>  - perhaps something's off there?
> >
> >
> >Cheers,
> >Andrea
> >
>
>

Re: OODT Documentation

Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.

Thanks Andrea, OK bear with me as we go through our checklist :)

Does anything show in $OODT_HOME/log/oodt.log?

How are you kicking off the PGE? Does the terminal in which you
started the batch stub print anything?

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Andrea DeMarco <dr...@gmail.com>
Date: Tuesday, June 9, 2015 at 6:44 AM
To: jpluser <ch...@jpl.nasa.gov>
Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
"Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
<pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
"dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: Re: OODT Documentation

>Hi Chris,
>
>
>Yup, the jar file is present in both resmgr/lib and workflow/lib. You can
>have a look at my JAR contents at:
>
>https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar?dl=
>0 
><https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar?dl
>=0>  - perhaps something's off there?
>
>
>Cheers,
>Andrea
>

RE: OODT Documentation

Posted by "Mallder, Valerie" <Va...@jhuapl.edu>.

Thanks Chris,


No worries. I never fixed it myself because I didn't have a good understanding of how y'all organized all of your files and how they get deployed and I didn't know exactly where the change should be made.  But, since then I figured out a lot of all of that and if I ran into the problem again, I was going to go head and fix it for you. I just haven't had a chance to run with 0.9 yet.  No need to make a point release just for this.

Val



Valerie A. Mallder
New Horizons Deputy Mission System Engineer
Johns Hopkins University/Applied Physics Laboratory


> -----Original Message-----
> From: Mattmann, Chris A (3980) [mailto:chris.a.mattmann@jpl.nasa.gov]
> Sent: Tuesday, June 09, 2015 10:06 AM
> To: Mallder, Valerie; dev@oodt.apache.org
> Cc: Wyngaard, Jane R (3980-Affiliate); Starch, Michael D (398M); Kristian ZARB
> ADAMI; Ramirez, Paul M (398M); Alessio Magro
> Subject: Re: OODT Documentation
> 
> Sigh, sorry Val:
> 
> [chipotle:~/tmp/oodt0.9] mattmann% svn commit -m "Finally fully implement
> OODT-785 and get the batch stub legacy mode props in there. Thanks to Val."
> Sending
> mvn/archetypes/radix/src/main/resources/archetype-resources/resmgr/src/main
> /resources/bin/batch_stub
> Transmitting file data .
> Committed revision 1684425.
> [chipotle:~/tmp/oodt0.9] mattmann%
> 
> 
> Looking at:
> http://svn.apache.org/viewvc?view=revision&revision=1637785
> 
> 
> I see that this was only done and committed in the resmgr code for trunk and not
> in OODT RADIX. Anyways, above I just committed it in RADIX. Ugh, sorry for that.
> Happy to make a point release if needed for an 0.9.1.
> 
> Cheers,
> Chris
> 
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398) NASA Jet
> Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW:  http://sunset.usc.edu/~mattmann/
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++
> Adjunct Associate Professor, Computer Science Department University of
> Southern California, Los Angeles, CA 90089 USA
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++
> 
> 
> 
> 
> -----Original Message-----
> From: <Mallder>, Valerie <Va...@jhuapl.edu>
> Date: Tuesday, June 9, 2015 at 6:59 AM
> To: "dev@oodt.apache.org" <de...@oodt.apache.org>, jpluser
> <ch...@jpl.nasa.gov>
> Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
> "Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
> ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
> <pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>
> Subject: RE: OODT Documentation
> 
> >If you are using the RADiX archetype, make sure the following two lines
> >are in the batch_stub script The batch_stub script is located at:
> >oodt/resmgr/bin.
> >
> >-Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \
> >-Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \
> >
> >I just checked here
> >
> >http://svn.apache.org/viewvc/oodt/tags/0.9-rc1/mvn/archetypes/radix/src
> >/ma
> >in/resources/archetype-resources/resmgr/src/main/resources/bin/batch_stub?
> >view=markup
> >
> >and see that it batch_stub in the RADiX archetype is still missing
> >those two lines.  JIRA issue OODT-785 is marked fixed in 0.8, but it
> >was never been implemented in the RADiX archetype.
> >
> >Val
> >
> >
> >
> >
> >Valerie A. Mallder
> >New Horizons Deputy Mission System Engineer Johns Hopkins
> >University/Applied Physics Laboratory
> >
> >> -----Original Message-----
> >> From: Andrea DeMarco [mailto:dr.andrea.demarco@gmail.com]
> >> Sent: Tuesday, June 09, 2015 9:44 AM
> >> To: Mattmann, Chris A (3980)
> >> Cc: Wyngaard, Jane R (3980-Affiliate); Starch, Michael D (398M);
> >>Kristian ZARB  ADAMI; Ramirez, Paul M (398M); Alessio Magro;
> >>dev@oodt.apache.org
> >> Subject: Re: OODT Documentation
> >>
> >> Hi Chris,
> >>
> >> Yup, the jar file is present in both resmgr/lib and workflow/lib. You
> >>can have a look  at my JAR contents at:
> >>
> >>https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar
> >>?dl
> >>=0
> >>  - perhaps something's off there?
> >>
> >> Cheers,
> >> Andrea

Re: OODT Documentation

Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.

Sigh, sorry Val:

[chipotle:~/tmp/oodt0.9] mattmann% svn commit -m "Finally fully implement
OODT-785 and get the batch stub legacy mode props in there. Thanks to Val."
Sending        
mvn/archetypes/radix/src/main/resources/archetype-resources/resmgr/src/main
/resources/bin/batch_stub
Transmitting file data .
Committed revision 1684425.
[chipotle:~/tmp/oodt0.9] mattmann%


Looking at:
http://svn.apache.org/viewvc?view=revision&revision=1637785


I see that this was only done and committed in the resmgr code for
trunk and not in OODT RADIX. Anyways, above I just committed it in
RADIX. Ugh, sorry for that. Happy to make a point release if needed
for an 0.9.1.

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: <Mallder>, Valerie <Va...@jhuapl.edu>
Date: Tuesday, June 9, 2015 at 6:59 AM
To: "dev@oodt.apache.org" <de...@oodt.apache.org>, jpluser
<ch...@jpl.nasa.gov>
Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
"Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
<pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>
Subject: RE: OODT Documentation

>If you are using the RADiX archetype, make sure the following two lines
>are in the batch_stub script The batch_stub script is located at:
>oodt/resmgr/bin.
>
>-Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \
>-Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \
>
>I just checked here
>
>http://svn.apache.org/viewvc/oodt/tags/0.9-rc1/mvn/archetypes/radix/src/ma
>in/resources/archetype-resources/resmgr/src/main/resources/bin/batch_stub?
>view=markup
>
>and see that it batch_stub in the RADiX archetype is still missing those
>two lines.  JIRA issue OODT-785 is marked fixed in 0.8, but it was never
>been implemented in the RADiX archetype.
>
>Val
>
>
>
>
>Valerie A. Mallder
>New Horizons Deputy Mission System Engineer
>Johns Hopkins University/Applied Physics Laboratory
>
>> -----Original Message-----
>> From: Andrea DeMarco [mailto:dr.andrea.demarco@gmail.com]
>> Sent: Tuesday, June 09, 2015 9:44 AM
>> To: Mattmann, Chris A (3980)
>> Cc: Wyngaard, Jane R (3980-Affiliate); Starch, Michael D (398M);
>>Kristian ZARB
>> ADAMI; Ramirez, Paul M (398M); Alessio Magro; dev@oodt.apache.org
>> Subject: Re: OODT Documentation
>> 
>> Hi Chris,
>> 
>> Yup, the jar file is present in both resmgr/lib and workflow/lib. You
>>can have a look
>> at my JAR contents at:
>> 
>>https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar?dl
>>=0
>>  - perhaps something's off there?
>> 
>> Cheers,
>> Andrea

RE: OODT Documentation

Posted by "Mallder, Valerie" <Va...@jhuapl.edu>.

If you are using the RADiX archetype, make sure the following two lines are in the batch_stub script The batch_stub script is located at: oodt/resmgr/bin.

-Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \
-Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \

I just checked here

http://svn.apache.org/viewvc/oodt/tags/0.9-rc1/mvn/archetypes/radix/src/main/resources/archetype-resources/resmgr/src/main/resources/bin/batch_stub?view=markup

and see that it batch_stub in the RADiX archetype is still missing those two lines.  JIRA issue OODT-785 is marked fixed in 0.8, but it was never been implemented in the RADiX archetype. 

Val




Valerie A. Mallder
New Horizons Deputy Mission System Engineer
Johns Hopkins University/Applied Physics Laboratory

> -----Original Message-----
> From: Andrea DeMarco [mailto:dr.andrea.demarco@gmail.com]
> Sent: Tuesday, June 09, 2015 9:44 AM
> To: Mattmann, Chris A (3980)
> Cc: Wyngaard, Jane R (3980-Affiliate); Starch, Michael D (398M); Kristian ZARB
> ADAMI; Ramirez, Paul M (398M); Alessio Magro; dev@oodt.apache.org
> Subject: Re: OODT Documentation
> 
> Hi Chris,
> 
> Yup, the jar file is present in both resmgr/lib and workflow/lib. You can have a look
> at my JAR contents at:
> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar?dl=0
>  - perhaps something's off there?
> 
> Cheers,
> Andrea

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi Chris,

Yup, the jar file is present in both resmgr/lib and workflow/lib. You can
have a look at my JAR contents at:
https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.jar?dl=0
 - perhaps something's off there?

Cheers,
Andrea

Re: OODT Documentation

Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.

Hi Andrea,

Did you put the file concatenator PGE jar file inside of resmgr/lib?
Try that, it may be the issue :) Please remember to restart resmgr
after you do this.

Thanks!

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Andrea DeMarco <dr...@gmail.com>
Date: Tuesday, June 9, 2015 at 6:07 AM
To: jpluser <ch...@jpl.nasa.gov>
Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
"Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
<pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
"dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: Re: OODT Documentation

>Hi all, 
>
>
>I've been experimenting a little bit with the FileConcatenator sample
>(https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Learn+by+Example
>).
> The workflow seems set up, and the scheduling of the job seems to be in
>place. I even get a nice display of jobs running on the OPSUI. However,
>the actual running of the job fails. I think I have tracked down the
>problem to this log entry (relevant log file
> attached):
>
>SEVERE: Job execution failed for jobId '2015-06-09T14:40:44.896+02:00' :
>batchstub.executeJob returned false
>java.lang.Exception: batchstub.executeJob returned false
>at 
>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatchM
>grProxy.java:125)
>
>
>The batch_stub seems running, and shows "green" on OPSUI as well on port
>2001. Not sure how to move on from here.
>
>
>I have scouted some forums for this error, and found a couple of what
>seem to be relevant posts from Chris :) I'm also attaching my tasks.xml
>file, since I read there might be something in there which is set up
>incorrectly.
>
>
>If I can give you any additional information, please let me know.
>
>
>Thanks,
>Andrea
>
>

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi all,

I've been experimenting a little bit with the FileConcatenator sample (
https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Learn+by+Example).
The workflow seems set up, and the scheduling of the job seems to be in
place. I even get a nice display of jobs running on the OPSUI. However, the
actual running of the job fails. I think I have tracked down the problem to
this log entry (relevant log file attached):

SEVERE: Job execution failed for jobId '2015-06-09T14:40:44.896+02:00' :
batchstub.executeJob returned false
java.lang.Exception: batchstub.executeJob returned false
at
org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatchMgrProxy.java:125)

The batch_stub seems running, and shows "green" on OPSUI as well on port
2001. Not sure how to move on from here.

I have scouted some forums for this error, and found a couple of what seem
to be relevant posts from Chris :) I'm also attaching my tasks.xml file,
since I read there might be something in there which is set up incorrectly.

If I can give you any additional information, please let me know.

Thanks,
Andrea

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi Chris,

Thanks a lot. I have set up 0.9 and all the previously reported problems
are indeed gone. Will start experimenting today :)

Thanks,
Andrea

On 8 June 2015 at 08:06, Mattmann, Chris A (3980) <
chris.a.mattmann@jpl.nasa.gov> wrote:

> Hi Andrea, 0.9 is out :) http://oodt.apache.org/download
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW:  http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
> -----Original Message-----
> From: Andrea DeMarco <dr...@gmail.com>
> Date: Friday, June 5, 2015 at 7:38 AM
> To: jpluser <ch...@jpl.nasa.gov>
> Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
> "Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
> ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
> <pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
> "dev@oodt.apache.org" <de...@oodt.apache.org>
> Subject: Re: OODT Documentation
>
> >Hi Chris,
> >
> >
> >Thanks for that info - no problem, we'd rather move to 0.9 as well.
> >
> >
> >Cheers,
> >Andrea
> >
>
>


-- 
Best Regards,
Andrea DeMarco
Institute of Space Sciences and Astronomy
University of Malta

Re: OODT Documentation

Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.

Hi Andrea, 0.9 is out :) http://oodt.apache.org/download

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Andrea DeMarco <dr...@gmail.com>
Date: Friday, June 5, 2015 at 7:38 AM
To: jpluser <ch...@jpl.nasa.gov>
Cc: "Wyngaard, Jane R (3980-Affiliate)" <Ja...@jpl.nasa.gov>,
"Starch, Michael D (398M)" <Mi...@jpl.nasa.gov>, Kristian ZARB
ADAMI <kz...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)"
<pa...@jpl.nasa.gov>, Alessio Magro <le...@gmail.com>,
"dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: Re: OODT Documentation

>Hi Chris, 
>
>
>Thanks for that info - no problem, we'd rather move to 0.9 as well.
>
>
>Cheers,
>Andrea
>

Re: OODT Documentation

Posted by Andrea DeMarco <dr...@gmail.com>.

Hi Chris,

Thanks for that info - no problem, we'd rather move to 0.9 as well.

Cheers,
Andrea