You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by "Mallder, Valerie" <Va...@jhuapl.edu> on 2014/10/20 22:05:53 UTC
multiple post ingest actions with PGE
Hi All,
I am trying to run two post ingest success actions after PGE runs the crawler and I have come across something that Cameron also came across back in February. It thinks my two action names separated by a comma is one long bean name. And, despite my best efforts I haven't been able to solve this by following the instructions provided by an email thread in the mailing list archives here:
http://markmail.org/search/?q=oodt+ACTION_IDS+order%3Adate-backward#query:oodt%20ACTION_IDS%20order%3Adate-backward+page:2+mid:gxnbptf4zamq2qdp+state:results
or by following Cameron's subsequent documentation on the wiki here:
https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Help+and+Documentation
I am sure I am missing something subtle, but I would appreciate your help. Here are the lines from my tasks.xml and PgeConfig.xml files. And the error I am getting (which is similar to the error Cameron received back in February) is below that.
Thanks in advance!
Val
In tasks.xml
<property name="PostIngestSuccessActions" value="MoveFileToLevel0Dir,MoveMetFileToBackupDir"/>
In PgeConfig.xml
<customMetadata>
<metadata key="PCS_ActionsIds" val="[PostIngestSuccessActions]" />
</customMetadata>
INFO: Updating status to workflow as [CRAWLING]
Oct 20, 2014 3:56:52 PM org.apache.oodt.cas.pge.PGETaskInstance runIngestCrawler
INFO: Crawling for products in [/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/staging]
Oct 20, 2014 3:56:52 PM org.apache.oodt.cas.pge.PGETaskInstance run
SEVERE: PGETask FAILED!!! : No bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:360)
at org.springframework.beans.factory.support.AbstractBeanFactory.getMergedBeanDefinition(AbstractBeanFactory.java:914)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:241)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:761)
at org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanFactory(CrawlerActionRepo.java:71)
at org.apache.oodt.cas.crawl.ProductCrawler.loadAndValidateActions(ProductCrawler.java:228)
at org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:83)
at org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstance.java:575)
at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:144)
at org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.executeTaskLocally(IterativeWorkflowProcessorThread.java:574)
at org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(IterativeWorkflowProcessorThread.java:321)
at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Thread.java:745)
org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceException: PGETask FAILED!!! : No bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
at org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.executeTaskLocally(IterativeWorkflowProcessorThread.java:574)
at org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(IterativeWorkflowProcessorThread.java:321)
at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:360)
at org.springframework.beans.factory.support.AbstractBeanFactory.getMergedBeanDefinition(AbstractBeanFactory.java:914)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:241)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:761)
at org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanFactory(CrawlerActionRepo.java:71)
at org.apache.oodt.cas.crawl.ProductCrawler.loadAndValidateActions(ProductCrawler.java:228)
at org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:83)
at org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstance.java:575)
at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:144)
... 4 more
Oct 20, 2014 3:56:52 PM org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread executeTaskLocally
WARNING: Exception executing task: [feiCrawlerTask] locally: Message: PGETask FAILED!!! : No bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
Valerie A. Mallder
New Horizons Deputy Mission System Engineer
The Johns Hopkins University/Applied Physics Laboratory
11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
240-228-7846 (Office) 410-504-2233 (Blackberry)
Re: multiple post ingest actions with PGE
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
You did great Val, glad you got there! :) You also helped us
to flush out some of the issues here and get them on the list.
The next step will be to write a new improved get up and running
with CAS-PGE guide. If you don¹t beat me to it, I¹ll try and do
that over the next few weeks.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: <Mallder>, Valerie <Va...@jhuapl.edu>
Reply-To: "dev@oodt.apache.org" <de...@oodt.apache.org>
Date: Tuesday, October 21, 2014 at 8:40 AM
To: "dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: RE: multiple post ingest actions with PGE
>Yay Chris! Thanks, adding the "split" attribute and moving the list of
>actions to the pgeConfig (without referencing the list from the workflow
>task properties) did the trick. Thanks so much. The email thread you
>pointed me to is the same email thread I referenced in my previous email.
>The concept of why you would need to put the actions in the pgeConfig is
>discussed there, but the "split" attribute is not mentioned anywhere.
>Hence my confusion. I will be happy to update the wiki page with this
>special little tidbit of information. :)
>
>Ok, so now, I am back to where I was 2-3 weeks ago. I'm crawling for new
>files, ingesting them to the archive and copying some of the files to a
>new directory for further processing. The only difference is back then I
>was using just the crawler and filemgr, and now I'm using CAS-PGE with
>crawler, filemgr, workflow, and resmgr. Whew! It's been quite an
>adventure.
>
>I really appreciate everyone's help! Now, I have to start working on the
>more complicated aspects of this pipeline.
>
>Val
>
>
>
>
>Valerie A. Mallder
>New Horizons Deputy Mission System Engineer
>Johns Hopkins University/Applied Physics Laboratory
>
>> -----Original Message-----
>> From: Mattmann, Chris A (3980) [mailto:chris.a.mattmann@jpl.nasa.gov]
>> Sent: Monday, October 20, 2014 11:38 PM
>> To: dev@oodt.apache.org
>> Subject: Re: multiple post ingest actions with PGE
>>
>> Hey Val,
>>
>> I think the answer is here:
>>
>> 1. First even though it?s documented on the wiki (incorrectly) that you
>>can put the
>> met key split ?,? in the tasks.xml file, you can?t:
>> http://markmail.org/message/gxnbptf4zamq2qdp
>>
>>
>> 2. The parser for workflow task config doesn?t process the ?split?
>> attribute. However the CAS-PGE XML config file reader *does*, so if you
>>simply
>> move PCS_ActionsIds into your PGE config xml file, and you add the
>>split=?true?
>> attribute, you can use the comma delimiter to have multiple actions.
>>
>> <metadata key="PCS_ActionsIds"
>> val="TriggerPostIngestWorkflow,DeleteDataFile" split="true"/>
>>
>>
>> Special kudos if you can update the wiki above to have that correct
>>info! :)
>>
>> Cheers,
>> Chris
>>
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>> Chris Mattmann, Ph.D.
>> Chief Architect
>> Instrument Software and Science Data Systems Section (398) NASA Jet
>> Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 168-519, Mailstop: 168-527
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>> Adjunct Associate Professor, Computer Science Department University of
>> Southern California, Los Angeles, CA 90089 USA
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: <Mallder>, Valerie <Va...@jhuapl.edu>
>> Reply-To: "dev@oodt.apache.org" <de...@oodt.apache.org>
>> Date: Monday, October 20, 2014 at 1:05 PM
>> To: "dev@oodt.apache.org" <de...@oodt.apache.org>
>> Subject: multiple post ingest actions with PGE
>>
>> >Hi All,
>> >I am trying to run two post ingest success actions after PGE runs the
>> >crawler and I have come across something that Cameron also came across
>> >back in February. It thinks my two action names separated by a comma is
>> >one long bean name. And, despite my best efforts I haven't been able to
>> >solve this by following the instructions provided by an email thread in
>> >the mailing list archives here:
>> >
>> >http://markmail.org/search/?q=oodt+ACTION_IDS+order%3Adate-
>> backward#query:
>> >oodt%20ACTION_IDS%20order%3Adate-
>> backward+page:2+mid:gxnbptf4zamq2qdp+s
>> >tat
>> >e:results
>> >
>> >or by following Cameron's subsequent documentation on the wiki here:
>> >
>> >https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Help+and+Docum
>> >ent
>> >ation
>> >
>> >I am sure I am missing something subtle, but I would appreciate your
>> >help. Here are the lines from my tasks.xml and PgeConfig.xml files.
>> >And the error I am getting (which is similar to the error Cameron
>> >received back in February) is below that.
>> >
>> >Thanks in advance!
>> >Val
>> >
>> >In tasks.xml
>> > <property name="PostIngestSuccessActions"
>> >value="MoveFileToLevel0Dir,MoveMetFileToBackupDir"/>
>> >
>> >In PgeConfig.xml
>> > <customMetadata>
>> > <metadata key="PCS_ActionsIds" val="[PostIngestSuccessActions]"
>>/>
>> > </customMetadata>
>> >
>> >
>> >INFO: Updating status to workflow as [CRAWLING] Oct 20, 2014 3:56:52 PM
>> >org.apache.oodt.cas.pge.PGETaskInstance
>> >runIngestCrawler
>> >INFO: Crawling for products in
>> >[/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/stag
>> >ing
>> >]
>> >Oct 20, 2014 3:56:52 PM org.apache.oodt.cas.pge.PGETaskInstance run
>> >SEVERE: PGETask FAILED!!! : No bean named
>> >'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
>> >org.springframework.beans.factory.NoSuchBeanDefinitionException: No
>> >bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
>> > at
>> >org.springframework.beans.factory.support.DefaultListableBeanFactory.ge
>> >tBe
>> >anDefinition(DefaultListableBeanFactory.java:360)
>> > at
>> >org.springframework.beans.factory.support.AbstractBeanFactory.getMerged
>> >Bea
>> >nDefinition(AbstractBeanFactory.java:914)
>> > at
>> >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A
>> >bst
>> >ractBeanFactory.java:241)
>> > at
>> >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A
>> >bst
>> >ractBeanFactory.java:164)
>> > at
>> >org.springframework.context.support.AbstractApplicationContext.getBean(
>> >Abs
>> >tractApplicationContext.java:761)
>> > at
>> >org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanF
>> >act
>> >ory(CrawlerActionRepo.java:71)
>> > at
>> >org.apache.oodt.cas.crawl.ProductCrawler.loadAndValidateActions(Product
>> >Cra
>> >wler.java:228)
>> > at
>> >org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:83)
>> > at
>> >org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstanc
>> >e.j
>> >ava:575)
>> > at
>> >org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:144)
>> > at
>> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ex
>> >ecu
>> >teTaskLocally(IterativeWorkflowProcessorThread.java:574)
>> > at
>> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ru
>> >n(I
>> >terativeWorkflowProcessorThread.java:321)
>> > at
>> >EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown
>>Source)
>> > at java.lang.Thread.run(Thread.java:745)
>> >
>> >org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExc
>> >ept
>> >ion: PGETask FAILED!!! : No bean named
>> >'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
>> > at
>> >org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>> > at
>> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ex
>> >ecu
>> >teTaskLocally(IterativeWorkflowProcessorThread.java:574)
>> > at
>> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ru
>> >n(I
>> >terativeWorkflowProcessorThread.java:321)
>> > at
>> >EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown
>>Source)
>> > at java.lang.Thread.run(Thread.java:745)
>> >Caused by:
>> >org.springframework.beans.factory.NoSuchBeanDefinitionException: No
>> >bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
>> > at
>> >org.springframework.beans.factory.support.DefaultListableBeanFactory.ge
>> >tBe
>> >anDefinition(DefaultListableBeanFactory.java:360)
>> > at
>> >org.springframework.beans.factory.support.AbstractBeanFactory.getMerged
>> >Bea
>> >nDefinition(AbstractBeanFactory.java:914)
>> > at
>> >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A
>> >bst
>> >ractBeanFactory.java:241)
>> > at
>> >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A
>> >bst
>> >ractBeanFactory.java:164)
>> > at
>> >org.springframework.context.support.AbstractApplicationContext.getBean(
>> >Abs
>> >tractApplicationContext.java:761)
>> > at
>> >org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanF
>> >act
>> >ory(CrawlerActionRepo.java:71)
>> > at
>> >org.apache.oodt.cas.crawl.ProductCrawler.loadAndValidateActions(Product
>> >Cra
>> >wler.java:228)
>> > at
>> >org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:83)
>> > at
>> >org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstanc
>> >e.j
>> >ava:575)
>> > at
>> >org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:144)
>> > ... 4 more
>> >Oct 20, 2014 3:56:52 PM
>> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> >executeTaskLocally
>> >WARNING: Exception executing task: [feiCrawlerTask] locally: Message:
>> >PGETask FAILED!!! : No bean named
>> >'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
>> >
>> >Valerie A. Mallder
>> >
>> >New Horizons Deputy Mission System Engineer The Johns Hopkins
>> >University/Applied Physics Laboratory
>> >11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>> >240-228-7846 (Office) 410-504-2233 (Blackberry)
>> >
>
RE: multiple post ingest actions with PGE
Posted by "Mallder, Valerie" <Va...@jhuapl.edu>.
Yay Chris! Thanks, adding the "split" attribute and moving the list of actions to the pgeConfig (without referencing the list from the workflow task properties) did the trick. Thanks so much. The email thread you pointed me to is the same email thread I referenced in my previous email. The concept of why you would need to put the actions in the pgeConfig is discussed there, but the "split" attribute is not mentioned anywhere. Hence my confusion. I will be happy to update the wiki page with this special little tidbit of information. :)
Ok, so now, I am back to where I was 2-3 weeks ago. I'm crawling for new files, ingesting them to the archive and copying some of the files to a new directory for further processing. The only difference is back then I was using just the crawler and filemgr, and now I'm using CAS-PGE with crawler, filemgr, workflow, and resmgr. Whew! It's been quite an adventure.
I really appreciate everyone's help! Now, I have to start working on the more complicated aspects of this pipeline.
Val
Valerie A. Mallder
New Horizons Deputy Mission System Engineer
Johns Hopkins University/Applied Physics Laboratory
> -----Original Message-----
> From: Mattmann, Chris A (3980) [mailto:chris.a.mattmann@jpl.nasa.gov]
> Sent: Monday, October 20, 2014 11:38 PM
> To: dev@oodt.apache.org
> Subject: Re: multiple post ingest actions with PGE
>
> Hey Val,
>
> I think the answer is here:
>
> 1. First even though it?s documented on the wiki (incorrectly) that you can put the
> met key split ?,? in the tasks.xml file, you can?t:
> http://markmail.org/message/gxnbptf4zamq2qdp
>
>
> 2. The parser for workflow task config doesn?t process the ?split?
> attribute. However the CAS-PGE XML config file reader *does*, so if you simply
> move PCS_ActionsIds into your PGE config xml file, and you add the split=?true?
> attribute, you can use the comma delimiter to have multiple actions.
>
> <metadata key="PCS_ActionsIds"
> val="TriggerPostIngestWorkflow,DeleteDataFile" split="true"/>
>
>
> Special kudos if you can update the wiki above to have that correct info! :)
>
> Cheers,
> Chris
>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398) NASA Jet
> Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++
> Adjunct Associate Professor, Computer Science Department University of
> Southern California, Los Angeles, CA 90089 USA
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++
>
>
>
>
>
>
> -----Original Message-----
> From: <Mallder>, Valerie <Va...@jhuapl.edu>
> Reply-To: "dev@oodt.apache.org" <de...@oodt.apache.org>
> Date: Monday, October 20, 2014 at 1:05 PM
> To: "dev@oodt.apache.org" <de...@oodt.apache.org>
> Subject: multiple post ingest actions with PGE
>
> >Hi All,
> >I am trying to run two post ingest success actions after PGE runs the
> >crawler and I have come across something that Cameron also came across
> >back in February. It thinks my two action names separated by a comma is
> >one long bean name. And, despite my best efforts I haven't been able to
> >solve this by following the instructions provided by an email thread in
> >the mailing list archives here:
> >
> >http://markmail.org/search/?q=oodt+ACTION_IDS+order%3Adate-
> backward#query:
> >oodt%20ACTION_IDS%20order%3Adate-
> backward+page:2+mid:gxnbptf4zamq2qdp+s
> >tat
> >e:results
> >
> >or by following Cameron's subsequent documentation on the wiki here:
> >
> >https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Help+and+Docum
> >ent
> >ation
> >
> >I am sure I am missing something subtle, but I would appreciate your
> >help. Here are the lines from my tasks.xml and PgeConfig.xml files.
> >And the error I am getting (which is similar to the error Cameron
> >received back in February) is below that.
> >
> >Thanks in advance!
> >Val
> >
> >In tasks.xml
> > <property name="PostIngestSuccessActions"
> >value="MoveFileToLevel0Dir,MoveMetFileToBackupDir"/>
> >
> >In PgeConfig.xml
> > <customMetadata>
> > <metadata key="PCS_ActionsIds" val="[PostIngestSuccessActions]" />
> > </customMetadata>
> >
> >
> >INFO: Updating status to workflow as [CRAWLING] Oct 20, 2014 3:56:52 PM
> >org.apache.oodt.cas.pge.PGETaskInstance
> >runIngestCrawler
> >INFO: Crawling for products in
> >[/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/stag
> >ing
> >]
> >Oct 20, 2014 3:56:52 PM org.apache.oodt.cas.pge.PGETaskInstance run
> >SEVERE: PGETask FAILED!!! : No bean named
> >'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
> >org.springframework.beans.factory.NoSuchBeanDefinitionException: No
> >bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
> > at
> >org.springframework.beans.factory.support.DefaultListableBeanFactory.ge
> >tBe
> >anDefinition(DefaultListableBeanFactory.java:360)
> > at
> >org.springframework.beans.factory.support.AbstractBeanFactory.getMerged
> >Bea
> >nDefinition(AbstractBeanFactory.java:914)
> > at
> >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A
> >bst
> >ractBeanFactory.java:241)
> > at
> >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A
> >bst
> >ractBeanFactory.java:164)
> > at
> >org.springframework.context.support.AbstractApplicationContext.getBean(
> >Abs
> >tractApplicationContext.java:761)
> > at
> >org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanF
> >act
> >ory(CrawlerActionRepo.java:71)
> > at
> >org.apache.oodt.cas.crawl.ProductCrawler.loadAndValidateActions(Product
> >Cra
> >wler.java:228)
> > at
> >org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:83)
> > at
> >org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstanc
> >e.j
> >ava:575)
> > at
> >org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:144)
> > at
> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ex
> >ecu
> >teTaskLocally(IterativeWorkflowProcessorThread.java:574)
> > at
> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ru
> >n(I
> >terativeWorkflowProcessorThread.java:321)
> > at
> >EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
> > at java.lang.Thread.run(Thread.java:745)
> >
> >org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExc
> >ept
> >ion: PGETask FAILED!!! : No bean named
> >'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
> > at
> >org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> > at
> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ex
> >ecu
> >teTaskLocally(IterativeWorkflowProcessorThread.java:574)
> > at
> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ru
> >n(I
> >terativeWorkflowProcessorThread.java:321)
> > at
> >EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
> > at java.lang.Thread.run(Thread.java:745)
> >Caused by:
> >org.springframework.beans.factory.NoSuchBeanDefinitionException: No
> >bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
> > at
> >org.springframework.beans.factory.support.DefaultListableBeanFactory.ge
> >tBe
> >anDefinition(DefaultListableBeanFactory.java:360)
> > at
> >org.springframework.beans.factory.support.AbstractBeanFactory.getMerged
> >Bea
> >nDefinition(AbstractBeanFactory.java:914)
> > at
> >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A
> >bst
> >ractBeanFactory.java:241)
> > at
> >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A
> >bst
> >ractBeanFactory.java:164)
> > at
> >org.springframework.context.support.AbstractApplicationContext.getBean(
> >Abs
> >tractApplicationContext.java:761)
> > at
> >org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanF
> >act
> >ory(CrawlerActionRepo.java:71)
> > at
> >org.apache.oodt.cas.crawl.ProductCrawler.loadAndValidateActions(Product
> >Cra
> >wler.java:228)
> > at
> >org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:83)
> > at
> >org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstanc
> >e.j
> >ava:575)
> > at
> >org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:144)
> > ... 4 more
> >Oct 20, 2014 3:56:52 PM
> >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >executeTaskLocally
> >WARNING: Exception executing task: [feiCrawlerTask] locally: Message:
> >PGETask FAILED!!! : No bean named
> >'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
> >
> >Valerie A. Mallder
> >
> >New Horizons Deputy Mission System Engineer The Johns Hopkins
> >University/Applied Physics Laboratory
> >11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
> >240-228-7846 (Office) 410-504-2233 (Blackberry)
> >
Re: multiple post ingest actions with PGE
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
Hey Val,
I think the answer is here:
1. First even though it¹s documented on the wiki (incorrectly) that
you can put the met key split ³,² in the tasks.xml file, you can¹t:
http://markmail.org/message/gxnbptf4zamq2qdp
2. The parser for workflow task config doesn¹t process the ³split²
attribute. However the CAS-PGE XML config file reader *does*, so if you
simply move PCS_ActionsIds into your PGE config xml file, and you add the
split=³true²
attribute, you can use the comma delimiter to have multiple actions.
<metadata key="PCS_ActionsIds"
val="TriggerPostIngestWorkflow,DeleteDataFile" split="true"/>
Special kudos if you can update the wiki above to have that correct
info! :)
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: <Mallder>, Valerie <Va...@jhuapl.edu>
Reply-To: "dev@oodt.apache.org" <de...@oodt.apache.org>
Date: Monday, October 20, 2014 at 1:05 PM
To: "dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: multiple post ingest actions with PGE
>Hi All,
>I am trying to run two post ingest success actions after PGE runs the
>crawler and I have come across something that Cameron also came across
>back in February. It thinks my two action names separated by a comma is
>one long bean name. And, despite my best efforts I haven't been able to
>solve this by following the instructions provided by an email thread in
>the mailing list archives here:
>
>http://markmail.org/search/?q=oodt+ACTION_IDS+order%3Adate-backward#query:
>oodt%20ACTION_IDS%20order%3Adate-backward+page:2+mid:gxnbptf4zamq2qdp+stat
>e:results
>
>or by following Cameron's subsequent documentation on the wiki here:
>
>https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Help+and+Document
>ation
>
>I am sure I am missing something subtle, but I would appreciate your
>help. Here are the lines from my tasks.xml and PgeConfig.xml files. And
>the error I am getting (which is similar to the error Cameron received
>back in February) is below that.
>
>Thanks in advance!
>Val
>
>In tasks.xml
> <property name="PostIngestSuccessActions"
>value="MoveFileToLevel0Dir,MoveMetFileToBackupDir"/>
>
>In PgeConfig.xml
> <customMetadata>
> <metadata key="PCS_ActionsIds" val="[PostIngestSuccessActions]" />
> </customMetadata>
>
>
>INFO: Updating status to workflow as [CRAWLING]
>Oct 20, 2014 3:56:52 PM org.apache.oodt.cas.pge.PGETaskInstance
>runIngestCrawler
>INFO: Crawling for products in
>[/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/staging
>]
>Oct 20, 2014 3:56:52 PM org.apache.oodt.cas.pge.PGETaskInstance run
>SEVERE: PGETask FAILED!!! : No bean named
>'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
>org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean
>named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
> at
>org.springframework.beans.factory.support.DefaultListableBeanFactory.getBe
>anDefinition(DefaultListableBeanFactory.java:360)
> at
>org.springframework.beans.factory.support.AbstractBeanFactory.getMergedBea
>nDefinition(AbstractBeanFactory.java:914)
> at
>org.springframework.beans.factory.support.AbstractBeanFactory.getBean(Abst
>ractBeanFactory.java:241)
> at
>org.springframework.beans.factory.support.AbstractBeanFactory.getBean(Abst
>ractBeanFactory.java:164)
> at
>org.springframework.context.support.AbstractApplicationContext.getBean(Abs
>tractApplicationContext.java:761)
> at
>org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanFact
>ory(CrawlerActionRepo.java:71)
> at
>org.apache.oodt.cas.crawl.ProductCrawler.loadAndValidateActions(ProductCra
>wler.java:228)
> at
>org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:83)
> at
>org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstance.j
>ava:575)
> at
>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:144)
> at
>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.execu
>teTaskLocally(IterativeWorkflowProcessorThread.java:574)
> at
>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(I
>terativeWorkflowProcessorThread.java:321)
> at
>EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:745)
>
>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcept
>ion: PGETask FAILED!!! : No bean named
>'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
> at
>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> at
>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.execu
>teTaskLocally(IterativeWorkflowProcessorThread.java:574)
> at
>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(I
>terativeWorkflowProcessorThread.java:321)
> at
>EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Thread.java:745)
>Caused by:
>org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean
>named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
> at
>org.springframework.beans.factory.support.DefaultListableBeanFactory.getBe
>anDefinition(DefaultListableBeanFactory.java:360)
> at
>org.springframework.beans.factory.support.AbstractBeanFactory.getMergedBea
>nDefinition(AbstractBeanFactory.java:914)
> at
>org.springframework.beans.factory.support.AbstractBeanFactory.getBean(Abst
>ractBeanFactory.java:241)
> at
>org.springframework.beans.factory.support.AbstractBeanFactory.getBean(Abst
>ractBeanFactory.java:164)
> at
>org.springframework.context.support.AbstractApplicationContext.getBean(Abs
>tractApplicationContext.java:761)
> at
>org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanFact
>ory(CrawlerActionRepo.java:71)
> at
>org.apache.oodt.cas.crawl.ProductCrawler.loadAndValidateActions(ProductCra
>wler.java:228)
> at
>org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:83)
> at
>org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstance.j
>ava:575)
> at
>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:144)
> ... 4 more
>Oct 20, 2014 3:56:52 PM
>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>executeTaskLocally
>WARNING: Exception executing task: [feiCrawlerTask] locally: Message:
>PGETask FAILED!!! : No bean named
>'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined
>
>Valerie A. Mallder
>
>New Horizons Deputy Mission System Engineer
>The Johns Hopkins University/Applied Physics Laboratory
>11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>240-228-7846 (Office) 410-504-2233 (Blackberry)
>