You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oodt.apache.org by Thomas Bennett <lm...@gmail.com> on 2012/03/13 15:32:47 UTC
cas-crawler wierdness
Hey,
I've just installed a fresh version of the cas-crawler which uses the new
cli interface and I've got three things that I'm trying to fix:
1) Repeated warning messages:
13 Mar 2012 3:50:09 PM org.springframework.beans.TypeConverterDelegate
findDefaultEditor
WARNING: PropertyEditor [sun.beans.editors.EnumEditor] found through
deprecated global PropertyEditorManager fallback - consider using a more
isolated form of registration, e.g. on the BeanWrapper/BeanFactory!
Any ideas what this is all about?
2) Configuration files.
I use absolute paths to my configuration files which fails for spring
configuration:
...
Caused by: java.io.FileNotFoundException:
usr/local/oodt/cas-crawler/policy/cmd-line-actions.xml (No such file or
directory)
The code that's doing the work is:
public SpringCmdLineActionStoreFactory() {
config = System.getProperty(
"org.apache.oodt.cas.cli.action.spring.config", null);
}
If I put a double slash in front of the spring configuration items it works:
-Dorg.apache.oodt.cas.cli.action.spring.config=//usr/local/oodt/cas-crawler/policy/cmd-line-actions.xml
\
-Dorg.apache.oodt.cas.cli.option.spring.config=//usr/local/oodt/cas-crawler/policy/cmd-line-options.xml
\
So it's ignoring the first char?
3) I see that policy/crawler-config.xml contains three resources, but how
does it know where to find the actual files. Do these need to be pre-pended
by actual directory locations?
<import resource="crawler-beans.xml" /> /*should the location be
included on the command line?*/
<import resource="action-beans.xml" />
<import resource="precondition-beans.xml" />
I'm Using Ubuntu 10.04 LTS with open jdk...
$ java -version
java version "1.6.0_20"
OpenJDK Runtime Environment (IcedTea6 1.9.13) (6b20-1.9.13-0ubuntu1~10.04.1)
OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
Any help/pointers welcome :)
Regards,
Tom
Re: cas-crawler wierdness
Posted by Thomas Bennett <lm...@gmail.com>.
Sorry I actually ran:
$ java
-Djava.util.logging.config.file=/var/kat/katconfig/static/oodt/etc/logging.properties
-Djava.ext.dirs=/usr/local/oodt/cas-crawler/lib
-Dorg.apache.oodt.cas.crawl.bean.repo=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/crawler-config.xml
-Dorg.apache.oodt.cas.cli.action.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-actions.xml
-Dorg.apache.oodt.cas.cli.option.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-options.xml
org.apache.oodt.cas.crawl.CrawlerLauncher --operation --launchAutoCrawler
--productPath /var/kat/data --filemgrUrl
http://127.0.0.1:9101--mimeExtractorRepo
/var/kat/katconfig/static/oodt/cas-crawler/policy/mime-extractor-map.xml
--failureDir /var/kat/failed --actionIds MoveDataFileToFailureDir
>
> The output looks like:
> Setting property 'AutoDetectProductCrawler.mimeExtractorRepo'
> Setting property 'StdProductCrawler.productPath'
> Setting property 'MetExtractorProductCrawler.productPath'
> Setting property 'AutoDetectProductCrawler.productPath'
> Setting property 'MoveMetadataToFailureDir.toDir'
> Setting property 'MovePushpullMetFileToFailureDir.toDir'
> Setting property 'MoveDataFileToFailureDir.toDir'
> Setting property 'StdProductCrawler.filemgrUrl'
> Setting property 'MetExtractorProductCrawler.filemgrUrl'
> Setting property 'AutoDetectProductCrawler.filemgrUrl'
> ERROR: Could not process key 'MoveMetadataToFailureDir.toDir' in
> PropertyOverrideConfigurer; nested exception is
> org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean
> named 'MoveMetadataToFailureDir' is defined
>
> I'm a bit uncertain how to properly define failed to ingest actions with
> the new interface.
>
> I see when calling help:
>
> -fd, --failureDir <directory> Directory
> where files will be moved on failure
> Requirement
> Rules:
>
> [launchStdCrawler : OPTIONAL, launchMetCrawler :
> OPTIONAL,
> launchAutoCrawler : OPTIONAL]
>
> Handler:
> Set the
> following bean properties:
>
> [MoveMetadataToFailureDir.toDir,
>
> MovePushpullMetFileToFailureDir.toDir,
>
> MoveDataFileToFailureDir.toDir]
>
> How do I set the bean properties from the command line :)
>
> I see there are two definitions - MoveMetadataFileToFailureDir and
> MoveMetadataToFailureDir, should these be the same?
>
> Cheers,
> Tom
>
Re: cas-crawler wierdness
Posted by Thomas Bennett <lm...@gmail.com>.
Sorry I actually ran:
$ java
-Djava.util.logging.config.file=/var/kat/katconfig/static/oodt/etc/logging.properties
-Djava.ext.dirs=/usr/local/oodt/cas-crawler/lib
-Dorg.apache.oodt.cas.crawl.bean.repo=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/crawler-config.xml
-Dorg.apache.oodt.cas.cli.action.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-actions.xml
-Dorg.apache.oodt.cas.cli.option.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-options.xml
org.apache.oodt.cas.crawl.CrawlerLauncher --operation --launchAutoCrawler
--productPath /var/kat/data --filemgrUrl
http://127.0.0.1:9101--mimeExtractorRepo
/var/kat/katconfig/static/oodt/cas-crawler/policy/mime-extractor-map.xml
--failureDir /var/kat/failed --actionIds MoveDataFileToFailureDir
>
> The output looks like:
> Setting property 'AutoDetectProductCrawler.mimeExtractorRepo'
> Setting property 'StdProductCrawler.productPath'
> Setting property 'MetExtractorProductCrawler.productPath'
> Setting property 'AutoDetectProductCrawler.productPath'
> Setting property 'MoveMetadataToFailureDir.toDir'
> Setting property 'MovePushpullMetFileToFailureDir.toDir'
> Setting property 'MoveDataFileToFailureDir.toDir'
> Setting property 'StdProductCrawler.filemgrUrl'
> Setting property 'MetExtractorProductCrawler.filemgrUrl'
> Setting property 'AutoDetectProductCrawler.filemgrUrl'
> ERROR: Could not process key 'MoveMetadataToFailureDir.toDir' in
> PropertyOverrideConfigurer; nested exception is
> org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean
> named 'MoveMetadataToFailureDir' is defined
>
> I'm a bit uncertain how to properly define failed to ingest actions with
> the new interface.
>
> I see when calling help:
>
> -fd, --failureDir <directory> Directory
> where files will be moved on failure
> Requirement
> Rules:
>
> [launchStdCrawler : OPTIONAL, launchMetCrawler :
> OPTIONAL,
> launchAutoCrawler : OPTIONAL]
>
> Handler:
> Set the
> following bean properties:
>
> [MoveMetadataToFailureDir.toDir,
>
> MovePushpullMetFileToFailureDir.toDir,
>
> MoveDataFileToFailureDir.toDir]
>
> How do I set the bean properties from the command line :)
>
> I see there are two definitions - MoveMetadataFileToFailureDir and
> MoveMetadataToFailureDir, should these be the same?
>
> Cheers,
> Tom
>
Re: cas-crawler wierdness
Posted by Thomas Bennett <lm...@gmail.com>.
Hi Brian + Chris,
Thanks, for you're help! All sorted.
I attempted to try out Review Board, I was unable to load a diff, it kept
on rejecting my diff files.
I've attached the diff (to update logging from WARNING to SEVERE) to the
JIRA issue.
For #2, what chris said about the file://** makes sense... try that and let
> me know how that works ou
>
> Yup that did the trick, although I needed three slashes file:/// :)
I also had to modify the constructor-arg values in cmd-line-options.xml.
<constructor-arg
value="file:///var/kat/katconfig/static/oodt/cas-crawler/policy/precondition-beans.xml"
/>
All working fine now except when I execute:
$ java
-Djava.util.logging.config.file=/var/kat/katconfig/static/oodt/etc/logging.properties
-Djava.ext.dirs=/usr/local/oodt/cas-crawler/lib
-Dorg.apache.oodt.cas.crawl.bean.repo=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/crawler-config.xml
-Dorg.apache.oodt.cas.cli.action.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-actions.xml
-Dorg.apache.oodt.cas.cli.option.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-options.xml
org.apache.oodt.cas.crawl.CrawlerLauncher --operation --launchAutoCrawler
--productPath /var/kat/data --filemgrUrl
http://127.0.0.1:9101--mimeExtractorRepo
/var/kat/katconfig/static/oodt/cas-crawler/policy/mime-extractor-map.xml
--failureDir /var/kat/failed
The output looks like:
Setting property 'AutoDetectProductCrawler.mimeExtractorRepo'
Setting property 'StdProductCrawler.productPath'
Setting property 'MetExtractorProductCrawler.productPath'
Setting property 'AutoDetectProductCrawler.productPath'
Setting property 'MoveMetadataToFailureDir.toDir'
Setting property 'MovePushpullMetFileToFailureDir.toDir'
Setting property 'MoveDataFileToFailureDir.toDir'
Setting property 'StdProductCrawler.filemgrUrl'
Setting property 'MetExtractorProductCrawler.filemgrUrl'
Setting property 'AutoDetectProductCrawler.filemgrUrl'
ERROR: Could not process key 'MoveMetadataToFailureDir.toDir' in
PropertyOverrideConfigurer; nested exception is
org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean
named 'MoveMetadataToFailureDir' is defined
I'm a bit uncertain how to properly define failed to ingest actions with
the new interface.
I see when calling help:
-fd, --failureDir <directory> Directory where
files will be moved on failure
Requirement
Rules:
[launchStdCrawler : OPTIONAL, launchMetCrawler :
OPTIONAL,
launchAutoCrawler : OPTIONAL]
Handler:
Set the
following bean properties:
[MoveMetadataToFailureDir.toDir,
MovePushpullMetFileToFailureDir.toDir,
MoveDataFileToFailureDir.toDir]
How do I set the bean properties from the command line :)
I see there are two definitions - MoveMetadataFileToFailureDir and
MoveMetadataToFailureDir, should these be the same?
Cheers,
Tom
Re: cas-crawler wierdness
Posted by Thomas Bennett <lm...@gmail.com>.
Hi Brian + Chris,
Thanks, for you're help! All sorted.
I attempted to try out Review Board, I was unable to load a diff, it kept
on rejecting my diff files.
I've attached the diff (to update logging from WARNING to SEVERE) to the
JIRA issue.
For #2, what chris said about the file://** makes sense... try that and let
> me know how that works ou
>
> Yup that did the trick, although I needed three slashes file:/// :)
I also had to modify the constructor-arg values in cmd-line-options.xml.
<constructor-arg
value="file:///var/kat/katconfig/static/oodt/cas-crawler/policy/precondition-beans.xml"
/>
All working fine now except when I execute:
$ java
-Djava.util.logging.config.file=/var/kat/katconfig/static/oodt/etc/logging.properties
-Djava.ext.dirs=/usr/local/oodt/cas-crawler/lib
-Dorg.apache.oodt.cas.crawl.bean.repo=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/crawler-config.xml
-Dorg.apache.oodt.cas.cli.action.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-actions.xml
-Dorg.apache.oodt.cas.cli.option.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-options.xml
org.apache.oodt.cas.crawl.CrawlerLauncher --operation --launchAutoCrawler
--productPath /var/kat/data --filemgrUrl
http://127.0.0.1:9101--mimeExtractorRepo
/var/kat/katconfig/static/oodt/cas-crawler/policy/mime-extractor-map.xml
--failureDir /var/kat/failed
The output looks like:
Setting property 'AutoDetectProductCrawler.mimeExtractorRepo'
Setting property 'StdProductCrawler.productPath'
Setting property 'MetExtractorProductCrawler.productPath'
Setting property 'AutoDetectProductCrawler.productPath'
Setting property 'MoveMetadataToFailureDir.toDir'
Setting property 'MovePushpullMetFileToFailureDir.toDir'
Setting property 'MoveDataFileToFailureDir.toDir'
Setting property 'StdProductCrawler.filemgrUrl'
Setting property 'MetExtractorProductCrawler.filemgrUrl'
Setting property 'AutoDetectProductCrawler.filemgrUrl'
ERROR: Could not process key 'MoveMetadataToFailureDir.toDir' in
PropertyOverrideConfigurer; nested exception is
org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean
named 'MoveMetadataToFailureDir' is defined
I'm a bit uncertain how to properly define failed to ingest actions with
the new interface.
I see when calling help:
-fd, --failureDir <directory> Directory where
files will be moved on failure
Requirement
Rules:
[launchStdCrawler : OPTIONAL, launchMetCrawler :
OPTIONAL,
launchAutoCrawler : OPTIONAL]
Handler:
Set the
following bean properties:
[MoveMetadataToFailureDir.toDir,
MovePushpullMetFileToFailureDir.toDir,
MoveDataFileToFailureDir.toDir]
How do I set the bean properties from the command line :)
I see there are two definitions - MoveMetadataFileToFailureDir and
MoveMetadataToFailureDir, should these be the same?
Cheers,
Tom
Re: cas-crawler wierdness
Posted by Brian Foster <ho...@mac.com>.
Hey Tom,
Looked into #1 more, let's drop the spring logging levels from WARNING to SEVERE in the logging.properties file:
org.springframework.beans.level = SEVERE
org.springframework.core.level = SEVERE
org.springframework.level = SEVERE
org.springframework.beans.factory.level = SEVERE
org.springframework.beans.factory.config.level = SEVERE
org.springframework.beans.factory.config.PropertyPlaceholderConfigurer.level = SEVERE
org.apache.oodt.cas.crawl.util.CasPropertyPlaceholderConfigurer.level = SEVERE
sun.net.level = SEVERE
I'll create an issue to update all the logging.properties files for all the components which use cas-cli
For #2, what chris said about the file://** makes sense... try that and let me know how that works out
-brian
On Mar 13, 2012, at 02:52 PM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:
Hey Tom,
On Mar 13, 2012, at 7:32 AM, Thomas Bennett wrote:
> Hey,
>
> I've just installed a fresh version of the cas-crawler which uses the new cli interface and I've got three things that I'm trying to fix:
>
> 1) Repeated warning messages:
>
> 13 Mar 2012 3:50:09 PM org.springframework.beans.TypeConverterDelegate findDefaultEditor
> WARNING: PropertyEditor [sun.beans.editors.EnumEditor] found through deprecated global PropertyEditorManager fallback - consider using a more isolated form of registration, e.g. on the BeanWrapper/BeanFactory!
>
> Any ideas what this is all about?
This sounds like CAS-CLI introduced logging issues. We can suppress these via CrawlerLauncher (or via some other Java program that you
are calling) by adding:
org.springframework.beans.level = OFF
To the logging.properties file or setting via Java API's LoggingManager.
Can you open a JIRA for this for 0.4?
>
> 2) Configuration files.
>
> I use absolute paths to my configuration files which fails for spring configuration:
>
> ...
> Caused by: java.io.FileNotFoundException: usr/local/oodt/cas-crawler/policy/cmd-line-actions.xml (No such file or directory)
>
> The code that's doing the work is:
> public SpringCmdLineActionStoreFactory() {
>
> config = System.getProperty(
>
> "org.apache.oodt.cas.cli.action.spring.config", null);
>
> }
>
> If I put a double slash in front of the spring configuration items it works:
>
> -Dorg.apache.oodt.cas.cli.action.spring.config=//usr/local/oodt/cas-crawler/policy/cmd-line-actions.xml \
> -Dorg.apache.oodt.cas.cli.option.spring.config=//usr/local/oodt/cas-crawler/policy/cmd-line-options.xml \
>
> So it's ignoring the first char?
I think with Spring, you have to specify:
file:relative/path/to/file
See here:
http://stackoverflow.com/questions/1601106/relative-paths-in-spring-classpath-resource
http://stackoverflow.com/questions/3611250/reference-spring-properties-file-using-path-relative-to-config-file
Are you saying that this used to work (prior to CAS-CLI) and now you are seeing a regression? If so, can
you open up a JIRA issue and we can take a look?
>
> 3) I see that policy/crawler-config.xml contains three resources, but how does it know where to find the actual files. Do these need to be pre-pended by actual directory locations?
>
> <import resource="crawler-beans.xml" /> /*should the location be included on the command line?*/
> <import resource="action-beans.xml" />
> <import resource="precondition-beans.xml" />
>
> I'm Using Ubuntu 10.04 LTS with open jdk...
>
> $ java -version
> java version "1.6.0_20"
> OpenJDK Runtime Environment (IcedTea6 1.9.13) (6b20-1.9.13-0ubuntu1~10.04.1)
> OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
>
> Any help/pointers welcome :)
Yeah I think that's dictated by the classpath level resources (which are baked into the crawler JAR file).
Does that make sense?
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: cas-crawler wierdness
Posted by Brian Foster <ho...@mac.com>.
Hey Tom,
Looked into #1 more, let's drop the spring logging levels from WARNING to SEVERE in the logging.properties file:
org.springframework.beans.level = SEVERE
org.springframework.core.level = SEVERE
org.springframework.level = SEVERE
org.springframework.beans.factory.level = SEVERE
org.springframework.beans.factory.config.level = SEVERE
org.springframework.beans.factory.config.PropertyPlaceholderConfigurer.level = SEVERE
org.apache.oodt.cas.crawl.util.CasPropertyPlaceholderConfigurer.level = SEVERE
sun.net.level = SEVERE
I'll create an issue to update all the logging.properties files for all the components which use cas-cli
For #2, what chris said about the file://** makes sense... try that and let me know how that works out
-brian
On Mar 13, 2012, at 02:52 PM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:
Hey Tom,
On Mar 13, 2012, at 7:32 AM, Thomas Bennett wrote:
> Hey,
>
> I've just installed a fresh version of the cas-crawler which uses the new cli interface and I've got three things that I'm trying to fix:
>
> 1) Repeated warning messages:
>
> 13 Mar 2012 3:50:09 PM org.springframework.beans.TypeConverterDelegate findDefaultEditor
> WARNING: PropertyEditor [sun.beans.editors.EnumEditor] found through deprecated global PropertyEditorManager fallback - consider using a more isolated form of registration, e.g. on the BeanWrapper/BeanFactory!
>
> Any ideas what this is all about?
This sounds like CAS-CLI introduced logging issues. We can suppress these via CrawlerLauncher (or via some other Java program that you
are calling) by adding:
org.springframework.beans.level = OFF
To the logging.properties file or setting via Java API's LoggingManager.
Can you open a JIRA for this for 0.4?
>
> 2) Configuration files.
>
> I use absolute paths to my configuration files which fails for spring configuration:
>
> ...
> Caused by: java.io.FileNotFoundException: usr/local/oodt/cas-crawler/policy/cmd-line-actions.xml (No such file or directory)
>
> The code that's doing the work is:
> public SpringCmdLineActionStoreFactory() {
>
> config = System.getProperty(
>
> "org.apache.oodt.cas.cli.action.spring.config", null);
>
> }
>
> If I put a double slash in front of the spring configuration items it works:
>
> -Dorg.apache.oodt.cas.cli.action.spring.config=//usr/local/oodt/cas-crawler/policy/cmd-line-actions.xml \
> -Dorg.apache.oodt.cas.cli.option.spring.config=//usr/local/oodt/cas-crawler/policy/cmd-line-options.xml \
>
> So it's ignoring the first char?
I think with Spring, you have to specify:
file:relative/path/to/file
See here:
http://stackoverflow.com/questions/1601106/relative-paths-in-spring-classpath-resource
http://stackoverflow.com/questions/3611250/reference-spring-properties-file-using-path-relative-to-config-file
Are you saying that this used to work (prior to CAS-CLI) and now you are seeing a regression? If so, can
you open up a JIRA issue and we can take a look?
>
> 3) I see that policy/crawler-config.xml contains three resources, but how does it know where to find the actual files. Do these need to be pre-pended by actual directory locations?
>
> <import resource="crawler-beans.xml" /> /*should the location be included on the command line?*/
> <import resource="action-beans.xml" />
> <import resource="precondition-beans.xml" />
>
> I'm Using Ubuntu 10.04 LTS with open jdk...
>
> $ java -version
> java version "1.6.0_20"
> OpenJDK Runtime Environment (IcedTea6 1.9.13) (6b20-1.9.13-0ubuntu1~10.04.1)
> OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
>
> Any help/pointers welcome :)
Yeah I think that's dictated by the classpath level resources (which are baked into the crawler JAR file).
Does that make sense?
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: cas-crawler wierdness
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Tom,
On Mar 13, 2012, at 7:32 AM, Thomas Bennett wrote:
> Hey,
>
> I've just installed a fresh version of the cas-crawler which uses the new cli interface and I've got three things that I'm trying to fix:
>
> 1) Repeated warning messages:
>
> 13 Mar 2012 3:50:09 PM org.springframework.beans.TypeConverterDelegate findDefaultEditor
> WARNING: PropertyEditor [sun.beans.editors.EnumEditor] found through deprecated global PropertyEditorManager fallback - consider using a more isolated form of registration, e.g. on the BeanWrapper/BeanFactory!
>
> Any ideas what this is all about?
This sounds like CAS-CLI introduced logging issues. We can suppress these via CrawlerLauncher (or via some other Java program that you
are calling) by adding:
org.springframework.beans.level = OFF
To the logging.properties file or setting via Java API's LoggingManager.
Can you open a JIRA for this for 0.4?
>
> 2) Configuration files.
>
> I use absolute paths to my configuration files which fails for spring configuration:
>
> ...
> Caused by: java.io.FileNotFoundException: usr/local/oodt/cas-crawler/policy/cmd-line-actions.xml (No such file or directory)
>
> The code that's doing the work is:
> public SpringCmdLineActionStoreFactory() {
>
> config = System.getProperty(
>
> "org.apache.oodt.cas.cli.action.spring.config", null);
>
> }
>
> If I put a double slash in front of the spring configuration items it works:
>
> -Dorg.apache.oodt.cas.cli.action.spring.config=//usr/local/oodt/cas-crawler/policy/cmd-line-actions.xml \
> -Dorg.apache.oodt.cas.cli.option.spring.config=//usr/local/oodt/cas-crawler/policy/cmd-line-options.xml \
>
> So it's ignoring the first char?
I think with Spring, you have to specify:
file:relative/path/to/file
See here:
http://stackoverflow.com/questions/1601106/relative-paths-in-spring-classpath-resource
http://stackoverflow.com/questions/3611250/reference-spring-properties-file-using-path-relative-to-config-file
Are you saying that this used to work (prior to CAS-CLI) and now you are seeing a regression? If so, can
you open up a JIRA issue and we can take a look?
>
> 3) I see that policy/crawler-config.xml contains three resources, but how does it know where to find the actual files. Do these need to be pre-pended by actual directory locations?
>
> <import resource="crawler-beans.xml" /> /*should the location be included on the command line?*/
> <import resource="action-beans.xml" />
> <import resource="precondition-beans.xml" />
>
> I'm Using Ubuntu 10.04 LTS with open jdk...
>
> $ java -version
> java version "1.6.0_20"
> OpenJDK Runtime Environment (IcedTea6 1.9.13) (6b20-1.9.13-0ubuntu1~10.04.1)
> OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
>
> Any help/pointers welcome :)
Yeah I think that's dictated by the classpath level resources (which are baked into the crawler JAR file).
Does that make sense?
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: cas-crawler wierdness
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Tom,
On Mar 13, 2012, at 7:32 AM, Thomas Bennett wrote:
> Hey,
>
> I've just installed a fresh version of the cas-crawler which uses the new cli interface and I've got three things that I'm trying to fix:
>
> 1) Repeated warning messages:
>
> 13 Mar 2012 3:50:09 PM org.springframework.beans.TypeConverterDelegate findDefaultEditor
> WARNING: PropertyEditor [sun.beans.editors.EnumEditor] found through deprecated global PropertyEditorManager fallback - consider using a more isolated form of registration, e.g. on the BeanWrapper/BeanFactory!
>
> Any ideas what this is all about?
This sounds like CAS-CLI introduced logging issues. We can suppress these via CrawlerLauncher (or via some other Java program that you
are calling) by adding:
org.springframework.beans.level = OFF
To the logging.properties file or setting via Java API's LoggingManager.
Can you open a JIRA for this for 0.4?
>
> 2) Configuration files.
>
> I use absolute paths to my configuration files which fails for spring configuration:
>
> ...
> Caused by: java.io.FileNotFoundException: usr/local/oodt/cas-crawler/policy/cmd-line-actions.xml (No such file or directory)
>
> The code that's doing the work is:
> public SpringCmdLineActionStoreFactory() {
>
> config = System.getProperty(
>
> "org.apache.oodt.cas.cli.action.spring.config", null);
>
> }
>
> If I put a double slash in front of the spring configuration items it works:
>
> -Dorg.apache.oodt.cas.cli.action.spring.config=//usr/local/oodt/cas-crawler/policy/cmd-line-actions.xml \
> -Dorg.apache.oodt.cas.cli.option.spring.config=//usr/local/oodt/cas-crawler/policy/cmd-line-options.xml \
>
> So it's ignoring the first char?
I think with Spring, you have to specify:
file:relative/path/to/file
See here:
http://stackoverflow.com/questions/1601106/relative-paths-in-spring-classpath-resource
http://stackoverflow.com/questions/3611250/reference-spring-properties-file-using-path-relative-to-config-file
Are you saying that this used to work (prior to CAS-CLI) and now you are seeing a regression? If so, can
you open up a JIRA issue and we can take a look?
>
> 3) I see that policy/crawler-config.xml contains three resources, but how does it know where to find the actual files. Do these need to be pre-pended by actual directory locations?
>
> <import resource="crawler-beans.xml" /> /*should the location be included on the command line?*/
> <import resource="action-beans.xml" />
> <import resource="precondition-beans.xml" />
>
> I'm Using Ubuntu 10.04 LTS with open jdk...
>
> $ java -version
> java version "1.6.0_20"
> OpenJDK Runtime Environment (IcedTea6 1.9.13) (6b20-1.9.13-0ubuntu1~10.04.1)
> OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
>
> Any help/pointers welcome :)
Yeah I think that's dictated by the classpath level resources (which are baked into the crawler JAR file).
Does that make sense?
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++