You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by "Hardman, Sean H (388J)" <se...@jpl.nasa.gov> on 2012/02/21 03:57:56 UTC
Issue with File Manager initiating a Lucene Catalog
Sorry for spamming the list today but I have been doing some comparative tests today with the File Manager. When populating a newly installed version (0.4-SNAPSHOT) of the File Manager, I was unable to ingest a product until I supplied an existing (yet empty of products) Lucene Catalog for the File Manager to populate. I have included the stack trace from both the client and the server below. This isn't the behavior seen in the 0.3 release. If the Lucene Catalog does not exist in 0.3, it is created upon the first file ingestion.
Once again, I am not sure whether this is worthy of a JIRA issue. I am just pointing out differing behavior.
Thanks,
Sean
Stack trace from filemgr-client:
bash-3.2$ ./filemgr-client --url http://localhost:9000 --operation --ingestProduct --productName Test --productStructure Flat --productTypeName CatalogFile --metadataFile "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt.met" --refs "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt"
Feb 20, 2012 6:49:30 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties]
org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1162)
at org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:102)
at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:296)
at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:179)
at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1307)
Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
SEVERE: Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) -- rolling back ingest
Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
SEVERE: Failed to rollback ingest of product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
ERROR: Failed to ingest product 'Test' : Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
Stack trace from filemgr:
Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [CatalogFile]
Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [CatalogFile]
Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor] for product type: [CatalogFile]
Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog addCompleteProductToIndex
WARNING: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
java.io.FileNotFoundException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
at org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:430)
at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:439)
at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:329)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:45)
at org.apache.lucene.index.IndexWriter$1.doBody(IndexWriter.java:264)
at org.apache.lucene.store.Lock$With.run(Lock.java:99)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:259)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:204)
at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:990)
at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:1005)
at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog removeProductDocument
WARNING: Exception removing product: [Test] from index: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager removeProduct
WARNING: Exception modifying product: [null]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProductDocument(LuceneCatalog.java:962)
at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProduct(LuceneCatalog.java:298)
at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.removeProduct(XmlRpcFileManager.java:926)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
Re: Issue with File Manager initiating a Lucene Catalog
Posted by "Cinquini, Luca (3880)" <Lu...@jpl.nasa.gov>.
Thanks Chris,
I created an issue:
https://issues.apache.org/jira/browse/OODT-382
but could not find a 0.5 unreleased version to assign it to. I'll reassign it once it is created.
thanks, Luca
On Feb 21, 2012, at 2:50 PM, Mattmann, Chris A (388J) wrote:
> Hi Luca,
>
> +1 to filing an issue to fix this in 0.5...
>
> Cheers,
> Chris
>
> On Feb 21, 2012, at 7:26 AM, Cinquini, Luca (3880) wrote:
>
>> Hi Sean and Chris,
>> on the same topic, I think the desired behavior would be that if the FM is configured to use Lucene in the back-end, an empty but usable catalog would be created at startup, without having to wait for the first ingestion. Otherwise, as Sean noted, you can't query the FM until something gets ingested.
>> If nobody disagree, I can file a Jira issue for 0.5 ?
>> thanks, Luca
>>
>> On Feb 21, 2012, at 12:24 AM, Mattmann, Chris A (388J) wrote:
>>
>>> Hey Sean,
>>>
>>> Looking at the stack trace below, it looks like you had an existing catalog directory, but it didn't have anything inside of it.
>>>
>>> I think you need to *first* delete the catalog directory, restart the file manager, and then try again. It should work at that point,
>>> and that was the same behavior from 0.3.
>>>
>>> Let me know if that addresses it.
>>>
>>> Cheers,
>>> Chris
>>>
>>> On Feb 20, 2012, at 6:57 PM, Hardman, Sean H (388J) wrote:
>>>
>>>> Sorry for spamming the list today but I have been doing some comparative tests today with the File Manager. When populating a newly installed version (0.4-SNAPSHOT) of the File Manager, I was unable to ingest a product until I supplied an existing (yet empty of products) Lucene Catalog for the File Manager to populate. I have included the stack trace from both the client and the server below. This isn't the behavior seen in the 0.3 release. If the Lucene Catalog does not exist in 0.3, it is created upon the first file ingestion.
>>>>
>>>> Once again, I am not sure whether this is worthy of a JIRA issue. I am just pointing out differing behavior.
>>>>
>>>> Thanks,
>>>> Sean
>>>>
>>>>
>>>> Stack trace from filemgr-client:
>>>>
>>>> bash-3.2$ ./filemgr-client --url http://localhost:9000 --operation --ingestProduct --productName Test --productStructure Flat --productTypeName CatalogFile --metadataFile "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt.met" --refs "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt"
>>>> Feb 20, 2012 6:49:30 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
>>>> INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties]
>>>> org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
>>>> at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
>>>> at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
>>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1162)
>>>> at org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:102)
>>>> at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:296)
>>>> at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:179)
>>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1307)
>>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
>>>> SEVERE: Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) -- rolling back ingest
>>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
>>>> SEVERE: Failed to rollback ingest of product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> ERROR: Failed to ingest product 'Test' : Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>>
>>>> Stack trace from filemgr:
>>>>
>>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [CatalogFile]
>>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [CatalogFile]
>>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor] for product type: [CatalogFile]
>>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog addCompleteProductToIndex
>>>> WARNING: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> java.io.FileNotFoundException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> at java.io.RandomAccessFile.open(Native Method)
>>>> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
>>>> at org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:430)
>>>> at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:439)
>>>> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:329)
>>>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:45)
>>>> at org.apache.lucene.index.IndexWriter$1.doBody(IndexWriter.java:264)
>>>> at org.apache.lucene.store.Lock$With.run(Lock.java:99)
>>>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:259)
>>>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:204)
>>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:990)
>>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
>>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
>>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:1005)
>>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
>>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
>>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog removeProductDocument
>>>> WARNING: Exception removing product: [Test] from index: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager removeProduct
>>>> WARNING: Exception modifying product: [null]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProductDocument(LuceneCatalog.java:962)
>>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProduct(LuceneCatalog.java:298)
>>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.removeProduct(XmlRpcFileManager.java:926)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>>
>>>
>>>
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattmann@nasa.gov
>>> WWW: http://sunset.usc.edu/~mattmann/
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
Re: Issue with File Manager initiating a Lucene Catalog
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Luca,
+1 to filing an issue to fix this in 0.5...
Cheers,
Chris
On Feb 21, 2012, at 7:26 AM, Cinquini, Luca (3880) wrote:
> Hi Sean and Chris,
> on the same topic, I think the desired behavior would be that if the FM is configured to use Lucene in the back-end, an empty but usable catalog would be created at startup, without having to wait for the first ingestion. Otherwise, as Sean noted, you can't query the FM until something gets ingested.
> If nobody disagree, I can file a Jira issue for 0.5 ?
> thanks, Luca
>
> On Feb 21, 2012, at 12:24 AM, Mattmann, Chris A (388J) wrote:
>
>> Hey Sean,
>>
>> Looking at the stack trace below, it looks like you had an existing catalog directory, but it didn't have anything inside of it.
>>
>> I think you need to *first* delete the catalog directory, restart the file manager, and then try again. It should work at that point,
>> and that was the same behavior from 0.3.
>>
>> Let me know if that addresses it.
>>
>> Cheers,
>> Chris
>>
>> On Feb 20, 2012, at 6:57 PM, Hardman, Sean H (388J) wrote:
>>
>>> Sorry for spamming the list today but I have been doing some comparative tests today with the File Manager. When populating a newly installed version (0.4-SNAPSHOT) of the File Manager, I was unable to ingest a product until I supplied an existing (yet empty of products) Lucene Catalog for the File Manager to populate. I have included the stack trace from both the client and the server below. This isn't the behavior seen in the 0.3 release. If the Lucene Catalog does not exist in 0.3, it is created upon the first file ingestion.
>>>
>>> Once again, I am not sure whether this is worthy of a JIRA issue. I am just pointing out differing behavior.
>>>
>>> Thanks,
>>> Sean
>>>
>>>
>>> Stack trace from filemgr-client:
>>>
>>> bash-3.2$ ./filemgr-client --url http://localhost:9000 --operation --ingestProduct --productName Test --productStructure Flat --productTypeName CatalogFile --metadataFile "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt.met" --refs "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt"
>>> Feb 20, 2012 6:49:30 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
>>> INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties]
>>> org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
>>> at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
>>> at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1162)
>>> at org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:102)
>>> at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:296)
>>> at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:179)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1307)
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
>>> SEVERE: Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) -- rolling back ingest
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
>>> SEVERE: Failed to rollback ingest of product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> ERROR: Failed to ingest product 'Test' : Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>
>>> Stack trace from filemgr:
>>>
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [CatalogFile]
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [CatalogFile]
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor] for product type: [CatalogFile]
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog addCompleteProductToIndex
>>> WARNING: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> java.io.FileNotFoundException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> at java.io.RandomAccessFile.open(Native Method)
>>> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
>>> at org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:430)
>>> at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:439)
>>> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:329)
>>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:45)
>>> at org.apache.lucene.index.IndexWriter$1.doBody(IndexWriter.java:264)
>>> at org.apache.lucene.store.Lock$With.run(Lock.java:99)
>>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:259)
>>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:204)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:990)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>> at java.lang.Thread.run(Thread.java:680)
>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:1005)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog removeProductDocument
>>> WARNING: Exception removing product: [Test] from index: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager removeProduct
>>> WARNING: Exception modifying product: [null]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProductDocument(LuceneCatalog.java:962)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProduct(LuceneCatalog.java:298)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.removeProduct(XmlRpcFileManager.java:926)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>> at java.lang.Thread.run(Thread.java:680)
>>>
>>
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Issue with File Manager initiating a Lucene Catalog
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Sean,
On Feb 21, 2012, at 10:03 AM, Hardman, Sean H (388J) wrote:
> Hi Chris,
>
> Once again, Dr. Mattmann, you are correct. I could have sworn that I
> created that "catalog" directory because it failed the first time around.
> Anyway, removing the directory resolved the problem.
Great to hear.
>
> I do agree with Luca though. It would be nice if the File Manager created
> an empty Lucene Catalog at startup. My current procedure for CARVE is to
> launch FM, ingest dummy file, delete dummy file, now execute the Crawler
> which performs a unique test prior to ingestion which will fail without
> the catalog being in place.
Yeah agreed, and +1 to Luca's suggestion. I'll reply in his prior email and I think
we should move forward to create a JIRA issue for 0.5. I'm knee deep in wengine
this week and will plan on trying to gear it up to include at least partial support
already implemented in OODT-215 for the RC this week.
Thanks!
Cheers,
Chris
> On 2/21/12 6:26 AM, "Cinquini, Luca (3880)" <Lu...@jpl.nasa.gov>
> wrote:
>
>> Hi Sean and Chris,
>> on the same topic, I think the desired behavior would be that if
>> the FM is configured to use Lucene in the back-end, an empty but usable
>> catalog would be created at startup, without having to wait for the first
>> ingestion. Otherwise, as Sean noted, you can't query the FM until
>> something gets ingested.
>> If nobody disagree, I can file a Jira issue for 0.5 ?
>> thanks, Luca
>>
>> On Feb 21, 2012, at 12:24 AM, Mattmann, Chris A (388J) wrote:
>>
>>> Hey Sean,
>>>
>>> Looking at the stack trace below, it looks like you had an existing
>>> catalog directory, but it didn't have anything inside of it.
>>>
>>> I think you need to *first* delete the catalog directory, restart the
>>> file manager, and then try again. It should work at that point,
>>> and that was the same behavior from 0.3.
>>>
>>> Let me know if that addresses it.
>>>
>>> Cheers,
>>> Chris
>>>
>>> On Feb 20, 2012, at 6:57 PM, Hardman, Sean H (388J) wrote:
>>>
>>>> Sorry for spamming the list today but I have been doing some
>>>> comparative tests today with the File Manager. When populating a newly
>>>> installed version (0.4-SNAPSHOT) of the File Manager, I was unable to
>>>> ingest a product until I supplied an existing (yet empty of products)
>>>> Lucene Catalog for the File Manager to populate. I have included the
>>>> stack trace from both the client and the server below. This isn't the
>>>> behavior seen in the 0.3 release. If the Lucene Catalog does not exist
>>>> in 0.3, it is created upon the first file ingestion.
>>>>
>>>> Once again, I am not sure whether this is worthy of a JIRA issue. I am
>>>> just pointing out differing behavior.
>>>>
>>>> Thanks,
>>>> Sean
>>>>
>>>>
>>>> Stack trace from filemgr-client:
>>>>
>>>> bash-3.2$ ./filemgr-client --url http://localhost:9000 --operation
>>>> --ingestProduct --productName Test --productStructure Flat
>>>> --productTypeName CatalogFile --metadataFile
>>>> "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt.m
>>>> et" --refs
>>>> "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt"
>>>> Feb 20, 2012 6:49:30 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
>>>> INFO: Loading File Manager Configuration Properties from:
>>>> [../etc/filemgr.properties]
>>>> org.apache.xmlrpc.XmlRpcException: java.lang.Exception:
>>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
>>>> ingesting product
>>>> [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to
>>>> index product: [Test]: Message:
>>>> /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> at
>>>> org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcCl
>>>> ientResponseProcessor.java:104)
>>>> at
>>>> org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcCli
>>>> entResponseProcessor.java:71)
>>>> at
>>>> org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
>>>> at
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct
>>>> (XmlRpcFileManagerClient.java:1162)
>>>> at
>>>> org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(In
>>>> gestProductCliAction.java:102)
>>>> at
>>>> org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:296)
>>>> at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:179)
>>>> at
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFi
>>>> leManagerClient.java:1307)
>>>> Feb 20, 2012 6:49:31 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
>>>> SEVERE: Failed to ingest product
>>>> [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] :
>>>> java.lang.Exception:
>>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
>>>> ingesting product
>>>> [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to
>>>> index product: [Test]: Message:
>>>> /Users/shardman/deploy/oodt/catalog/segments (No such file or
>>>> directory) -- rolling back ingest
>>>> Feb 20, 2012 6:49:31 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
>>>> SEVERE: Failed to rollback ingest of product
>>>> [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] :
>>>> java.lang.Exception:
>>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
>>>> ingesting product
>>>> [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to
>>>> index product: [Test]: Message:
>>>> /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> ERROR: Failed to ingest product 'Test' : Failed to ingest product
>>>> [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] :
>>>> java.lang.Exception:
>>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
>>>> ingesting product
>>>> [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to
>>>> index product: [Test]: Message:
>>>> /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>>
>>>> Stack trace from filemgr:
>>>>
>>>> Feb 20, 2012 6:49:31 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>>> INFO: Running Met Extractor:
>>>> [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for
>>>> product type: [CatalogFile]
>>>> Feb 20, 2012 6:49:31 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>>> INFO: Running Met Extractor:
>>>> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtrac
>>>> tor] for product type: [CatalogFile]
>>>> Feb 20, 2012 6:49:31 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>>> INFO: Running Met Extractor:
>>>> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocat
>>>> ionExtractor] for product type: [CatalogFile]
>>>> Feb 20, 2012 6:49:31 PM
>>>> org.apache.oodt.cas.filemgr.catalog.LuceneCatalog
>>>> addCompleteProductToIndex
>>>> WARNING: Unable to index product: [Test]: Message:
>>>> /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> java.io.FileNotFoundException:
>>>> /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> at java.io.RandomAccessFile.open(Native Method)
>>>> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
>>>> at
>>>> org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:
>>>> 430)
>>>> at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:439)
>>>> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:329)
>>>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:45)
>>>> at org.apache.lucene.index.IndexWriter$1.doBody(IndexWriter.java:264)
>>>> at org.apache.lucene.store.Lock$With.run(Lock.java:99)
>>>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:259)
>>>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:204)
>>>> at
>>>> org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIn
>>>> dex(LuceneCatalog.java:990)
>>>> at
>>>> org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(L
>>>> uceneCatalog.java:336)
>>>> at
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReference
>>>> s(XmlRpcFileManager.java:1097)
>>>> at
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRp
>>>> cFileManager.java:740)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
>>>> a:39)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
>>>> Impl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException:
>>>> Unable to index product: [Test]: Message:
>>>> /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> at
>>>> org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIn
>>>> dex(LuceneCatalog.java:1005)
>>>> at
>>>> org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(L
>>>> uceneCatalog.java:336)
>>>> at
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReference
>>>> s(XmlRpcFileManager.java:1097)
>>>> at
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRp
>>>> cFileManager.java:740)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
>>>> a:39)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
>>>> Impl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> Feb 20, 2012 6:49:31 PM
>>>> org.apache.oodt.cas.filemgr.catalog.LuceneCatalog removeProductDocument
>>>> WARNING: Exception removing product: [Test] from index: Message:
>>>> /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> Feb 20, 2012 6:49:31 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager removeProduct
>>>> WARNING: Exception modifying product: [null]: Message:
>>>> /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException:
>>>> /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>> at
>>>> org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProductDocument(
>>>> LuceneCatalog.java:962)
>>>> at
>>>> org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProduct(LuceneCa
>>>> talog.java:298)
>>>> at
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.removeProduct(XmlRp
>>>> cFileManager.java:926)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
>>>> a:39)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
>>>> Impl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>>
>>>
>>>
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattmann@nasa.gov
>>> WWW: http://sunset.usc.edu/~mattmann/
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Issue with File Manager initiating a Lucene Catalog
Posted by "Hardman, Sean H (388J)" <se...@jpl.nasa.gov>.
Hi Chris,
Once again, Dr. Mattmann, you are correct. I could have sworn that I
created that "catalog" directory because it failed the first time around.
Anyway, removing the directory resolved the problem.
I do agree with Luca though. It would be nice if the File Manager created
an empty Lucene Catalog at startup. My current procedure for CARVE is to
launch FM, ingest dummy file, delete dummy file, now execute the Crawler
which performs a unique test prior to ingestion which will fail without
the catalog being in place.
Thanks,
Sean
On 2/21/12 6:26 AM, "Cinquini, Luca (3880)" <Lu...@jpl.nasa.gov>
wrote:
>Hi Sean and Chris,
> on the same topic, I think the desired behavior would be that if
>the FM is configured to use Lucene in the back-end, an empty but usable
>catalog would be created at startup, without having to wait for the first
>ingestion. Otherwise, as Sean noted, you can't query the FM until
>something gets ingested.
>If nobody disagree, I can file a Jira issue for 0.5 ?
>thanks, Luca
>
>On Feb 21, 2012, at 12:24 AM, Mattmann, Chris A (388J) wrote:
>
>> Hey Sean,
>>
>> Looking at the stack trace below, it looks like you had an existing
>>catalog directory, but it didn't have anything inside of it.
>>
>> I think you need to *first* delete the catalog directory, restart the
>>file manager, and then try again. It should work at that point,
>> and that was the same behavior from 0.3.
>>
>> Let me know if that addresses it.
>>
>> Cheers,
>> Chris
>>
>> On Feb 20, 2012, at 6:57 PM, Hardman, Sean H (388J) wrote:
>>
>>> Sorry for spamming the list today but I have been doing some
>>>comparative tests today with the File Manager. When populating a newly
>>>installed version (0.4-SNAPSHOT) of the File Manager, I was unable to
>>>ingest a product until I supplied an existing (yet empty of products)
>>>Lucene Catalog for the File Manager to populate. I have included the
>>>stack trace from both the client and the server below. This isn't the
>>>behavior seen in the 0.3 release. If the Lucene Catalog does not exist
>>>in 0.3, it is created upon the first file ingestion.
>>>
>>> Once again, I am not sure whether this is worthy of a JIRA issue. I am
>>>just pointing out differing behavior.
>>>
>>> Thanks,
>>> Sean
>>>
>>>
>>> Stack trace from filemgr-client:
>>>
>>> bash-3.2$ ./filemgr-client --url http://localhost:9000 --operation
>>>--ingestProduct --productName Test --productStructure Flat
>>>--productTypeName CatalogFile --metadataFile
>>>"file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt.m
>>>et" --refs
>>>"file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt"
>>> Feb 20, 2012 6:49:30 PM
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
>>> INFO: Loading File Manager Configuration Properties from:
>>>[../etc/filemgr.properties]
>>> org.apache.xmlrpc.XmlRpcException: java.lang.Exception:
>>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
>>>ingesting product
>>>[org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to
>>>index product: [Test]: Message:
>>>/Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> at
>>>org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcCl
>>>ientResponseProcessor.java:104)
>>> at
>>>org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcCli
>>>entResponseProcessor.java:71)
>>> at
>>>org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
>>> at
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct
>>>(XmlRpcFileManagerClient.java:1162)
>>> at
>>>org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(In
>>>gestProductCliAction.java:102)
>>> at
>>>org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:296)
>>> at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:179)
>>> at
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFi
>>>leManagerClient.java:1307)
>>> Feb 20, 2012 6:49:31 PM
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
>>> SEVERE: Failed to ingest product
>>>[org.apache.oodt.cas.filemgr.structs.Product@535c97e4] :
>>>java.lang.Exception:
>>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
>>>ingesting product
>>>[org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to
>>>index product: [Test]: Message:
>>>/Users/shardman/deploy/oodt/catalog/segments (No such file or
>>>directory) -- rolling back ingest
>>> Feb 20, 2012 6:49:31 PM
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
>>> SEVERE: Failed to rollback ingest of product
>>>[org.apache.oodt.cas.filemgr.structs.Product@535c97e4] :
>>>java.lang.Exception:
>>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
>>>ingesting product
>>>[org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to
>>>index product: [Test]: Message:
>>>/Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> ERROR: Failed to ingest product 'Test' : Failed to ingest product
>>>[org.apache.oodt.cas.filemgr.structs.Product@535c97e4] :
>>>java.lang.Exception:
>>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
>>>ingesting product
>>>[org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to
>>>index product: [Test]: Message:
>>>/Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>>
>>> Stack trace from filemgr:
>>>
>>> Feb 20, 2012 6:49:31 PM
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor:
>>>[org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for
>>>product type: [CatalogFile]
>>> Feb 20, 2012 6:49:31 PM
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor:
>>>[org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtrac
>>>tor] for product type: [CatalogFile]
>>> Feb 20, 2012 6:49:31 PM
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor:
>>>[org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocat
>>>ionExtractor] for product type: [CatalogFile]
>>> Feb 20, 2012 6:49:31 PM
>>>org.apache.oodt.cas.filemgr.catalog.LuceneCatalog
>>>addCompleteProductToIndex
>>> WARNING: Unable to index product: [Test]: Message:
>>>/Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> java.io.FileNotFoundException:
>>>/Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> at java.io.RandomAccessFile.open(Native Method)
>>> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
>>> at
>>>org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:
>>>430)
>>> at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:439)
>>> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:329)
>>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:45)
>>> at org.apache.lucene.index.IndexWriter$1.doBody(IndexWriter.java:264)
>>> at org.apache.lucene.store.Lock$With.run(Lock.java:99)
>>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:259)
>>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:204)
>>> at
>>>org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIn
>>>dex(LuceneCatalog.java:990)
>>> at
>>>org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(L
>>>uceneCatalog.java:336)
>>> at
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReference
>>>s(XmlRpcFileManager.java:1097)
>>> at
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRp
>>>cFileManager.java:740)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
>>>a:39)
>>> at
>>>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
>>>Impl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>> at java.lang.Thread.run(Thread.java:680)
>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException:
>>>Unable to index product: [Test]: Message:
>>>/Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> at
>>>org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIn
>>>dex(LuceneCatalog.java:1005)
>>> at
>>>org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(L
>>>uceneCatalog.java:336)
>>> at
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReference
>>>s(XmlRpcFileManager.java:1097)
>>> at
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRp
>>>cFileManager.java:740)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
>>>a:39)
>>> at
>>>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
>>>Impl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Feb 20, 2012 6:49:31 PM
>>>org.apache.oodt.cas.filemgr.catalog.LuceneCatalog removeProductDocument
>>> WARNING: Exception removing product: [Test] from index: Message:
>>>/Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> Feb 20, 2012 6:49:31 PM
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManager removeProduct
>>> WARNING: Exception modifying product: [null]: Message:
>>>/Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException:
>>>/Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> at
>>>org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProductDocument(
>>>LuceneCatalog.java:962)
>>> at
>>>org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProduct(LuceneCa
>>>talog.java:298)
>>> at
>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.removeProduct(XmlRp
>>>cFileManager.java:926)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
>>>a:39)
>>> at
>>>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
>>>Impl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>> at java.lang.Thread.run(Thread.java:680)
>>>
>>
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>
Re: Issue with File Manager initiating a Lucene Catalog
Posted by "Cinquini, Luca (3880)" <Lu...@jpl.nasa.gov>.
Hi Sean and Chris,
on the same topic, I think the desired behavior would be that if the FM is configured to use Lucene in the back-end, an empty but usable catalog would be created at startup, without having to wait for the first ingestion. Otherwise, as Sean noted, you can't query the FM until something gets ingested.
If nobody disagree, I can file a Jira issue for 0.5 ?
thanks, Luca
On Feb 21, 2012, at 12:24 AM, Mattmann, Chris A (388J) wrote:
> Hey Sean,
>
> Looking at the stack trace below, it looks like you had an existing catalog directory, but it didn't have anything inside of it.
>
> I think you need to *first* delete the catalog directory, restart the file manager, and then try again. It should work at that point,
> and that was the same behavior from 0.3.
>
> Let me know if that addresses it.
>
> Cheers,
> Chris
>
> On Feb 20, 2012, at 6:57 PM, Hardman, Sean H (388J) wrote:
>
>> Sorry for spamming the list today but I have been doing some comparative tests today with the File Manager. When populating a newly installed version (0.4-SNAPSHOT) of the File Manager, I was unable to ingest a product until I supplied an existing (yet empty of products) Lucene Catalog for the File Manager to populate. I have included the stack trace from both the client and the server below. This isn't the behavior seen in the 0.3 release. If the Lucene Catalog does not exist in 0.3, it is created upon the first file ingestion.
>>
>> Once again, I am not sure whether this is worthy of a JIRA issue. I am just pointing out differing behavior.
>>
>> Thanks,
>> Sean
>>
>>
>> Stack trace from filemgr-client:
>>
>> bash-3.2$ ./filemgr-client --url http://localhost:9000 --operation --ingestProduct --productName Test --productStructure Flat --productTypeName CatalogFile --metadataFile "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt.met" --refs "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt"
>> Feb 20, 2012 6:49:30 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
>> INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties]
>> org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>> at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
>> at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
>> at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1162)
>> at org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:102)
>> at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:296)
>> at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:179)
>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1307)
>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
>> SEVERE: Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) -- rolling back ingest
>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
>> SEVERE: Failed to rollback ingest of product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>> ERROR: Failed to ingest product 'Test' : Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>
>> Stack trace from filemgr:
>>
>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [CatalogFile]
>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [CatalogFile]
>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor] for product type: [CatalogFile]
>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog addCompleteProductToIndex
>> WARNING: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>> java.io.FileNotFoundException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>> at java.io.RandomAccessFile.open(Native Method)
>> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
>> at org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:430)
>> at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:439)
>> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:329)
>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:45)
>> at org.apache.lucene.index.IndexWriter$1.doBody(IndexWriter.java:264)
>> at org.apache.lucene.store.Lock$With.run(Lock.java:99)
>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:259)
>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:204)
>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:990)
>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>> at java.lang.Thread.run(Thread.java:680)
>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:1005)
>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>> at java.lang.Thread.run(Thread.java:680)
>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog removeProductDocument
>> WARNING: Exception removing product: [Test] from index: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager removeProduct
>> WARNING: Exception modifying product: [null]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProductDocument(LuceneCatalog.java:962)
>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProduct(LuceneCatalog.java:298)
>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.removeProduct(XmlRpcFileManager.java:926)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>> at java.lang.Thread.run(Thread.java:680)
>>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
Re: Issue with File Manager initiating a Lucene Catalog
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Sean,
Looking at the stack trace below, it looks like you had an existing catalog directory, but it didn't have anything inside of it.
I think you need to *first* delete the catalog directory, restart the file manager, and then try again. It should work at that point,
and that was the same behavior from 0.3.
Let me know if that addresses it.
Cheers,
Chris
On Feb 20, 2012, at 6:57 PM, Hardman, Sean H (388J) wrote:
> Sorry for spamming the list today but I have been doing some comparative tests today with the File Manager. When populating a newly installed version (0.4-SNAPSHOT) of the File Manager, I was unable to ingest a product until I supplied an existing (yet empty of products) Lucene Catalog for the File Manager to populate. I have included the stack trace from both the client and the server below. This isn't the behavior seen in the 0.3 release. If the Lucene Catalog does not exist in 0.3, it is created upon the first file ingestion.
>
> Once again, I am not sure whether this is worthy of a JIRA issue. I am just pointing out differing behavior.
>
> Thanks,
> Sean
>
>
> Stack trace from filemgr-client:
>
> bash-3.2$ ./filemgr-client --url http://localhost:9000 --operation --ingestProduct --productName Test --productStructure Flat --productTypeName CatalogFile --metadataFile "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt.met" --refs "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt"
> Feb 20, 2012 6:49:30 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
> INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties]
> org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
> at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
> at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
> at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1162)
> at org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:102)
> at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:296)
> at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:179)
> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1307)
> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
> SEVERE: Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) -- rolling back ingest
> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
> SEVERE: Failed to rollback ingest of product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
> ERROR: Failed to ingest product 'Test' : Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>
> Stack trace from filemgr:
>
> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [CatalogFile]
> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [CatalogFile]
> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor] for product type: [CatalogFile]
> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog addCompleteProductToIndex
> WARNING: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
> java.io.FileNotFoundException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
> at org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:430)
> at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:439)
> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:329)
> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:45)
> at org.apache.lucene.index.IndexWriter$1.doBody(IndexWriter.java:264)
> at org.apache.lucene.store.Lock$With.run(Lock.java:99)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:259)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:204)
> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:990)
> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
> at java.lang.Thread.run(Thread.java:680)
> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:1005)
> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
> at java.lang.Thread.run(Thread.java:680)
> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog removeProductDocument
> WARNING: Exception removing product: [Test] from index: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager removeProduct
> WARNING: Exception modifying product: [null]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProductDocument(LuceneCatalog.java:962)
> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProduct(LuceneCatalog.java:298)
> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.removeProduct(XmlRpcFileManager.java:926)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
> at java.lang.Thread.run(Thread.java:680)
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++