You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Sascha Szott <sz...@zib.de> on 2010/08/11 15:18:08 UTC

DataImportHandler in Solr 1.4.1: exception handling in FileListEntityProcessor

Hi folks,

why does FileListEntityProcessor ignores onError="continue" and abort 
indexing if a directory or a file does not exist?

I'm using both XPathEntityProcessor and FileListEntityProcessor with 
onError set to continue. In case a directory or file is not present an 
Exception is thrown and indexing is stopped immediately.

Below you can find a stack trace that is generated in case the directory 
/home/doe/foo does not exist:

SEVERE: Full Import failed
org.apache.solr.handler.dataimport.DataImportHandlerException: 'baseDir' 
value: /home/doe/foo/bar.xml is not a directory Processing Document # 3
         at 
org.apache.solr.handler.dataimport.FileListEntityProcessor.init(FileListEntityProcessor.java:122)
         at 
org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:71)
         at 
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:319)
         at 
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
         at 
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
         at 
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
         at 
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
         at 
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
         at 
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)

How should I configure both processors so that missing directories and 
files are ignored and the indexing process does not stop immediately?

Best,
Sascha

Re: DataImportHandler in Solr 1.4.1: exception handling in FileListEntityProcessor

Posted by Lance Norskog <go...@gmail.com>.
Please add a JIRA issue for this.

On Wed, Aug 11, 2010 at 6:24 AM, Sascha Szott <sz...@zib.de> wrote:
> Sorry, there was a mistake in the stack trace. The correct one is:
>
> SEVERE: Full Import failed
> org.apache.solr.handler.dataimport.DataImportHandlerException: 'baseDir'
> value: /home/doe/foo is not a directory Processing Document # 3
>        at
> org.apache.solr.handler.dataimport.FileListEntityProcessor.init(FileListEntityProcessor.java:122)
>        at
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:71)
>        at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:319)
>        at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
>        at
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
>        at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
>        at
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
>        at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
>        at
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
>
> -Sascha
>
> On 11.08.2010 15:18, Sascha Szott wrote:
>>
>> Hi folks,
>>
>> why does FileListEntityProcessor ignores onError="continue" and abort
>> indexing if a directory or a file does not exist?
>>
>> I'm using both XPathEntityProcessor and FileListEntityProcessor with
>> onError set to continue. In case a directory or file is not present an
>> Exception is thrown and indexing is stopped immediately.
>>
>> Below you can find a stack trace that is generated in case the directory
>> /home/doe/foo does not exist:
>>
>> SEVERE: Full Import failed
>> org.apache.solr.handler.dataimport.DataImportHandlerException: 'baseDir'
>> value: /home/doe/foo/bar.xml is not a directory Processing Document # 3
>> at
>>
>> org.apache.solr.handler.dataimport.FileListEntityProcessor.init(FileListEntityProcessor.java:122)
>>
>> at
>>
>> org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:71)
>>
>> at
>>
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:319)
>>
>> at
>>
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
>>
>> at
>>
>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
>>
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
>> at
>>
>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
>>
>> at
>>
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
>>
>> at
>>
>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
>>
>>
>> How should I configure both processors so that missing directories and
>> files are ignored and the indexing process does not stop immediately?
>>
>> Best,
>> Sascha
>



-- 
Lance Norskog
goksron@gmail.com

Re: DataImportHandler in Solr 1.4.1: exception handling in FileListEntityProcessor

Posted by Sascha Szott <sz...@zib.de>.
Sorry, there was a mistake in the stack trace. The correct one is:

SEVERE: Full Import failed
org.apache.solr.handler.dataimport.DataImportHandlerException: 'baseDir' 
value: /home/doe/foo is not a directory Processing Document # 3
         at 
org.apache.solr.handler.dataimport.FileListEntityProcessor.init(FileListEntityProcessor.java:122)
         at 
org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:71)
         at 
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:319)
         at 
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
         at 
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
         at 
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
         at 
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
         at 
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
         at 
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370) 


-Sascha

On 11.08.2010 15:18, Sascha Szott wrote:
> Hi folks,
>
> why does FileListEntityProcessor ignores onError="continue" and abort
> indexing if a directory or a file does not exist?
>
> I'm using both XPathEntityProcessor and FileListEntityProcessor with
> onError set to continue. In case a directory or file is not present an
> Exception is thrown and indexing is stopped immediately.
>
> Below you can find a stack trace that is generated in case the directory
> /home/doe/foo does not exist:
>
> SEVERE: Full Import failed
> org.apache.solr.handler.dataimport.DataImportHandlerException: 'baseDir'
> value: /home/doe/foo/bar.xml is not a directory Processing Document # 3
> at
> org.apache.solr.handler.dataimport.FileListEntityProcessor.init(FileListEntityProcessor.java:122)
>
> at
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:71)
>
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:319)
>
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:383)
>
> at
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242)
>
> at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180)
> at
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331)
>
> at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389)
>
> at
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370)
>
>
> How should I configure both processors so that missing directories and
> files are ignored and the indexing process does not stop immediately?
>
> Best,
> Sascha