reindexing fails on a particular file

[Scott Derrick <sc...@tnstaafl.net>]

HI,

     I have about 6,000 files to index using a FileDataSource as the 
dataSource.   This uses a FileListEntityProcessor to get the filenames 
in a folder to process.

     The reindexing is failing on file # 3091.   Something changed 
between version 7.3 and 8.11?

org.apache.solr.handler.dataimport.DataImportHandlerException: Exception 
in applying XSL Transformation Processing Document # 3091

     No filename given, just a number?  So I did a listing "ls -all" in 
the folder, found the 3091'st file deleted it ran the update and it 
failed on the 3091'st file again...

     I then deleted the first 16 files and it fails on file # 3080

     So I need to find out what file is causing the issue, but can't 
seem to get that to out put.  I can see a FileDataSource in the 
dataimport section of the Logging UI in the web app. I set it to ALL but 
it didn't change the output of the log file.

Any ideas how to get the logger to output something useful?

Scott

Re: reindexing fails on a particular file

[Mikhail Khludnev <mk...@apache.org>]

Hi,
You can enable log on wider category, and it give you a clue:
https://github.com/SearchScale/dataimporthandler/blob/branch_9x/src/main/java/org/apache/solr/handler/dataimport/FileDataSource.java#L120
Assuming the error occurs in XPathEntityProcessor, it log in case if
onError=continue
https://github.com/SearchScale/dataimporthandler/blob/branch_9x/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java#L316

On Sat, Mar 9, 2024 at 3:32 AM Scott Derrick <sc...@tnstaafl.net> wrote:

> HI,
>
>      I have about 6,000 files to index using a FileDataSource as the
> dataSource.   This uses a FileListEntityProcessor to get the filenames
> in a folder to process.
>
>      The reindexing is failing on file # 3091.   Something changed
> between version 7.3 and 8.11?
>
> org.apache.solr.handler.dataimport.DataImportHandlerException: Exception
> in applying XSL Transformation Processing Document # 3091
>
>      No filename given, just a number?  So I did a listing "ls -all" in
> the folder, found the 3091'st file deleted it ran the update and it
> failed on the 3091'st file again...
>
>      I then deleted the first 16 files and it fails on file # 3080
>
>      So I need to find out what file is causing the issue, but can't
> seem to get that to out put.  I can see a FileDataSource in the
> dataimport section of the Logging UI in the web app. I set it to ALL but
> it didn't change the output of the log file.
>
> Any ideas how to get the logger to output something useful?
>
> Scott
>
>

-- 
Sincerely yours
Mikhail Khludnev