FileNotFoundException

[Chantal Ackermann <ch...@biomax.de>]

hello all,

I am still trying to find the best way to index a really big amount of data. 
at the moment I am trying to index each of the 29 textfiles in a single 
thread using for each an own IndexWriter and an own directory where to place 
the index. there are always six threads working the same time.

the problem that occures now is that every second thread stops due to a 
FileNotFoundException or an ArrayIndexOutOfBoundsException (the latter only 
once) while the other half finishes fine. the file's name is different for 
each thread but has always the extension ".fnm".

for example:
java.io.FileNotFoundException: 
/lucenetest/medlineIndex/1976-1977/_2zfj.fnm (Datei oder 
Verzeichnis nicht gefunden)
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java(Compiled 
Code))        at 
java.io.RandomAccessFile.<init>(RandomAccessFile.java(Compiled Code))        
at org.apache.lucene.store.FSInputStream.<init>(Unknown Source)
        at org.apache.lucene.store.FSInputStream.<init>(Unknown Source)
        at org.apache.lucene.store.FSDirectory.openFile(Unknown Source)
        at org.apache.lucene.index.FieldInfos.<init>(Unknown Source)
        at org.apache.lucene.index.SegmentReader.<init>(Unknown Source)
        at org.apache.lucene.index.IndexWriter.mergeSegments(Unknown Source)
        at org.apache.lucene.index.IndexWriter.maybeMergeSegments(Unknown 
Source)
        at org.apache.lucene.index.IndexWriter.maybeMergeSegments(Unknown 
Source)
        at org.apache.lucene.index.IndexWriter.addDocument(Unknown Source)
        at 
de.biomax.lucenetest.MedlineRecordIndexer.indexDocs(MedlineRecordIndexer.java(Compiled 
Code))

since half of the files are indexed without throwing that kind of exception 
I'm at a loss where to start debugging. any ideas?

thanks a lot
chantal

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: FileNotFoundException

[Ian Lea <ia...@blackwell.co.uk>]

I don't have an explanation for this, but if it was me indexing
this large amount of data I'd be running each of the 6 in a
completely separate process.  More control, less damage when
one bit fails, perhaps better performance on a multi-processor
machine.  And perhaps you wouldn't get this problem!


--
Ian.
ian.lea@blackwell.co.uk


Chantal Ackermann wrote:
> 
> hello all,
> 
> I am still trying to find the best way to index a really big amount of data.
> at the moment I am trying to index each of the 29 textfiles in a single
> thread using for each an own IndexWriter and an own directory where to place
> the index. there are always six threads working the same time.
> 
> the problem that occures now is that every second thread stops due to a
> FileNotFoundException or an ArrayIndexOutOfBoundsException (the latter only
> once) while the other half finishes fine. the file's name is different for
> each thread but has always the extension ".fnm".
> 
> for example:
> java.io.FileNotFoundException:
> /lucenetest/medlineIndex/1976-1977/_2zfj.fnm (Datei oder
> Verzeichnis nicht gefunden)
>         at java.io.RandomAccessFile.open(Native Method)
>         at java.io.RandomAccessFile.<init>(RandomAccessFile.java(Compiled
> Code))        at
> java.io.RandomAccessFile.<init>(RandomAccessFile.java(Compiled Code))
> at org.apache.lucene.store.FSInputStream.<init>(Unknown Source)
>         at org.apache.lucene.store.FSInputStream.<init>(Unknown Source)
>         at org.apache.lucene.store.FSDirectory.openFile(Unknown Source)
>         at org.apache.lucene.index.FieldInfos.<init>(Unknown Source)
>         at org.apache.lucene.index.SegmentReader.<init>(Unknown Source)
>         at org.apache.lucene.index.IndexWriter.mergeSegments(Unknown Source)
>         at org.apache.lucene.index.IndexWriter.maybeMergeSegments(Unknown
> Source)
>         at org.apache.lucene.index.IndexWriter.maybeMergeSegments(Unknown
> Source)
>         at org.apache.lucene.index.IndexWriter.addDocument(Unknown Source)
>         at
> de.biomax.lucenetest.MedlineRecordIndexer.indexDocs(MedlineRecordIndexer.java(Compiled
> Code))
> 
> since half of the files are indexed without throwing that kind of exception
> I'm at a loss where to start debugging. any ideas?
> 
> thanks a lot
> chantal

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>