You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by "Nestel, Frank IZ/HZA-IOL" <ne...@de.ina.com> on 2005/01/25 13:54:00 UTC

Strange Index problem

Hello,

we've been using Lucene here with great success in different
projects. 

In one project we've a system which incrementally updates
an index every night. This has been working fine. We've 
upgraded to Lucene 1.4.2 when it was there without observing a 
difference instantly. But now we regularly run into trouble.
It seems like our index has "captured" a very defunc document
and as long as you work around this document the index
is still working, but as soon as you touch that particular
document, you run into trouble:

	java.lang.IndexOutOfBoundsException: Index: 114, Size: 19
at java.util.ArrayList.RangeCheck	(ArrayList.java:547) 	at
java.util.ArrayList.get(ArrayList.java:322) 	at
org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:155)
at 	org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:66)
at
org.apache.lucene.index.SegmentReader.document(SegmentReader.java:237)
at
org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:185
) 	at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:92)
at
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:487)
at
org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:366) 

We can also create this by calling the indexReader.document() function
directly
with the right document number. It seems as the whole index is only
containing
only one such document.

The questions remains, how can one "achieve" to insert such a defunc
document 
in an index, or more interesting how can that be avoided in the future?
Note 
that we rely on that "update" and do not rebuild strategy at least on
most 
of the days!

Best wishes,
Frank

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Re: Strange Index problem

Posted by Geir Ove Grønmo <gr...@ontopia.net>.
On Tue, 25 Jan 2005 13:54:00 +0100, Nestel, Frank  IZ/HZA-IOL  
<ne...@de.ina.com> wrote:

> In one project we've a system which incrementally updates
> an index every night. This has been working fine. We've
> upgraded to Lucene 1.4.2 when it was there without observing a
> difference instantly. But now we regularly run into trouble.
> It seems like our index has "captured" a very defunc document
> and as long as you work around this document the index
> is still working, but as soon as you touch that particular
> document, you run into trouble:
>
> 	java.lang.IndexOutOfBoundsException: Index: 114, Size: 19
> at java.util.ArrayList.RangeCheck	(ArrayList.java:547) 	at
> java.util.ArrayList.get(ArrayList.java:322) 	at
> org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:155)
> at 	org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:66)
> at
> org.apache.lucene.index.SegmentReader.document(SegmentReader.java:237)
> at
> org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:185
> ) 	at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:92)
> at
> org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:487)
> at
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:366)

I've just found a similar traceback in one of our deployed systems (using
version 1.4.3):

java.lang.IndexOutOfBoundsException: Index: 104, Size: 11
         at java.util.ArrayList.RangeCheck(ArrayList.java:507)
         at java.util.ArrayList.get(ArrayList.java:324)
         at  
org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:155)
         at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:66)
         at  
org.apache.lucene.index.SegmentReader.document(SegmentReader.java:237)
         at  
org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:185)
         at  
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:92)
         at  
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:487)
         at  
org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:366)

After having gotten this error from the optimize() call, it is no longer  
possible
to search:

java.io.IOException: read past EOF
         at org.apache.lucene.store.InputStream.refill(InputStream.java:154)
         at  
org.apache.lucene.store.InputStream.readByte(InputStream.java:43)
         at  
org.apache.lucene.store.InputStream.readBytes(InputStream.java:57)
         at  
org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:356)
         at org.apache.lucene.index.MultiReader.norms(MultiReader.java:159)
         at  
org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:64)
         at  
org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:165)
         at  
org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:165)
         at  
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:85)
         at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:64)
         at org.apache.lucene.search.Hits.<init>(Hits.java:43)
         at org.apache.lucene.search.Searcher.search(Searcher.java:33)
         at org.apache.lucene.search.Searcher.search(Searcher.java:27)

Any ideas?

-- 
Geir O.


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org