You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Jason Rutherglen <ja...@gmail.com> on 2008/07/12 00:39:06 UTC
CorruptIndexException docs out of order
Periodically seeing this exception when testing out Ocean. What would be a
possible cause for this? I assume it is a problem in the index. The code
is merging custom segmentreaders that are created using
http://issues.apache.org/jira/browse/LUCENE-1314 IndexReader.clone. I need
to isolate if it is in the LUCENE-1314 code or my own somehow. Is there an
assertion that can be made at index creation to insure the docs are not
added out of order? This may help.
org.apache.lucene.index.CorruptIndexException: docs out of order (3 <= 9 )
at
org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:621)
at
org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:567)
at
org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:536)
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:489)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:135)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3045)
Re: CorruptIndexException docs out of order
Posted by Jason Rutherglen <ja...@gmail.com>.
Thanks. Yes what you said is correct. There were two bugs, one in the
patch and a few in Ocean since fixed.
On Fri, Jul 11, 2008 at 7:49 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:
>
> I think if your SegmentReader is not returning the right result when
> numDocs() is called it can lead to this.
>
> Eg if your maxDoc() is 1000 and you think you have 100 deleted docs (so
> numDocs() returns 900) but upon iterating through the docs you only saw say
> 50 that were deleted, then, when appendPostings hits the next segment it
> will see the docID go backwards and throw that exception, I think.
>
> Mike
>
>
> Jason Rutherglen wrote:
>
> Periodically seeing this exception when testing out Ocean. What would be
>> a possible cause for this? I assume it is a problem in the index. The code
>> is merging custom segmentreaders that are created using
>> http://issues.apache.org/jira/browse/LUCENE-1314 IndexReader.clone. I
>> need to isolate if it is in the LUCENE-1314 code or my own somehow. Is
>> there an assertion that can be made at index creation to insure the docs are
>> not added out of order? This may help.
>>
>> org.apache.lucene.index.CorruptIndexException: docs out of order (3 <= 9 )
>> at
>> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:621)
>> at
>> org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:567)
>> at
>> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:536)
>> at
>> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:489)
>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:135)
>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
>> at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3045)
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
Re: CorruptIndexException docs out of order
Posted by Michael McCandless <lu...@mikemccandless.com>.
I think if your SegmentReader is not returning the right result when
numDocs() is called it can lead to this.
Eg if your maxDoc() is 1000 and you think you have 100 deleted docs
(so numDocs() returns 900) but upon iterating through the docs you
only saw say 50 that were deleted, then, when appendPostings hits the
next segment it will see the docID go backwards and throw that
exception, I think.
Mike
Jason Rutherglen wrote:
> Periodically seeing this exception when testing out Ocean. What
> would be a possible cause for this? I assume it is a problem in the
> index. The code is merging custom segmentreaders that are created
> using http://issues.apache.org/jira/browse/LUCENE-1314
> IndexReader.clone. I need to isolate if it is in the LUCENE-1314
> code or my own somehow. Is there an assertion that can be made at
> index creation to insure the docs are not added out of order? This
> may help.
>
> org.apache.lucene.index.CorruptIndexException: docs out of order (3
> <= 9 )
> at
> org
> .apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:
> 621)
> at
> org
> .apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:
> 567)
> at
> org
> .apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:
> 536)
> at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:
> 489)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:135)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
> at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:
> 3045)
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org