You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by James Brady <ja...@gmail.com> on 2009/02/22 13:30:22 UTC
Persistent, seemingly unfixable corrupt indices
Hi,My indices sometime become corrupted - normally when Solr has to be
KILLed - these are not normally too much of a problem, as
Lucene's CheckIndex tool can normally detect missing / broken segments and
fix them.
However, I now have a few indices throwing errors like this:
INFO: [core4] webapp=/solr path=/update params={} status=0 QTime=2
Exception in thread "Thread-75"
org.apache.lucene.index.MergePolicy$MergeException:
org.apache.lucene.index.CorruptIndexException: docs out of order (1124 <=
1138 )
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:271)
Caused by: org.apache.lucene.index.CorruptIndexException: docs out of order
(1124 <= 1138 )
at
org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502)
at
org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456)
at
org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425)
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:389)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3109)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:240)
and
INFO: [core7] webapp=/solr path=/update params={} status=500 QTime=5457
Feb 22, 2009 12:14:07 PM org.apache.solr.common.SolrException log
SEVERE: org.apache.lucene.index.CorruptIndexException: docs out of order
(242 <= 248 )
at
org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502)
at
org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456)
at
org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425)
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:389)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3109)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
at
org.apache.lucene.index.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:193)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1800)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1795)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1791)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2398)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1465)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1424)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:278)
CheckIndex reports these cores as being completely healthy, and yet I can't
commit new documents in to them.
Rebuilding indices isn't an option for me: is there any other way to fix
this? If not, any ideas on what I can do to prevent it in the future?
Many thanks,
James
Re: Persistent, seemingly unfixable corrupt indices
Posted by Michael McCandless <lu...@mikemccandless.com>.
This is spooky!
First off, why are you hitting so much index corruption? Many classes
of failure (unhandled exception exits JVM, JVM killed or SEGVs, OS
crashes, power cord is pulled, etc.) should never result in index
corruption. Other failures (bad RAM, bad hard drives) can easily
cause corruption. So I'd really like to understand what kind of
corruption you're seeing and how/why. Why does Solr need to be
killed, and how do you kill it? When CheckIndex does catch the
failure, what failures is it seeing? Is there any pattern to which
indexes become corrupt?
Hmm -- you seem to be using Lucene 2.3.1, so in fact OS crashes and
power cord pulling could lead to corruption. But JVM crashing or
being killed should not. Upgrading to Solr 1.3 (Lucene 2.4) would be
a good idea, though I'd still like to understand what's causing your
corruption.
Second off, you're right: CheckIndex fails to detect the docs-out-of-
order form of corruption. I will open Jira issue & fix it.
Mike
James Brady wrote:
> Hi,My indices sometime become corrupted - normally when Solr has to be
> KILLed - these are not normally too much of a problem, as
> Lucene's CheckIndex tool can normally detect missing / broken
> segments and
> fix them.
>
> However, I now have a few indices throwing errors like this:
>
> INFO: [core4] webapp=/solr path=/update params={} status=0 QTime=2
> Exception in thread "Thread-75"
> org.apache.lucene.index.MergePolicy$MergeException:
> org.apache.lucene.index.CorruptIndexException: docs out of order
> (1124 <=
> 1138 )
> at
> org.apache.lucene.index.ConcurrentMergeScheduler
> $MergeThread.run(ConcurrentMergeScheduler.java:271)
> Caused by: org.apache.lucene.index.CorruptIndexException: docs out
> of order
> (1124 <= 1138 )
> at
> org
> .apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:
> 502)
> at
> org
> .apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:
> 456)
> at
> org
> .apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:
> 425)
> at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:
> 389)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:
> 3109)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler
> $MergeThread.run(ConcurrentMergeScheduler.java:240)
>
> and
>
> INFO: [core7] webapp=/solr path=/update params={} status=500
> QTime=5457
> Feb 22, 2009 12:14:07 PM org.apache.solr.common.SolrException log
> SEVERE: org.apache.lucene.index.CorruptIndexException: docs out of
> order
> (242 <= 248 )
> at
> org
> .apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:
> 502)
> at
> org
> .apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:
> 456)
> at
> org
> .apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:
> 425)
> at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:
> 389)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:
> 3109)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
> at
> org
> .apache
> .lucene
> .index.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:
> 193)
> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
> 1800)
> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
> 1795)
> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
> 1791)
> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2398)
> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:
> 1465)
> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:
> 1424)
> at
> org
> .apache
> .solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:
> 278)
>
>
> CheckIndex reports these cores as being completely healthy, and yet
> I can't
> commit new documents in to them.
>
> Rebuilding indices isn't an option for me: is there any other way to
> fix
> this? If not, any ideas on what I can do to prevent it in the future?
>
> Many thanks,
> James
Re: Persistent, seemingly unfixable corrupt indices
Posted by Michael McCandless <lu...@mikemccandless.com>.
James Brady wrote:
> Thanks for your answers Michael! I was using a pre-1.3 Solr build,
> but I've
> now upgraded to the 1.3 release, run the new CheckIndex shipped as
> part of
> the Lucene 2.4 dev build and I'm still getting the
> CorruptIndexException:
> docs out of order exceptions I'm afraid.
Did you create a new index after upgrading? Or are you saying that
the CheckIndex in Lucene 2.4 is also failing to catch the "docs out of
order" corruption in your original index?
> Upon a fresh start, on newly Checked indices, I actually get a lot of
> Exceptions like:
But when you ran CheckIndex it made no fixes to the index? (you said
"newly Checked").
> SEVERE: java.lang.RuntimeException: [was class
> org.mortbay.jetty.EofException] null
> at
> com
> .ctc
> .wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
> at
> com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
> at
> com
> .ctc
> .wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:
> 3657)
> at
> com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
> at
> org
> .apache
> .solr
> .handler
> .XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:327)
> at
> org
> .apache
> .solr
> .handler
> .XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:
> 195)
> at
> org
> .apache
> .solr
> .handler
> .XmlUpdateRequestHandler
> .handleRequestBody(XmlUpdateRequestHandler.java:123)
>
> Before any CorruptIndexExceptions - could that be the root cause?
This exception looks like something new. I don't think this could
cause index corruption.
I'm not certain, but I think that stack trace is happening when Solr
is attempting to read/parse the incoming XML that your client is
sending it. The XML parser seems to feel it encountered an EOF. So
this looks like a client/server communication issue, I'm guessing.
> Unfortunately the indices are large and contain confidential
> information; is
> there anything else I can do to identify where the problem is and why
> CheckIndex isn't catching it?
Hmm, OK...
Could you post the full output of CheckIndex on the index?
Is there any chance you are allowing two Solr instances to run in the
same area? What's most important here is we get to the root cause of
this corruption. And, I'd like to explain why CheckIndex fails to
catch the "docs out of order" corruption, too.
Mike
Re: Persistent, seemingly unfixable corrupt indices
Posted by Michael McCandless <lu...@mikemccandless.com>.
One more question: when you run CheckIndex, are you enabling asserts?
(java -ea:org.apache.lucene)?
Mike
James Brady wrote:
> Thanks for your answers Michael! I was using a pre-1.3 Solr build,
> but I've
> now upgraded to the 1.3 release, run the new CheckIndex shipped as
> part of
> the Lucene 2.4 dev build and I'm still getting the
> CorruptIndexException:
> docs out of order exceptions I'm afraid.
>
> Upon a fresh start, on newly Checked indices, I actually get a lot of
> Exceptions like:
>
> SEVERE: java.lang.RuntimeException: [was class
> org.mortbay.jetty.EofException] null
> at
> com
> .ctc
> .wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
> at
> com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
> at
> com
> .ctc
> .wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:
> 3657)
> at
> com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
> at
> org
> .apache
> .solr
> .handler
> .XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:327)
> at
> org
> .apache
> .solr
> .handler
> .XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:
> 195)
> at
> org
> .apache
> .solr
> .handler
> .XmlUpdateRequestHandler
> .handleRequestBody(XmlUpdateRequestHandler.java:123)
>
> Before any CorruptIndexExceptions - could that be the root cause?
>
> Unfortunately the indices are large and contain confidential
> information; is
> there anything else I can do to identify where the problem is and why
> CheckIndex isn't catching it?
>
> Thanks
> James
>
> 2009/2/23 Michael McCandless <lu...@mikemccandless.com>
>
>>
>> Actually, even in 2.3.1, CheckIndex checks for docs-out-of-order both
>> within and across segments, so now I'm at a loss as to why it's not
>> catching
>> your case. Any of these indexes small enough to post somewhere i
>> could
>> access?
>>
>> Mike
>>
>>
>> James Brady wrote:
>>
>> Hi,My indices sometime become corrupted - normally when Solr has to
>> be
>>> KILLed - these are not normally too much of a problem, as
>>> Lucene's CheckIndex tool can normally detect missing / broken
>>> segments and
>>> fix them.
>>>
>>> However, I now have a few indices throwing errors like this:
>>>
>>> INFO: [core4] webapp=/solr path=/update params={} status=0 QTime=2
>>> Exception in thread "Thread-75"
>>> org.apache.lucene.index.MergePolicy$MergeException:
>>> org.apache.lucene.index.CorruptIndexException: docs out of order
>>> (1124 <=
>>> 1138 )
>>> at
>>>
>>> org.apache.lucene.index.ConcurrentMergeScheduler
>>> $MergeThread.run(ConcurrentMergeScheduler.java:271)
>>> Caused by: org.apache.lucene.index.CorruptIndexException: docs out
>>> of
>>> order
>>> (1124 <= 1138 )
>>> at
>>>
>>> org
>>> .apache
>>> .lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502)
>>> at
>>>
>>> org
>>> .apache
>>> .lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456)
>>> at
>>>
>>> org
>>> .apache
>>> .lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425)
>>> at
>>> org
>>> .apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:
>>> 389)
>>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:
>>> 134)
>>> at
>>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:
>>> 3109)
>>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
>>> at
>>>
>>> org.apache.lucene.index.ConcurrentMergeScheduler
>>> $MergeThread.run(ConcurrentMergeScheduler.java:240)
>>>
>>> and
>>>
>>> INFO: [core7] webapp=/solr path=/update params={} status=500
>>> QTime=5457
>>> Feb 22, 2009 12:14:07 PM org.apache.solr.common.SolrException log
>>> SEVERE: org.apache.lucene.index.CorruptIndexException: docs out of
>>> order
>>> (242 <= 248 )
>>> at
>>>
>>> org
>>> .apache
>>> .lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502)
>>> at
>>>
>>> org
>>> .apache
>>> .lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456)
>>> at
>>>
>>> org
>>> .apache
>>> .lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425)
>>> at
>>> org
>>> .apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:
>>> 389)
>>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:
>>> 134)
>>> at
>>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:
>>> 3109)
>>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
>>> at
>>>
>>> org
>>> .apache
>>> .lucene
>>> .index
>>> .ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:193)
>>> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
>>> 1800)
>>> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
>>> 1795)
>>> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
>>> 1791)
>>> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2398)
>>> at
>>> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:
>>> 1465)
>>> at
>>> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:
>>> 1424)
>>> at
>>>
>>> org
>>> .apache
>>> .solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:
>>> 278)
>>>
>>>
>>> CheckIndex reports these cores as being completely healthy, and
>>> yet I
>>> can't
>>> commit new documents in to them.
>>>
>>> Rebuilding indices isn't an option for me: is there any other way
>>> to fix
>>> this? If not, any ideas on what I can do to prevent it in the
>>> future?
>>>
>>> Many thanks,
>>> James
>>>
>>
>>
Re: Persistent, seemingly unfixable corrupt indices
Posted by James Brady <ja...@gmail.com>.
Thanks for your answers Michael! I was using a pre-1.3 Solr build, but I've
now upgraded to the 1.3 release, run the new CheckIndex shipped as part of
the Lucene 2.4 dev build and I'm still getting the CorruptIndexException:
docs out of order exceptions I'm afraid.
Upon a fresh start, on newly Checked indices, I actually get a lot of
Exceptions like:
SEVERE: java.lang.RuntimeException: [was class
org.mortbay.jetty.EofException] null
at
com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
at
com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
at
com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657)
at
com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
at
org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:327)
at
org.apache.solr.handler.XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:195)
at
org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:123)
Before any CorruptIndexExceptions - could that be the root cause?
Unfortunately the indices are large and contain confidential information; is
there anything else I can do to identify where the problem is and why
CheckIndex isn't catching it?
Thanks
James
2009/2/23 Michael McCandless <lu...@mikemccandless.com>
>
> Actually, even in 2.3.1, CheckIndex checks for docs-out-of-order both
> within and across segments, so now I'm at a loss as to why it's not catching
> your case. Any of these indexes small enough to post somewhere i could
> access?
>
> Mike
>
>
> James Brady wrote:
>
> Hi,My indices sometime become corrupted - normally when Solr has to be
>> KILLed - these are not normally too much of a problem, as
>> Lucene's CheckIndex tool can normally detect missing / broken segments and
>> fix them.
>>
>> However, I now have a few indices throwing errors like this:
>>
>> INFO: [core4] webapp=/solr path=/update params={} status=0 QTime=2
>> Exception in thread "Thread-75"
>> org.apache.lucene.index.MergePolicy$MergeException:
>> org.apache.lucene.index.CorruptIndexException: docs out of order (1124 <=
>> 1138 )
>> at
>>
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:271)
>> Caused by: org.apache.lucene.index.CorruptIndexException: docs out of
>> order
>> (1124 <= 1138 )
>> at
>>
>> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502)
>> at
>>
>> org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456)
>> at
>>
>> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425)
>> at
>> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:389)
>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134)
>> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3109)
>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
>> at
>>
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:240)
>>
>> and
>>
>> INFO: [core7] webapp=/solr path=/update params={} status=500 QTime=5457
>> Feb 22, 2009 12:14:07 PM org.apache.solr.common.SolrException log
>> SEVERE: org.apache.lucene.index.CorruptIndexException: docs out of order
>> (242 <= 248 )
>> at
>>
>> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502)
>> at
>>
>> org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456)
>> at
>>
>> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425)
>> at
>> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:389)
>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134)
>> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3109)
>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
>> at
>>
>> org.apache.lucene.index.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:193)
>> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1800)
>> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1795)
>> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1791)
>> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2398)
>> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1465)
>> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1424)
>> at
>>
>> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:278)
>>
>>
>> CheckIndex reports these cores as being completely healthy, and yet I
>> can't
>> commit new documents in to them.
>>
>> Rebuilding indices isn't an option for me: is there any other way to fix
>> this? If not, any ideas on what I can do to prevent it in the future?
>>
>> Many thanks,
>> James
>>
>
>
Re: Persistent, seemingly unfixable corrupt indices
Posted by Michael McCandless <lu...@mikemccandless.com>.
Actually, even in 2.3.1, CheckIndex checks for docs-out-of-order both
within and across segments, so now I'm at a loss as to why it's not
catching your case. Any of these indexes small enough to post
somewhere i could access?
Mike
James Brady wrote:
> Hi,My indices sometime become corrupted - normally when Solr has to be
> KILLed - these are not normally too much of a problem, as
> Lucene's CheckIndex tool can normally detect missing / broken
> segments and
> fix them.
>
> However, I now have a few indices throwing errors like this:
>
> INFO: [core4] webapp=/solr path=/update params={} status=0 QTime=2
> Exception in thread "Thread-75"
> org.apache.lucene.index.MergePolicy$MergeException:
> org.apache.lucene.index.CorruptIndexException: docs out of order
> (1124 <=
> 1138 )
> at
> org.apache.lucene.index.ConcurrentMergeScheduler
> $MergeThread.run(ConcurrentMergeScheduler.java:271)
> Caused by: org.apache.lucene.index.CorruptIndexException: docs out
> of order
> (1124 <= 1138 )
> at
> org
> .apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:
> 502)
> at
> org
> .apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:
> 456)
> at
> org
> .apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:
> 425)
> at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:
> 389)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:
> 3109)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler
> $MergeThread.run(ConcurrentMergeScheduler.java:240)
>
> and
>
> INFO: [core7] webapp=/solr path=/update params={} status=500
> QTime=5457
> Feb 22, 2009 12:14:07 PM org.apache.solr.common.SolrException log
> SEVERE: org.apache.lucene.index.CorruptIndexException: docs out of
> order
> (242 <= 248 )
> at
> org
> .apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:
> 502)
> at
> org
> .apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:
> 456)
> at
> org
> .apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:
> 425)
> at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:
> 389)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:
> 3109)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
> at
> org
> .apache
> .lucene
> .index.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:
> 193)
> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
> 1800)
> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
> 1795)
> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:
> 1791)
> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2398)
> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:
> 1465)
> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:
> 1424)
> at
> org
> .apache
> .solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:
> 278)
>
>
> CheckIndex reports these cores as being completely healthy, and yet
> I can't
> commit new documents in to them.
>
> Rebuilding indices isn't an option for me: is there any other way to
> fix
> this? If not, any ideas on what I can do to prevent it in the future?
>
> Many thanks,
> James