You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Pradeep Pujari <Pr...@rocketmail.com> on 2010/08/16 23:41:49 UTC
Index corrupted after delta import using DIH
Hi All,
Any one see this error. After, delta import using DIH, no queries working. When I do a Full import again, everything starts working. I am using solr 1.4. Is there any bug in delta import in DIH?
Thanks,
Pradeep
HTTP Status 500 - Index: 49, Size: 11 java.lang.IndexOutOfBoundsException: Index: 49, Size: 11 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:264) at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:204) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:828) at org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:588) at org.apache.lucene.index.IndexReader.document(IndexReader.java:755) at org.apache.solr.search.SolrIndexReader.document(SolrIndexReader.java:454) at org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:427) at org.apache.solr.util.SolrPluginUtils.optimizePreFetchDocs(SolrPluginUtils.java:268) at org.apache.solr.handler.component.QueryComponent.doPrefetch(QueryComponent.java:287) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:186)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1323) at
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Index corrupted after delta import using DIH
Posted by Michael McCandless <lu...@mikemccandless.com>.
This is the issue:
https://issues.apache.org/jira/browse/LUCENE-2574
It was a low-level optimization on how we copy bytes between files.
The first commit (on Jul 30) introduced the corruption bug, and then
it took a few days (until Aug 4) to find & fix it. Any index built
using Lucene's trunk or 3.x during that time period could be corrupt
and needs to be fully reindexed.
Mike
On Tue, Aug 17, 2010 at 7:02 PM, <ka...@nokia.com> wrote:
> There is no patch because the fixes were committed directly to trunk.
>
> Also, the corrupted indexes will need to be rebuilt. You do not, however, need to "delete and reinstall" everything, just the Lucene jars.
>
> Karl
>
>
> -----Original Message-----
> From: ext Pradeep Pujari [mailto:Pradeepp@rocketmail.com]
> Sent: Tuesday, August 17, 2010 6:58 PM
> To: dev@lucene.apache.org
> Subject: RE: Index corrupted after delta import using DIH
>
> Hi Karl,
>
> Can you please help me JIRA- number or patch? How to get the patch files? I do not want to delete and reinstall everything in my dev box.
>
> Thank you in advance.
> Pradeep.
>
> --- On Mon, 8/16/10, karl.wright@nokia.com <ka...@nokia.com> wrote:
>
>> From: karl.wright@nokia.com <ka...@nokia.com>
>> Subject: RE: Index corrupted after delta import using DIH
>> To: dev@lucene.apache.org
>> Date: Monday, August 16, 2010, 4:52 PM
>> Yes, this was addressed by a set of
>> fixes from Mike McCandless back in early August.
>>
>> Karl
>>
>> ________________________________________
>> From: ext Pradeep Pujari [Pradeepp@rocketmail.com]
>> Sent: Monday, August 16, 2010 5:41 PM
>> To: dev@lucene.apache.org
>> Subject: Index corrupted after delta import using DIH
>>
>> Hi All,
>> Any one see this error. After, delta import using DIH, no
>> queries working. When I do a Full import again, everything
>> starts working. I am using solr 1.4. Is there any bug in
>> delta import in DIH?
>>
>> Thanks,
>> Pradeep
>>
>>
>> HTTP Status 500 - Index: 49, Size: 11
>> java.lang.IndexOutOfBoundsException: Index: 49, Size: 11 at
>> java.util.ArrayList.RangeCheck(ArrayList.java:547) at
>> java.util.ArrayList.get(ArrayList.java:322) at
>> org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:264)
>> at
>> org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:204)
>> at
>> org.apache.lucene.index.SegmentReader.document(SegmentReader.java:828)
>> at
>> org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:588)
>> at
>> org.apache.lucene.index.IndexReader.document(IndexReader.java:755)
>> at
>> org.apache.solr.search.SolrIndexReader.document(SolrIndexReader.java:454)
>> at
>> org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:427)
>> at
>> org.apache.solr.util.SolrPluginUtils.optimizePreFetchDocs(SolrPluginUtils.java:268)
>> at
>> org.apache.solr.handler.component.QueryComponent.doPrefetch(QueryComponent.java:287)
>> at
>> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:186)
>> at
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
>> at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1323)
>> at
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: Index corrupted after delta import using DIH
Posted by ka...@nokia.com.
There is no patch because the fixes were committed directly to trunk.
Also, the corrupted indexes will need to be rebuilt. You do not, however, need to "delete and reinstall" everything, just the Lucene jars.
Karl
-----Original Message-----
From: ext Pradeep Pujari [mailto:Pradeepp@rocketmail.com]
Sent: Tuesday, August 17, 2010 6:58 PM
To: dev@lucene.apache.org
Subject: RE: Index corrupted after delta import using DIH
Hi Karl,
Can you please help me JIRA- number or patch? How to get the patch files? I do not want to delete and reinstall everything in my dev box.
Thank you in advance.
Pradeep.
--- On Mon, 8/16/10, karl.wright@nokia.com <ka...@nokia.com> wrote:
> From: karl.wright@nokia.com <ka...@nokia.com>
> Subject: RE: Index corrupted after delta import using DIH
> To: dev@lucene.apache.org
> Date: Monday, August 16, 2010, 4:52 PM
> Yes, this was addressed by a set of
> fixes from Mike McCandless back in early August.
>
> Karl
>
> ________________________________________
> From: ext Pradeep Pujari [Pradeepp@rocketmail.com]
> Sent: Monday, August 16, 2010 5:41 PM
> To: dev@lucene.apache.org
> Subject: Index corrupted after delta import using DIH
>
> Hi All,
> Any one see this error. After, delta import using DIH, no
> queries working. When I do a Full import again, everything
> starts working. I am using solr 1.4. Is there any bug in
> delta import in DIH?
>
> Thanks,
> Pradeep
>
>
> HTTP Status 500 - Index: 49, Size: 11
> java.lang.IndexOutOfBoundsException: Index: 49, Size: 11 at
> java.util.ArrayList.RangeCheck(ArrayList.java:547) at
> java.util.ArrayList.get(ArrayList.java:322) at
> org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:264)
> at
> org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:204)
> at
> org.apache.lucene.index.SegmentReader.document(SegmentReader.java:828)
> at
> org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:588)
> at
> org.apache.lucene.index.IndexReader.document(IndexReader.java:755)
> at
> org.apache.solr.search.SolrIndexReader.document(SolrIndexReader.java:454)
> at
> org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:427)
> at
> org.apache.solr.util.SolrPluginUtils.optimizePreFetchDocs(SolrPluginUtils.java:268)
> at
> org.apache.solr.handler.component.QueryComponent.doPrefetch(QueryComponent.java:287)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:186)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1323)
> at
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: Index corrupted after delta import using DIH
Posted by Pradeep Pujari <Pr...@rocketmail.com>.
Hi Karl,
Can you please help me JIRA- number or patch? How to get the patch files? I do not want to delete and reinstall everything in my dev box.
Thank you in advance.
Pradeep.
--- On Mon, 8/16/10, karl.wright@nokia.com <ka...@nokia.com> wrote:
> From: karl.wright@nokia.com <ka...@nokia.com>
> Subject: RE: Index corrupted after delta import using DIH
> To: dev@lucene.apache.org
> Date: Monday, August 16, 2010, 4:52 PM
> Yes, this was addressed by a set of
> fixes from Mike McCandless back in early August.
>
> Karl
>
> ________________________________________
> From: ext Pradeep Pujari [Pradeepp@rocketmail.com]
> Sent: Monday, August 16, 2010 5:41 PM
> To: dev@lucene.apache.org
> Subject: Index corrupted after delta import using DIH
>
> Hi All,
> Any one see this error. After, delta import using DIH, no
> queries working. When I do a Full import again, everything
> starts working. I am using solr 1.4. Is there any bug in
> delta import in DIH?
>
> Thanks,
> Pradeep
>
>
> HTTP Status 500 - Index: 49, Size: 11
> java.lang.IndexOutOfBoundsException: Index: 49, Size: 11 at
> java.util.ArrayList.RangeCheck(ArrayList.java:547) at
> java.util.ArrayList.get(ArrayList.java:322) at
> org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:264)
> at
> org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:204)
> at
> org.apache.lucene.index.SegmentReader.document(SegmentReader.java:828)
> at
> org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:588)
> at
> org.apache.lucene.index.IndexReader.document(IndexReader.java:755)
> at
> org.apache.solr.search.SolrIndexReader.document(SolrIndexReader.java:454)
> at
> org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:427)
> at
> org.apache.solr.util.SolrPluginUtils.optimizePreFetchDocs(SolrPluginUtils.java:268)
> at
> org.apache.solr.handler.component.QueryComponent.doPrefetch(QueryComponent.java:287)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:186)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1323)
> at
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: Index corrupted after delta import using DIH
Posted by ka...@nokia.com.
Yes, this was addressed by a set of fixes from Mike McCandless back in early August.
Karl
________________________________________
From: ext Pradeep Pujari [Pradeepp@rocketmail.com]
Sent: Monday, August 16, 2010 5:41 PM
To: dev@lucene.apache.org
Subject: Index corrupted after delta import using DIH
Hi All,
Any one see this error. After, delta import using DIH, no queries working. When I do a Full import again, everything starts working. I am using solr 1.4. Is there any bug in delta import in DIH?
Thanks,
Pradeep
HTTP Status 500 - Index: 49, Size: 11 java.lang.IndexOutOfBoundsException: Index: 49, Size: 11 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:264) at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:204) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:828) at org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:588) at org.apache.lucene.index.IndexReader.document(IndexReader.java:755) at org.apache.solr.search.SolrIndexReader.document(SolrIndexReader.java:454) at org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:427) at org.apache.solr.util.SolrPluginUtils.optimizePreFetchDocs(SolrPluginUtils.java:268) at org.apache.solr.handler.component.QueryComponent.doPrefetch(QueryComponent.java:287) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:186)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1323) at
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org