You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ken Krugler <kk...@transpac.com> on 2010/07/01 22:03:50 UTC
IOException: read past EOF when opening index built directly w/Lucene
I've got a version 2.3 index that appears to be valid - I can open it
with Luke 1.0.1, and CheckIndex reports no problem.
Just for grins, I crafted a matching schema, and tried to use the
index with Solr 1.4 (and also Solr-trunk).
In either case, I get this exception during startup:
SEVERE: java.lang.RuntimeException: java.io.IOException: read past EOF
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1067)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:582)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:431)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:286)
at org.apache.solr.core.CoreContainer
$Initializer.initialize(CoreContainer.java:125)
at
org
.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:86)
...
Caused by: java.io.IOException: read past EOF
at
org
.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:
154)
at
org
.apache
.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
at
org
.apache
.lucene.store.ChecksumIndexInput.readByte(ChecksumIndexInput.java:40)
at org.apache.lucene.store.DataInput.readInt(DataInput.java:76)
at org.apache.lucene.index.SegmentInfo.<init>(SegmentInfo.java:171)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:230)
at org.apache.lucene.index.DirectoryReader
$1.doBody(DirectoryReader.java:91)
at org.apache.lucene.index.SegmentInfos
$FindSegmentsFile.run(SegmentInfos.java:649)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:
87)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:415)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:294)
at
org
.apache
.solr
.core
.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:
38)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1056)
... 30 more
and at the end of the startup logging, it says:
Jul 1, 2010 12:51:25 PM org.apache.solr.core.SolrCore finalize
SEVERE: REFCOUNT ERROR: unreferenced
org.apache.solr.core.SolrCore@4513e9fd () has a reference count of 1
Is what I'm trying to do something that's destined to fail? I would
have expected schema/index miss-matches to show up later, not right
when the index is being opened.
I'd seen various posts about this type of error due to corrupt
indexes, or having a buggy version of Java 1.6, or an obscure Lucene
bug (https://issues.apache.org/jira/browse/SOLR-1778 and https://issues.apache.org/jira/browse/LUCENE-2270)
, but none of those seem to apply to my situation.
Thanks,
-- Ken
PS - index dir looks like:
249K Jun 29 13:47 _0.fdt
12K Jun 29 13:47 _0.fdx
159B Jun 29 13:47 _0.fnm
3.6M Jun 29 13:47 _0.frq
23K Jun 29 13:47 _0.nrm
10M Jun 29 13:47 _0.prx
51K Jun 29 13:47 _0.tii
2.9M Jun 29 13:47 _0.tis
20B Jun 29 13:47 segments.gen
45B Jun 29 13:47 segments_2
and Luke overview says:
Re: IOException: read past EOF when opening index built directly
w/Lucene
Posted by Michael McCandless <lu...@mikemccandless.com>.
Indeed, I can reproduce this: if I create an index on 2.3 and try to
read it on trunk w/ CheckIndex, I hit that same exception. But: this
is [somewhat] expected, because trunk = 4.0, which can no longer read
indices created with Lucene <= 3.0. However... instead of throwing a
weird exception, we ought to throw some sort of IndexTooOldException,
I think. I'll open an issue.
But, when I try to check the 2.3 index with Lucene 2.9.x, 3.0.x, 3.x
(branch tips), it runs fine, which is counter to what you're seeing?
Ie Solr 1.4 uses Lucene 2.9.3, so I can't explain why you'd see this
exception on Solr 1.4...
Mike
On Thu, Jul 1, 2010 at 4:03 PM, Ken Krugler <kk...@transpac.com> wrote:
> I've got a version 2.3 index that appears to be valid - I can open it with
> Luke 1.0.1, and CheckIndex reports no problem.
>
> Just for grins, I crafted a matching schema, and tried to use the index with
> Solr 1.4 (and also Solr-trunk).
>
> In either case, I get this exception during startup:
>
> SEVERE: java.lang.RuntimeException: java.io.IOException: read past EOF
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1067)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:582)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:431)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:286)
> at
> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:125)
> at
> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:86)
> ...
> Caused by: java.io.IOException: read past EOF
> at
> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:154)
> at
> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
> at
> org.apache.lucene.store.ChecksumIndexInput.readByte(ChecksumIndexInput.java:40)
> at org.apache.lucene.store.DataInput.readInt(DataInput.java:76)
> at org.apache.lucene.index.SegmentInfo.<init>(SegmentInfo.java:171)
> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:230)
> at
> org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:91)
> at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:649)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:87)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:415)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:294)
> at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:38)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1056)
> ... 30 more
>
> and at the end of the startup logging, it says:
>
> Jul 1, 2010 12:51:25 PM org.apache.solr.core.SolrCore finalize
> SEVERE: REFCOUNT ERROR: unreferenced org.apache.solr.core.SolrCore@4513e9fd
> () has a reference count of 1
>
> Is what I'm trying to do something that's destined to fail? I would have
> expected schema/index miss-matches to show up later, not right when the
> index is being opened.
>
> I'd seen various posts about this type of error due to corrupt indexes, or
> having a buggy version of Java 1.6, or an obscure Lucene bug
> (https://issues.apache.org/jira/browse/SOLR-1778 and
> https://issues.apache.org/jira/browse/LUCENE-2270), but none of those seem
> to apply to my situation.
>
> Thanks,
>
> -- Ken
>
> PS - index dir looks like:
>
> 249K Jun 29 13:47 _0.fdt
> 12K Jun 29 13:47 _0.fdx
> 159B Jun 29 13:47 _0.fnm
> 3.6M Jun 29 13:47 _0.frq
> 23K Jun 29 13:47 _0.nrm
> 10M Jun 29 13:47 _0.prx
> 51K Jun 29 13:47 _0.tii
> 2.9M Jun 29 13:47 _0.tis
> 20B Jun 29 13:47 segments.gen
> 45B Jun 29 13:47 segments_2
>
> and Luke overview says:
>
>
>
>
>
> --------------------------------------------
> Ken Krugler
> +1 530-210-6378
> http://bixolabs.com
> e l a s t i c w e b m i n i n g
>
>
>
>
>
>
Re: IOException: read past EOF when opening index built directly w/Lucene
Posted by Ken Krugler <kk...@transpac.com>.
On Jul 1, 2010, at 1:03pm, Ken Krugler wrote:
> I've got a version 2.3 index that appears to be valid - I can open
> it with Luke 1.0.1, and CheckIndex reports no problem.
[snip]
> and Luke overview says:
This time as text:
Index version: 12984d2211c
Index format: -4 (Lucene 2.3)
Index functionality: lock-less, single norms file, shared doc store
Currently opened commit point: segments_2
-- Ken
--------------------------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
e l a s t i c w e b m i n i n g