You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Chris Gioran <ch...@neotechnology.com> on 2012/06/19 16:50:38 UTC
zero sized cfs files in index lead to IOException: read past EOF
Hello everyone,
I am having a problem with a lucene store. When starting an
IndexWriter on it, it throws the following exception:
Caused by: java.io.IOException: read past EOF:
MMapIndexInput(path="/path/to/index/_drs.cfs")
at org.apache.lucene.store.MMapDirectory$MMapIndexInput.readByte(MMapDirectory.java:279)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:69)
at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:53)
at org.apache.lucene.index.IndexWriter.getFieldInfos(IndexWriter.java:1218)
at org.apache.lucene.index.IndexWriter.getCurrentFieldInfos(IndexWriter.java:1238)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1171)
The store is weird in that it has 3, 0-sized cfs files, which of
course is the reason for the exception above. Here is the file
listing:
-rw-r--r-- 1 chris chris 36 Jun 6 16:42 _drr_1.del
-rw-r--r-- 1 chris chris 47794 Jun 5 21:15 _drr.fdt
-rw-r--r-- 1 chris chris 6476 Jun 5 21:15 _drr.fdx
-rw-r--r-- 1 chris chris 23 Jun 5 21:15 _drr.fnm
-rw-r--r-- 1 chris chris 2539 Jun 5 21:15 _drr.frq
-rw-r--r-- 1 chris chris 1622 Jun 5 21:15 _drr.nrm
-rw-r--r-- 1 chris chris 1618 Jun 5 21:15 _drr.prx
-rw-r--r-- 1 chris chris 115 Jun 5 21:15 _drr.tii
-rw-r--r-- 1 chris chris 6360 Jun 5 21:15 _drr.tis
-rw-r--r-- 1 chris chris 0 Jun 6 16:42 _drs.cfs
-rw-r--r-- 1 chris chris 0 Jun 6 16:42 _drt.cfs
-rw-r--r-- 1 chris chris 0 Jun 6 16:42 _drx.cfs
-rw-r--r-- 1 chris chris 974 Jun 6 16:42 segments_12
-rw-r--r-- 1 chris chris 20 Jun 6 16:42 segments.gen
The store was delivered to me like this with no steps for
reproduction, but it has been observed in two separate indexes (not
the same files of course, but the same symptoms as above).
It was running on a Linux 2.6.18 amd64 with Lucene version 3.5 on
HotSpot 1.6.0_22
My question to you is how can a Lucene store reach this state? Is it
possible to happen because of a crash while merging or something
similar? How easy will it be to recover the data and how can i reduce
the probability of this happening in the future?
Thank you very much for you time,
CG
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: zero sized cfs files in index lead to IOException: read past EOF
Posted by Michael McCandless <lu...@mikemccandless.com>.
Hmm which Lucene version are you using? For 3.x before 3.4, there was
a bug (https://issues.apache.org/jira/browse/LUCENE-3418) where we
failed to actually fsync...
More below:
On Tue, Jun 19, 2012 at 4:54 PM, Chris Gioran
<ch...@neotechnology.com> wrote:
> On Tue, Jun 19, 2012 at 6:18 PM, Michael McCandless
> <lu...@mikemccandless.com> wrote:
>> This shouldn't normally happen, even on crash, kill -9, power loss, etc.
>>
>> It can only mean either there is a bug in Lucene, or there's something
>> wrong with your hardware/IO system, or the fsync operation doesn't
>> actually work on the IO system.
>
> I have seen two instances of this, both on the same machine. It isn't
> far fetched to assume a hardware issue but no other problems have been
> reported there.
OK.
>> You can run CheckIndex to see what's broken (then, add -fix to remove
>> the broken segments)...
>
> Thank you for this information, will get back if something interesting comes up.
>
>> What IO system are you using? Is this reproducible? Are you sure
>> there wasn't a problem in transferring the index from its source to
>> you....?
>
> Commodity SATA disks on RHEL 5.
> No, just happened twice, no clear pattern.
> Yes, the the exception happened on site and afterwords the store was
> given to me - everything in there works but that index.
Those cfs files were also 0 length in the original index?
> Thank you for your response, i'll get back if i have more information.
Thanks.
Mike McCandless
http://blog.mikemccandless.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: zero sized cfs files in index lead to IOException: read past EOF
Posted by Chris Gioran <ch...@neotechnology.com>.
On Tue, Jun 19, 2012 at 6:18 PM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> This shouldn't normally happen, even on crash, kill -9, power loss, etc.
>
> It can only mean either there is a bug in Lucene, or there's something
> wrong with your hardware/IO system, or the fsync operation doesn't
> actually work on the IO system.
I have seen two instances of this, both on the same machine. It isn't
far fetched to assume a hardware issue but no other problems have been
reported there.
> You can run CheckIndex to see what's broken (then, add -fix to remove
> the broken segments)...
Thank you for this information, will get back if something interesting comes up.
> What IO system are you using? Is this reproducible? Are you sure
> there wasn't a problem in transferring the index from its source to
> you....?
Commodity SATA disks on RHEL 5.
No, just happened twice, no clear pattern.
Yes, the the exception happened on site and afterwords the store was
given to me - everything in there works but that index.
Thank you for your response, i'll get back if i have more information.
CG
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Tue, Jun 19, 2012 at 10:50 AM, Chris Gioran
> <ch...@neotechnology.com> wrote:
>> Hello everyone,
>>
>> I am having a problem with a lucene store. When starting an
>> IndexWriter on it, it throws the following exception:
>>
>> Caused by: java.io.IOException: read past EOF:
>> MMapIndexInput(path="/path/to/index/_drs.cfs")
>> at org.apache.lucene.store.MMapDirectory$MMapIndexInput.readByte(MMapDirectory.java:279)
>> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
>> at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:69)
>> at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:53)
>> at org.apache.lucene.index.IndexWriter.getFieldInfos(IndexWriter.java:1218)
>> at org.apache.lucene.index.IndexWriter.getCurrentFieldInfos(IndexWriter.java:1238)
>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1171)
>>
>> The store is weird in that it has 3, 0-sized cfs files, which of
>> course is the reason for the exception above. Here is the file
>> listing:
>>
>> -rw-r--r-- 1 chris chris 36 Jun 6 16:42 _drr_1.del
>> -rw-r--r-- 1 chris chris 47794 Jun 5 21:15 _drr.fdt
>> -rw-r--r-- 1 chris chris 6476 Jun 5 21:15 _drr.fdx
>> -rw-r--r-- 1 chris chris 23 Jun 5 21:15 _drr.fnm
>> -rw-r--r-- 1 chris chris 2539 Jun 5 21:15 _drr.frq
>> -rw-r--r-- 1 chris chris 1622 Jun 5 21:15 _drr.nrm
>> -rw-r--r-- 1 chris chris 1618 Jun 5 21:15 _drr.prx
>> -rw-r--r-- 1 chris chris 115 Jun 5 21:15 _drr.tii
>> -rw-r--r-- 1 chris chris 6360 Jun 5 21:15 _drr.tis
>> -rw-r--r-- 1 chris chris 0 Jun 6 16:42 _drs.cfs
>> -rw-r--r-- 1 chris chris 0 Jun 6 16:42 _drt.cfs
>> -rw-r--r-- 1 chris chris 0 Jun 6 16:42 _drx.cfs
>> -rw-r--r-- 1 chris chris 974 Jun 6 16:42 segments_12
>> -rw-r--r-- 1 chris chris 20 Jun 6 16:42 segments.gen
>>
>> The store was delivered to me like this with no steps for
>> reproduction, but it has been observed in two separate indexes (not
>> the same files of course, but the same symptoms as above).
>> It was running on a Linux 2.6.18 amd64 with Lucene version 3.5 on
>> HotSpot 1.6.0_22
>>
>> My question to you is how can a Lucene store reach this state? Is it
>> possible to happen because of a crash while merging or something
>> similar? How easy will it be to recover the data and how can i reduce
>> the probability of this happening in the future?
>>
>> Thank you very much for you time,
>> CG
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: zero sized cfs files in index lead to IOException: read past EOF
Posted by Michael McCandless <lu...@mikemccandless.com>.
This shouldn't normally happen, even on crash, kill -9, power loss, etc.
It can only mean either there is a bug in Lucene, or there's something
wrong with your hardware/IO system, or the fsync operation doesn't
actually work on the IO system.
You can run CheckIndex to see what's broken (then, add -fix to remove
the broken segments)...
What IO system are you using? Is this reproducible? Are you sure
there wasn't a problem in transferring the index from its source to
you....?
Mike McCandless
http://blog.mikemccandless.com
On Tue, Jun 19, 2012 at 10:50 AM, Chris Gioran
<ch...@neotechnology.com> wrote:
> Hello everyone,
>
> I am having a problem with a lucene store. When starting an
> IndexWriter on it, it throws the following exception:
>
> Caused by: java.io.IOException: read past EOF:
> MMapIndexInput(path="/path/to/index/_drs.cfs")
> at org.apache.lucene.store.MMapDirectory$MMapIndexInput.readByte(MMapDirectory.java:279)
> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
> at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:69)
> at org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:53)
> at org.apache.lucene.index.IndexWriter.getFieldInfos(IndexWriter.java:1218)
> at org.apache.lucene.index.IndexWriter.getCurrentFieldInfos(IndexWriter.java:1238)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1171)
>
> The store is weird in that it has 3, 0-sized cfs files, which of
> course is the reason for the exception above. Here is the file
> listing:
>
> -rw-r--r-- 1 chris chris 36 Jun 6 16:42 _drr_1.del
> -rw-r--r-- 1 chris chris 47794 Jun 5 21:15 _drr.fdt
> -rw-r--r-- 1 chris chris 6476 Jun 5 21:15 _drr.fdx
> -rw-r--r-- 1 chris chris 23 Jun 5 21:15 _drr.fnm
> -rw-r--r-- 1 chris chris 2539 Jun 5 21:15 _drr.frq
> -rw-r--r-- 1 chris chris 1622 Jun 5 21:15 _drr.nrm
> -rw-r--r-- 1 chris chris 1618 Jun 5 21:15 _drr.prx
> -rw-r--r-- 1 chris chris 115 Jun 5 21:15 _drr.tii
> -rw-r--r-- 1 chris chris 6360 Jun 5 21:15 _drr.tis
> -rw-r--r-- 1 chris chris 0 Jun 6 16:42 _drs.cfs
> -rw-r--r-- 1 chris chris 0 Jun 6 16:42 _drt.cfs
> -rw-r--r-- 1 chris chris 0 Jun 6 16:42 _drx.cfs
> -rw-r--r-- 1 chris chris 974 Jun 6 16:42 segments_12
> -rw-r--r-- 1 chris chris 20 Jun 6 16:42 segments.gen
>
> The store was delivered to me like this with no steps for
> reproduction, but it has been observed in two separate indexes (not
> the same files of course, but the same symptoms as above).
> It was running on a Linux 2.6.18 amd64 with Lucene version 3.5 on
> HotSpot 1.6.0_22
>
> My question to you is how can a Lucene store reach this state? Is it
> possible to happen because of a crash while merging or something
> similar? How easy will it be to recover the data and how can i reduce
> the probability of this happening in the future?
>
> Thank you very much for you time,
> CG
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org