You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by T Vinod Gupta <tv...@readypulse.com> on 2012/07/08 21:58:16 UTC
index.merge.scheduler exception - java.io.IOException: Input/output error
Hi,
My log files are showing the below exceptions almost at twice a minute
frequency. what is causing it and how can i fix it? I am not using lucene
directly but instead using elasticsearch (0.18.7 version). but since the
stack trace is all lucene, i am sending it to this mailing list.
also, my queries are taking a long time to execute (sometimes take a
minute). could this be contributing to it somehow?
[2012-07-08 19:44:19,887][WARN ][index.merge.scheduler ] [<my node
name>] [twitter][4] failed to merge
java.io.IOException: Input/output error:
NIOFSIndexInput(path="/media/ephemeral0
/ES_data/elasticsearch/nodes/0/indices/twitter/4/index/_2h29k.tis")
at
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(N
IOFSDirectory.java:180)
at
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.
java:229)
at
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
at
org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:197)
at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:86)
at
org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:133)
at
org.apache.lucene.index.SegmentMergeInfo.next(SegmentMergeInfo.java:72)
at
org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:546)
at
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:473)
at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4295)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3940)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
at
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
Caused by: java.io.IOException: Input/output error
at sun.nio.ch.FileDispatcher.pread0(Native Method)
at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:49)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:248)
at sun.nio.ch.IOUtil.read(IOUtil.java:224)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:663)
at
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:162)
... 15 more
thanks
Re: index.merge.scheduler exception - java.io.IOException:
Input/output error
Posted by Ian Lea <ia...@gmail.com>.
If you can live with the loss of 385395 documents, running with -fix
is an option. I'd create a new index. I'd also worry about why the
existing index got messed up in the first place.
I've no idea about running fsck on ec2 file systems. General file
system commands hanging for 10 secs doesn't sound good - I'd worry
about that first of all.
--
Ian.
On Mon, Jul 9, 2012 at 11:17 PM, T Vinod Gupta <tv...@readypulse.com> wrote:
> thanks this was really helpful to understand whats going on..
> i got these for 2 of my indexes -
>
> WARNING: 29 broken segments (containing 385395 documents) detected
> WARNING: would write new segments file, and 385395 documents would be lost,
> if -fix were specified
>
> WARNING: 29 broken segments (containing 385395 documents) detected
> WARNING: would write new segments file, and 385395 documents would be lost,
> if -fix were specified
>
> so my only option is to run with -fix and live with the data loss right? no
> other option right?
> will running fsck help? sometimes my ls or less commands also hang for a
> good 10 sec.. this somehow indicates that there is some corruption.
>
> thanks
>
> On Mon, Jul 9, 2012 at 6:27 AM, Erick Erickson <er...@gmail.com>wrote:
>
>> no, you can't delete those files, and you can't regenerate just those
>> files,
>> all the various segment files are necessary and intertwined...
>>
>> Consider using the CheckIndex facility, see:
>> http://solr.pl/en/2011/01/17/checkindex-for-the-rescue/
>>
>> note, the CheckIndex class is contained in the lucene core jar....
>>
>> You can run it with the -fix option to repair (at, perhaps, the expense
>> of loss of some documents) if you choose, but running it without
>> that option first is probably a good idea..
>>
>> Best
>> Erick
>>
>> On Mon, Jul 9, 2012 at 7:43 AM, T Vinod Gupta <tv...@readypulse.com>
>> wrote:
>> > this is on local file system on amazon ec2 host. the file system was fine
>> > until a week ago when the outage happened and there were probably some
>> > system glitches. i have seen this issue since then.. sometimes regular
>> > commands like less or ls hang for many seconds even though there is no
>> > cpu/memory pressure on the machine.
>> >
>> > in my case, there are only 2 unique entries for which i see this error.
>> one
>> > for a .fdt file and one for a .tis file. is it possible to regenerate
>> those
>> > files somehow? if i delete those 2 files, will the entire index get
>> > corrupted? im ok to live with some data loss if it makes it more stable
>> and
>> > performant.
>> >
>> > thanks
>> >
>> > On Mon, Jul 9, 2012 at 2:28 AM, Ian Lea <ia...@gmail.com> wrote:
>> >
>> >> Is this on a local or remote file system? Is the file system itself
>> >> OK? Is something else messing with your lucene index at the same
>> >> time?
>> >>
>> >>
>> >> --
>> >> Ian.
>> >>
>> >>
>> >> On Sun, Jul 8, 2012 at 8:58 PM, T Vinod Gupta <tv...@readypulse.com>
>> >> wrote:
>> >> > Hi,
>> >> > My log files are showing the below exceptions almost at twice a minute
>> >> > frequency. what is causing it and how can i fix it? I am not using
>> lucene
>> >> > directly but instead using elasticsearch (0.18.7 version). but since
>> the
>> >> > stack trace is all lucene, i am sending it to this mailing list.
>> >> >
>> >> > also, my queries are taking a long time to execute (sometimes take a
>> >> > minute). could this be contributing to it somehow?
>> >> >
>> >> > [2012-07-08 19:44:19,887][WARN ][index.merge.scheduler ] [<my node
>> >> > name>] [twitter][4] failed to merge
>> >> > java.io.IOException: Input/output error:
>> >> > NIOFSIndexInput(path="/media/ephemeral0
>> >> > /ES_data/elasticsearch/nodes/0/indices/twitter/4/index/_2h29k.tis")
>> >> > at
>> >> > org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(N
>> >> > IOFSDirectory.java:180)
>> >> > at
>> >> > org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.
>> >> > java:229)
>> >> > at
>> >> >
>> >>
>> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
>> >> > at
>> org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
>> >> > at
>> >> >
>> >>
>> org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:197)
>> >> > at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:86)
>> >> > at
>> >> > org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:133)
>> >> > at
>> >> >
>> org.apache.lucene.index.SegmentMergeInfo.next(SegmentMergeInfo.java:72)
>> >> > at
>> >> >
>> >>
>> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:546)
>> >> > at
>> >> >
>> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:473)
>> >> > at
>> >> > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
>> >> > at
>> >> > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4295)
>> >> > at
>> >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3940)
>> >> > at
>> >> >
>> >>
>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
>> >> > at
>> >> >
>> >>
>> org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
>> >> > at
>> >> >
>> >>
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
>> >> > Caused by: java.io.IOException: Input/output error
>> >> > at sun.nio.ch.FileDispatcher.pread0(Native Method)
>> >> > at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:49)
>> >> > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:248)
>> >> > at sun.nio.ch.IOUtil.read(IOUtil.java:224)
>> >> > at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:663)
>> >> > at
>> >> >
>> >>
>> org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:162)
>> >> > ... 15 more
>> >> >
>> >> > thanks
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: index.merge.scheduler exception - java.io.IOException:
Input/output error
Posted by T Vinod Gupta <tv...@readypulse.com>.
thanks this was really helpful to understand whats going on..
i got these for 2 of my indexes -
WARNING: 29 broken segments (containing 385395 documents) detected
WARNING: would write new segments file, and 385395 documents would be lost,
if -fix were specified
WARNING: 29 broken segments (containing 385395 documents) detected
WARNING: would write new segments file, and 385395 documents would be lost,
if -fix were specified
so my only option is to run with -fix and live with the data loss right? no
other option right?
will running fsck help? sometimes my ls or less commands also hang for a
good 10 sec.. this somehow indicates that there is some corruption.
thanks
On Mon, Jul 9, 2012 at 6:27 AM, Erick Erickson <er...@gmail.com>wrote:
> no, you can't delete those files, and you can't regenerate just those
> files,
> all the various segment files are necessary and intertwined...
>
> Consider using the CheckIndex facility, see:
> http://solr.pl/en/2011/01/17/checkindex-for-the-rescue/
>
> note, the CheckIndex class is contained in the lucene core jar....
>
> You can run it with the -fix option to repair (at, perhaps, the expense
> of loss of some documents) if you choose, but running it without
> that option first is probably a good idea..
>
> Best
> Erick
>
> On Mon, Jul 9, 2012 at 7:43 AM, T Vinod Gupta <tv...@readypulse.com>
> wrote:
> > this is on local file system on amazon ec2 host. the file system was fine
> > until a week ago when the outage happened and there were probably some
> > system glitches. i have seen this issue since then.. sometimes regular
> > commands like less or ls hang for many seconds even though there is no
> > cpu/memory pressure on the machine.
> >
> > in my case, there are only 2 unique entries for which i see this error.
> one
> > for a .fdt file and one for a .tis file. is it possible to regenerate
> those
> > files somehow? if i delete those 2 files, will the entire index get
> > corrupted? im ok to live with some data loss if it makes it more stable
> and
> > performant.
> >
> > thanks
> >
> > On Mon, Jul 9, 2012 at 2:28 AM, Ian Lea <ia...@gmail.com> wrote:
> >
> >> Is this on a local or remote file system? Is the file system itself
> >> OK? Is something else messing with your lucene index at the same
> >> time?
> >>
> >>
> >> --
> >> Ian.
> >>
> >>
> >> On Sun, Jul 8, 2012 at 8:58 PM, T Vinod Gupta <tv...@readypulse.com>
> >> wrote:
> >> > Hi,
> >> > My log files are showing the below exceptions almost at twice a minute
> >> > frequency. what is causing it and how can i fix it? I am not using
> lucene
> >> > directly but instead using elasticsearch (0.18.7 version). but since
> the
> >> > stack trace is all lucene, i am sending it to this mailing list.
> >> >
> >> > also, my queries are taking a long time to execute (sometimes take a
> >> > minute). could this be contributing to it somehow?
> >> >
> >> > [2012-07-08 19:44:19,887][WARN ][index.merge.scheduler ] [<my node
> >> > name>] [twitter][4] failed to merge
> >> > java.io.IOException: Input/output error:
> >> > NIOFSIndexInput(path="/media/ephemeral0
> >> > /ES_data/elasticsearch/nodes/0/indices/twitter/4/index/_2h29k.tis")
> >> > at
> >> > org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(N
> >> > IOFSDirectory.java:180)
> >> > at
> >> > org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.
> >> > java:229)
> >> > at
> >> >
> >>
> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
> >> > at
> org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
> >> > at
> >> >
> >>
> org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:197)
> >> > at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:86)
> >> > at
> >> > org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:133)
> >> > at
> >> >
> org.apache.lucene.index.SegmentMergeInfo.next(SegmentMergeInfo.java:72)
> >> > at
> >> >
> >>
> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:546)
> >> > at
> >> >
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:473)
> >> > at
> >> > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
> >> > at
> >> > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4295)
> >> > at
> >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3940)
> >> > at
> >> >
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
> >> > at
> >> >
> >>
> org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
> >> > at
> >> >
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
> >> > Caused by: java.io.IOException: Input/output error
> >> > at sun.nio.ch.FileDispatcher.pread0(Native Method)
> >> > at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:49)
> >> > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:248)
> >> > at sun.nio.ch.IOUtil.read(IOUtil.java:224)
> >> > at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:663)
> >> > at
> >> >
> >>
> org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:162)
> >> > ... 15 more
> >> >
> >> > thanks
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: index.merge.scheduler exception - java.io.IOException:
Input/output error
Posted by Erick Erickson <er...@gmail.com>.
no, you can't delete those files, and you can't regenerate just those files,
all the various segment files are necessary and intertwined...
Consider using the CheckIndex facility, see:
http://solr.pl/en/2011/01/17/checkindex-for-the-rescue/
note, the CheckIndex class is contained in the lucene core jar....
You can run it with the -fix option to repair (at, perhaps, the expense
of loss of some documents) if you choose, but running it without
that option first is probably a good idea..
Best
Erick
On Mon, Jul 9, 2012 at 7:43 AM, T Vinod Gupta <tv...@readypulse.com> wrote:
> this is on local file system on amazon ec2 host. the file system was fine
> until a week ago when the outage happened and there were probably some
> system glitches. i have seen this issue since then.. sometimes regular
> commands like less or ls hang for many seconds even though there is no
> cpu/memory pressure on the machine.
>
> in my case, there are only 2 unique entries for which i see this error. one
> for a .fdt file and one for a .tis file. is it possible to regenerate those
> files somehow? if i delete those 2 files, will the entire index get
> corrupted? im ok to live with some data loss if it makes it more stable and
> performant.
>
> thanks
>
> On Mon, Jul 9, 2012 at 2:28 AM, Ian Lea <ia...@gmail.com> wrote:
>
>> Is this on a local or remote file system? Is the file system itself
>> OK? Is something else messing with your lucene index at the same
>> time?
>>
>>
>> --
>> Ian.
>>
>>
>> On Sun, Jul 8, 2012 at 8:58 PM, T Vinod Gupta <tv...@readypulse.com>
>> wrote:
>> > Hi,
>> > My log files are showing the below exceptions almost at twice a minute
>> > frequency. what is causing it and how can i fix it? I am not using lucene
>> > directly but instead using elasticsearch (0.18.7 version). but since the
>> > stack trace is all lucene, i am sending it to this mailing list.
>> >
>> > also, my queries are taking a long time to execute (sometimes take a
>> > minute). could this be contributing to it somehow?
>> >
>> > [2012-07-08 19:44:19,887][WARN ][index.merge.scheduler ] [<my node
>> > name>] [twitter][4] failed to merge
>> > java.io.IOException: Input/output error:
>> > NIOFSIndexInput(path="/media/ephemeral0
>> > /ES_data/elasticsearch/nodes/0/indices/twitter/4/index/_2h29k.tis")
>> > at
>> > org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(N
>> > IOFSDirectory.java:180)
>> > at
>> > org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.
>> > java:229)
>> > at
>> >
>> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
>> > at org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
>> > at
>> >
>> org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:197)
>> > at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:86)
>> > at
>> > org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:133)
>> > at
>> > org.apache.lucene.index.SegmentMergeInfo.next(SegmentMergeInfo.java:72)
>> > at
>> >
>> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:546)
>> > at
>> > org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:473)
>> > at
>> > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
>> > at
>> > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4295)
>> > at
>> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3940)
>> > at
>> >
>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
>> > at
>> >
>> org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
>> > at
>> >
>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
>> > Caused by: java.io.IOException: Input/output error
>> > at sun.nio.ch.FileDispatcher.pread0(Native Method)
>> > at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:49)
>> > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:248)
>> > at sun.nio.ch.IOUtil.read(IOUtil.java:224)
>> > at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:663)
>> > at
>> >
>> org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:162)
>> > ... 15 more
>> >
>> > thanks
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: index.merge.scheduler exception - java.io.IOException:
Input/output error
Posted by T Vinod Gupta <tv...@readypulse.com>.
this is on local file system on amazon ec2 host. the file system was fine
until a week ago when the outage happened and there were probably some
system glitches. i have seen this issue since then.. sometimes regular
commands like less or ls hang for many seconds even though there is no
cpu/memory pressure on the machine.
in my case, there are only 2 unique entries for which i see this error. one
for a .fdt file and one for a .tis file. is it possible to regenerate those
files somehow? if i delete those 2 files, will the entire index get
corrupted? im ok to live with some data loss if it makes it more stable and
performant.
thanks
On Mon, Jul 9, 2012 at 2:28 AM, Ian Lea <ia...@gmail.com> wrote:
> Is this on a local or remote file system? Is the file system itself
> OK? Is something else messing with your lucene index at the same
> time?
>
>
> --
> Ian.
>
>
> On Sun, Jul 8, 2012 at 8:58 PM, T Vinod Gupta <tv...@readypulse.com>
> wrote:
> > Hi,
> > My log files are showing the below exceptions almost at twice a minute
> > frequency. what is causing it and how can i fix it? I am not using lucene
> > directly but instead using elasticsearch (0.18.7 version). but since the
> > stack trace is all lucene, i am sending it to this mailing list.
> >
> > also, my queries are taking a long time to execute (sometimes take a
> > minute). could this be contributing to it somehow?
> >
> > [2012-07-08 19:44:19,887][WARN ][index.merge.scheduler ] [<my node
> > name>] [twitter][4] failed to merge
> > java.io.IOException: Input/output error:
> > NIOFSIndexInput(path="/media/ephemeral0
> > /ES_data/elasticsearch/nodes/0/indices/twitter/4/index/_2h29k.tis")
> > at
> > org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(N
> > IOFSDirectory.java:180)
> > at
> > org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.
> > java:229)
> > at
> >
> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
> > at org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
> > at
> >
> org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:197)
> > at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:86)
> > at
> > org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:133)
> > at
> > org.apache.lucene.index.SegmentMergeInfo.next(SegmentMergeInfo.java:72)
> > at
> >
> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:546)
> > at
> > org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:473)
> > at
> > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
> > at
> > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4295)
> > at
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3940)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
> > at
> >
> org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
> > at
> >
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
> > Caused by: java.io.IOException: Input/output error
> > at sun.nio.ch.FileDispatcher.pread0(Native Method)
> > at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:49)
> > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:248)
> > at sun.nio.ch.IOUtil.read(IOUtil.java:224)
> > at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:663)
> > at
> >
> org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:162)
> > ... 15 more
> >
> > thanks
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: index.merge.scheduler exception - java.io.IOException:
Input/output error
Posted by Ian Lea <ia...@gmail.com>.
Is this on a local or remote file system? Is the file system itself
OK? Is something else messing with your lucene index at the same
time?
--
Ian.
On Sun, Jul 8, 2012 at 8:58 PM, T Vinod Gupta <tv...@readypulse.com> wrote:
> Hi,
> My log files are showing the below exceptions almost at twice a minute
> frequency. what is causing it and how can i fix it? I am not using lucene
> directly but instead using elasticsearch (0.18.7 version). but since the
> stack trace is all lucene, i am sending it to this mailing list.
>
> also, my queries are taking a long time to execute (sometimes take a
> minute). could this be contributing to it somehow?
>
> [2012-07-08 19:44:19,887][WARN ][index.merge.scheduler ] [<my node
> name>] [twitter][4] failed to merge
> java.io.IOException: Input/output error:
> NIOFSIndexInput(path="/media/ephemeral0
> /ES_data/elasticsearch/nodes/0/indices/twitter/4/index/_2h29k.tis")
> at
> org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(N
> IOFSDirectory.java:180)
> at
> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.
> java:229)
> at
> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:105)
> at
> org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:197)
> at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:86)
> at
> org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:133)
> at
> org.apache.lucene.index.SegmentMergeInfo.next(SegmentMergeInfo.java:72)
> at
> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:546)
> at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:473)
> at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4295)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3940)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
> at
> org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
> Caused by: java.io.IOException: Input/output error
> at sun.nio.ch.FileDispatcher.pread0(Native Method)
> at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:49)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:248)
> at sun.nio.ch.IOUtil.read(IOUtil.java:224)
> at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:663)
> at
> org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:162)
> ... 15 more
>
> thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org