You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Alexander L (Jira)" <ji...@apache.org> on 2021/03/24 23:12:00 UTC
[jira] [Created] (LUCENE-9867) CorruptIndexException after failed
segment merge caused by No space left on device
Alexander L created LUCENE-9867:
-----------------------------------
Summary: CorruptIndexException after failed segment merge caused by No space left on device
Key: LUCENE-9867
URL: https://issues.apache.org/jira/browse/LUCENE-9867
Project: Lucene - Core
Issue Type: Bug
Components: core/store
Affects Versions: 8.5
Reporter: Alexander L
A failed segment merge caused by "No space left on device" can't be recovered and Lucene fails with CorruptIndexException after restart. The expectation is that Lucene will be able to restart automatically without manual intervention.
We have 2 indexing patterns:
* Create and commit an empty index, then start long initial indexing process (might take hours), perform a second commit in the end
* Using existing index, add no more than 4k documents and commit after that
Lucene version: 8.5.0
Java version: OpenJDK 11
OS: CentOS Linux 7
Kernel: Linux 3.10.0-1160.11.1.el7.x86_64
Virtualization: kvm
Filesystem: xfs
Failed merge stacktrace:
{code:java}
2021-02-02T08:51:51.679+0000org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException: No space left on device at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)Caused by: java.io.IOException: No space left on device at java.base/sun.nio.ch.FileDispatcherImpl.write0(Native Method) at java.base/sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:62) at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113) at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:79) at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280) at java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74) at java.base/java.nio.channels.Channels.writeFully(Channels.java:97) at java.base/java.nio.channels.Channels$1.write(Channels.java:172) at org.apache.lucene.store.FSDirectory$FSIndexOutput$1.write(FSDirectory.java:416) at java.base/java.util.zip.CheckedOutputStream.write(CheckedOutputStream.java:74) at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81) at java.base/java.io.BufferedOutputStream.write(BufferedOutputStream.java:127) at org.apache.lucene.store.OutputStreamIndexOutput.writeBytes(OutputStreamIndexOutput.java:53) at org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73) at org.apache.lucene.util.compress.LZ4.encodeLiterals(LZ4.java:159) at org.apache.lucene.util.compress.LZ4.encodeSequence(LZ4.java:172) at org.apache.lucene.util.compress.LZ4.compress(LZ4.java:441) at org.apache.lucene.codecs.compressing.CompressionMode$LZ4FastCompressor.compress(CompressionMode.java:165) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.flush(CompressingStoredFieldsWriter.java:229) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finishDocument(CompressingStoredFieldsWriter.java:159) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:636) at org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:229) at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:106) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4463) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4057) at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
{code}
Followed by failed startup:
{code:java}
2021-02-02T08:51:51.679+0000org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException: No space left on device at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)Caused by: java.io.IOException: No space left on device at java.base/sun.nio.ch.FileDispatcherImpl.write0(Native Method) at java.base/sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:62) at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113) at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:79) at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280) at java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74) at java.base/java.nio.channels.Channels.writeFully(Channels.java:97) at java.base/java.nio.channels.Channels$1.write(Channels.java:172) at org.apache.lucene.store.FSDirectory$FSIndexOutput$1.write(FSDirectory.java:416) at java.base/java.util.zip.CheckedOutputStream.write(CheckedOutputStream.java:74) at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81) at java.base/java.io.BufferedOutputStream.write(BufferedOutputStream.java:127) at org.apache.lucene.store.OutputStreamIndexOutput.writeBytes(OutputStreamIndexOutput.java:53) at org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73) at org.apache.lucene.util.compress.LZ4.encodeLiterals(LZ4.java:159) at org.apache.lucene.util.compress.LZ4.encodeSequence(LZ4.java:172) at org.apache.lucene.util.compress.LZ4.compress(LZ4.java:441) at org.apache.lucene.codecs.compressing.CompressionMode$LZ4FastCompressor.compress(CompressionMode.java:165) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.flush(CompressingStoredFieldsWriter.java:229) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finishDocument(CompressingStoredFieldsWriter.java:159) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:636) at org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:229) at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:106) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4463) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4057) at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org