You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (JIRA)" <ji...@apache.org> on 2014/05/11 00:06:49 UTC
[jira] [Updated] (LUCENE-5656) IndexWriter leaks CFS handles in
some exceptional cases
[ https://issues.apache.org/jira/browse/LUCENE-5656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-5656:
--------------------------------
Description:
in trunk:
ant test -Dtestcase=TestIndexWriterOutOfMemory -Dtests.method=testBasics -Dtests.seed=3D485DE153FCA22D -Dtests.nightly=true -Dtests.locale=no_NO -Dtests.timezone=CAT -Dtests.file.encoding=US-ASCII
Seems to happen when an exception is thrown here:
{noformat}
[junit4] 1> java.lang.OutOfMemoryError: Fake OutOfMemoryError
[junit4] 1> at org.apache.lucene.index.TestIndexWriterOutOfMemory$2.eval(TestIndexWriterOutOfMemory.java:117)
[junit4] 1> at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:888)
[junit4] 1> at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:575)
[junit4] 1> at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:107)
[junit4] 1> at org.apache.lucene.codecs.lucene45.Lucene45DocValuesProducer.<init>(Lucene45DocValuesProducer.java:84)
[junit4] 1> at org.apache.lucene.codecs.lucene45.Lucene45DocValuesFormat.fieldsProducer(Lucene45DocValuesFormat.java:178)
[junit4] 1> at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:232)
[junit4] 1> at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:324)
[junit4] 1> at org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(SegmentDocValues.java:51)
[junit4] 1> at org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(SegmentDocValues.java:68)
[junit4] 1> at org.apache.lucene.index.SegmentReader.initDocValuesProducers(SegmentReader.java:189)
[junit4] 1> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:166)
[junit4] 1> at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:553)
[junit4] 1> at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:230)
[junit4] 1> at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3086)
[junit4] 1> at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3077)
[junit4] 1> at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2791)
[junit4] 1> at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2940)
[junit4] 1> at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2907)
{noformat}
and the leak is from here:
{noformat}
[junit4] > Caused by: java.lang.RuntimeException: unclosed IndexInput: _0_Asserting_0.dvd
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:560)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:604)
[junit4] > at org.apache.lucene.codecs.lucene45.Lucene45DocValuesProducer.<init>(Lucene45DocValuesProducer.java:116)
[junit4] > at org.apache.lucene.codecs.lucene45.Lucene45DocValuesFormat.fieldsProducer(Lucene45DocValuesFormat.java:178)
[junit4] > at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat.fieldsProducer(AssertingDocValuesFormat.java:61)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:232)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:324)
[junit4] > at org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(SegmentDocValues.java:51)
[junit4] > at org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(SegmentDocValues.java:68)
[junit4] > at org.apache.lucene.index.SegmentReader.initDocValuesProducers(SegmentReader.java:189)
[junit4] > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:116)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:133)
[junit4] > at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:211)
[junit4] > at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3086)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3077)
[junit4] > at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2791)
[junit4] > at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2940)
[junit4] > at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2907)
[junit4] > at org.apache.lucene.index.TestIndexWriterOutOfMemory.testBasics(TestIndexWriterOutOfMemory.java:209)
{noformat}
was:
So far i've only hit this on 4.8 branch, no fails on trunk/4.x:
ant test -Dtestcase=TestIndexWriterOutOfMemory -Dtests.method=testBasics -Dtests.seed=62D205D7767C40C -Dtests.nightly=true -Dtests.locale=ar_JO -Dtests.timezone=Africa/Bujumbura -Dtests.file.encoding=UTF-8
Seems to happen when an exception is thrown here:
{noformat}
[junit4] 1> java.lang.OutOfMemoryError: Fake OutOfMemoryError
[junit4] 1> at org.apache.lucene.index.TestIndexWriterOutOfMemory$2.eval(TestIndexWriterOutOfMemory.java:118)
[junit4] 1> at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:883)
[junit4] 1> at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:576)
[junit4] 1> at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:114)
[junit4] 1> at org.apache.lucene.codecs.lucene45.Lucene45DocValuesProducer.<init>(Lucene45DocValuesProducer.java:85)
[junit4] 1> at org.apache.lucene.codecs.diskdv.DiskDocValuesProducer.<init>(DiskDocValuesProducer.java:31)
[junit4] 1> at org.apache.lucene.codecs.diskdv.DiskDocValuesFormat.fieldsProducer(DiskDocValuesFormat.java:56)
[junit4] 1> at org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(SegmentDocValues.java:51)
[junit4] 1> at org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(SegmentDocValues.java:68)
[junit4] 1> at org.apache.lucene.index.SegmentReader.initDocValuesProducers(SegmentReader.java:189)
[junit4] 1> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:116)
[junit4] 1> at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:142)
[junit4] 1> at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:624)
[junit4] 1> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
[junit4] 1> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3728)
[junit4] 1> at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] 1> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1987)
[junit4] 1> at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2987)
[junit4] 1> at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3096)
[junit4] 1> at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3063)
[junit4] 1> at org.apache.lucene.index.TestIndexWriterOutOfMemory.testBasics(TestIndexWriterOutOfMemory.java:210)
{noformat}
and the leak is from here:
{noformat}
[junit4] > Caused by: java.lang.RuntimeException: unclosed IndexInput: _2.cfs
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:561)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper$1.openSlice(MockDirectoryWrapper.java:981)
[junit4] > at org.apache.lucene.store.CompoundFileDirectory.openInput(CompoundFileDirectory.java:279)
[junit4] > at org.apache.lucene.codecs.lucene45.Lucene45DocValuesProducer.<init>(Lucene45DocValuesProducer.java:117)
[junit4] > at org.apache.lucene.codecs.diskdv.DiskDocValuesProducer.<init>(DiskDocValuesProducer.java:31)
[junit4] > at org.apache.lucene.codecs.diskdv.DiskDocValuesFormat.fieldsProducer(DiskDocValuesFormat.java:56)
[junit4] > at org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(SegmentDocValues.java:51)
[junit4] > at org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(SegmentDocValues.java:68)
[junit4] > at org.apache.lucene.index.SegmentReader.initDocValuesProducers(SegmentReader.java:189)
[junit4] > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:116)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:142)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:624)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3728)
[junit4] > at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1987)
[junit4] > at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2987)
[junit4] > at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3096)
[junit4] > at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3063)
[junit4] > at org.apache.lucene.index.TestIndexWriterOutOfMemory.testBasics(TestIndexWriterOutOfMemory.java:210)
{noformat}
> IndexWriter leaks CFS handles in some exceptional cases
> -------------------------------------------------------
>
> Key: LUCENE-5656
> URL: https://issues.apache.org/jira/browse/LUCENE-5656
> Project: Lucene - Core
> Issue Type: Bug
> Affects Versions: 4.8.1
> Reporter: Robert Muir
> Fix For: 4.8.1, 4.9, 5.0
>
>
> in trunk:
> ant test -Dtestcase=TestIndexWriterOutOfMemory -Dtests.method=testBasics -Dtests.seed=3D485DE153FCA22D -Dtests.nightly=true -Dtests.locale=no_NO -Dtests.timezone=CAT -Dtests.file.encoding=US-ASCII
> Seems to happen when an exception is thrown here:
> {noformat}
> [junit4] 1> java.lang.OutOfMemoryError: Fake OutOfMemoryError
> [junit4] 1> at org.apache.lucene.index.TestIndexWriterOutOfMemory$2.eval(TestIndexWriterOutOfMemory.java:117)
> [junit4] 1> at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:888)
> [junit4] 1> at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:575)
> [junit4] 1> at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:107)
> [junit4] 1> at org.apache.lucene.codecs.lucene45.Lucene45DocValuesProducer.<init>(Lucene45DocValuesProducer.java:84)
> [junit4] 1> at org.apache.lucene.codecs.lucene45.Lucene45DocValuesFormat.fieldsProducer(Lucene45DocValuesFormat.java:178)
> [junit4] 1> at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:232)
> [junit4] 1> at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:324)
> [junit4] 1> at org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(SegmentDocValues.java:51)
> [junit4] 1> at org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(SegmentDocValues.java:68)
> [junit4] 1> at org.apache.lucene.index.SegmentReader.initDocValuesProducers(SegmentReader.java:189)
> [junit4] 1> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:166)
> [junit4] 1> at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:553)
> [junit4] 1> at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:230)
> [junit4] 1> at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3086)
> [junit4] 1> at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3077)
> [junit4] 1> at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2791)
> [junit4] 1> at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2940)
> [junit4] 1> at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2907)
> {noformat}
> and the leak is from here:
> {noformat}
> [junit4] > Caused by: java.lang.RuntimeException: unclosed IndexInput: _0_Asserting_0.dvd
> [junit4] > at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:560)
> [junit4] > at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:604)
> [junit4] > at org.apache.lucene.codecs.lucene45.Lucene45DocValuesProducer.<init>(Lucene45DocValuesProducer.java:116)
> [junit4] > at org.apache.lucene.codecs.lucene45.Lucene45DocValuesFormat.fieldsProducer(Lucene45DocValuesFormat.java:178)
> [junit4] > at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat.fieldsProducer(AssertingDocValuesFormat.java:61)
> [junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:232)
> [junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:324)
> [junit4] > at org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(SegmentDocValues.java:51)
> [junit4] > at org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(SegmentDocValues.java:68)
> [junit4] > at org.apache.lucene.index.SegmentReader.initDocValuesProducers(SegmentReader.java:189)
> [junit4] > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:116)
> [junit4] > at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:133)
> [junit4] > at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:211)
> [junit4] > at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3086)
> [junit4] > at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3077)
> [junit4] > at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2791)
> [junit4] > at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2940)
> [junit4] > at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2907)
> [junit4] > at org.apache.lucene.index.TestIndexWriterOutOfMemory.testBasics(TestIndexWriterOutOfMemory.java:209)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org