You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Simon Willnauer (JIRA)" <ji...@apache.org> on 2011/06/28 10:06:17 UTC
[jira] [Created] (LUCENE-3251) Directory#copy leaks file handles
Directory#copy leaks file handles
---------------------------------
Key: LUCENE-3251
URL: https://issues.apache.org/jira/browse/LUCENE-3251
Project: Lucene - Java
Issue Type: Bug
Components: core/store
Affects Versions: 3.2, 3.1, 3.0, 3.3, 3.4, 4.0
Reporter: Simon Willnauer
Fix For: 3.4, 4.0
Directory#copy doesn't close the target directories output stream if sourceDir.openInput(srcFile) throws an Exception. Before LUCENE-3218 Directory#copy wasn't used extensively so this wasn't likely to happen during tests. Today we had a failure on the 3.x branch that is likely caused by this bug:
{noformat}
[junit] Testsuite: org.apache.lucene.index.TestAddIndexes
[junit] Testcase: testAddIndexesWithRollback(org.apache.lucene.index.TestAddIndexes): Caused an ERROR
[junit] MockDirectoryWrapper: cannot close: there are still open files: {_co.cfs=1}
[junit] java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open files: {_co.cfs=1}
[junit] at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:483)
[junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads.closeDir(TestAddIndexes.java:693)
[junit] at org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:924)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1277)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1195)
[junit] Caused by: java.lang.RuntimeException: unclosed IndexOutput: _co.cfs
[junit] at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:410)
[junit] at org.apache.lucene.store.MockCompoundFileDirectoryWrapper.<init>(MockCompoundFileDirectoryWrapper.java:39)
[junit] at org.apache.lucene.store.MockDirectoryWrapper.createCompoundOutput(MockDirectoryWrapper.java:439)
[junit] at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:128)
[junit] at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3101)
[junit] at org.apache.lucene.index.TestAddIndexes$CommitAndAddIndexes3.doBody(TestAddIndexes.java:839)
[junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads$1.run(TestAddIndexes.java:667)
[junit]
[junit]
[junit] Tests run: 18, Failures: 0, Errors: 1, Time elapsed: 9.034 sec
[junit]
[junit] ------------- Standard Output ---------------
[junit] java.lang.IllegalStateException: CFS has pending open files
[junit] at org.apache.lucene.store.CompoundFileWriter.close(CompoundFileWriter.java:143)
[junit] at org.apache.lucene.store.CompoundFileDirectory.close(CompoundFileDirectory.java:181)
[junit] at org.apache.lucene.store.DefaultCompoundFileDirectory.close(DefaultCompoundFileDirectory.java:58)
[junit] at org.apache.lucene.store.MockCompoundFileDirectoryWrapper.close(MockCompoundFileDirectoryWrapper.java:55)
[junit] at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:139)
[junit] at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3101)
[junit] at org.apache.lucene.index.TestAddIndexes$CommitAndAddIndexes3.doBody(TestAddIndexes.java:839)
[junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads$1.run(TestAddIndexes.java:667)
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (LUCENE-3251) Directory#copy leaks file handles
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer reassigned LUCENE-3251:
---------------------------------------
Assignee: Simon Willnauer
> Directory#copy leaks file handles
> ---------------------------------
>
> Key: LUCENE-3251
> URL: https://issues.apache.org/jira/browse/LUCENE-3251
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/store
> Affects Versions: 3.0, 3.1, 3.2, 3.3, 3.4, 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: 3.4, 4.0
>
>
> Directory#copy doesn't close the target directories output stream if sourceDir.openInput(srcFile) throws an Exception. Before LUCENE-3218 Directory#copy wasn't used extensively so this wasn't likely to happen during tests. Today we had a failure on the 3.x branch that is likely caused by this bug:
> {noformat}
> [junit] Testsuite: org.apache.lucene.index.TestAddIndexes
> [junit] Testcase: testAddIndexesWithRollback(org.apache.lucene.index.TestAddIndexes): Caused an ERROR
> [junit] MockDirectoryWrapper: cannot close: there are still open files: {_co.cfs=1}
> [junit] java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open files: {_co.cfs=1}
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:483)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads.closeDir(TestAddIndexes.java:693)
> [junit] at org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:924)
> [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1277)
> [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1195)
> [junit] Caused by: java.lang.RuntimeException: unclosed IndexOutput: _co.cfs
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:410)
> [junit] at org.apache.lucene.store.MockCompoundFileDirectoryWrapper.<init>(MockCompoundFileDirectoryWrapper.java:39)
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.createCompoundOutput(MockDirectoryWrapper.java:439)
> [junit] at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:128)
> [junit] at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3101)
> [junit] at org.apache.lucene.index.TestAddIndexes$CommitAndAddIndexes3.doBody(TestAddIndexes.java:839)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads$1.run(TestAddIndexes.java:667)
> [junit]
> [junit]
> [junit] Tests run: 18, Failures: 0, Errors: 1, Time elapsed: 9.034 sec
> [junit]
> [junit] ------------- Standard Output ---------------
> [junit] java.lang.IllegalStateException: CFS has pending open files
> [junit] at org.apache.lucene.store.CompoundFileWriter.close(CompoundFileWriter.java:143)
> [junit] at org.apache.lucene.store.CompoundFileDirectory.close(CompoundFileDirectory.java:181)
> [junit] at org.apache.lucene.store.DefaultCompoundFileDirectory.close(DefaultCompoundFileDirectory.java:58)
> [junit] at org.apache.lucene.store.MockCompoundFileDirectoryWrapper.close(MockCompoundFileDirectoryWrapper.java:55)
> [junit] at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:139)
> [junit] at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3101)
> [junit] at org.apache.lucene.index.TestAddIndexes$CommitAndAddIndexes3.doBody(TestAddIndexes.java:839)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads$1.run(TestAddIndexes.java:667)
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3251) Directory#copy leaks file handles
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer resolved LUCENE-3251.
-------------------------------------
Resolution: Fixed
committed to trunk and backported to 3.x
> Directory#copy leaks file handles
> ---------------------------------
>
> Key: LUCENE-3251
> URL: https://issues.apache.org/jira/browse/LUCENE-3251
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/store
> Affects Versions: 3.0, 3.1, 3.2, 3.3, 3.4, 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: 3.4, 4.0
>
> Attachments: LUCENE-3251.patch
>
>
> Directory#copy doesn't close the target directories output stream if sourceDir.openInput(srcFile) throws an Exception. Before LUCENE-3218 Directory#copy wasn't used extensively so this wasn't likely to happen during tests. Today we had a failure on the 3.x branch that is likely caused by this bug:
> {noformat}
> [junit] Testsuite: org.apache.lucene.index.TestAddIndexes
> [junit] Testcase: testAddIndexesWithRollback(org.apache.lucene.index.TestAddIndexes): Caused an ERROR
> [junit] MockDirectoryWrapper: cannot close: there are still open files: {_co.cfs=1}
> [junit] java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open files: {_co.cfs=1}
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:483)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads.closeDir(TestAddIndexes.java:693)
> [junit] at org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:924)
> [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1277)
> [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1195)
> [junit] Caused by: java.lang.RuntimeException: unclosed IndexOutput: _co.cfs
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:410)
> [junit] at org.apache.lucene.store.MockCompoundFileDirectoryWrapper.<init>(MockCompoundFileDirectoryWrapper.java:39)
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.createCompoundOutput(MockDirectoryWrapper.java:439)
> [junit] at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:128)
> [junit] at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3101)
> [junit] at org.apache.lucene.index.TestAddIndexes$CommitAndAddIndexes3.doBody(TestAddIndexes.java:839)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads$1.run(TestAddIndexes.java:667)
> [junit]
> [junit]
> [junit] Tests run: 18, Failures: 0, Errors: 1, Time elapsed: 9.034 sec
> [junit]
> [junit] ------------- Standard Output ---------------
> [junit] java.lang.IllegalStateException: CFS has pending open files
> [junit] at org.apache.lucene.store.CompoundFileWriter.close(CompoundFileWriter.java:143)
> [junit] at org.apache.lucene.store.CompoundFileDirectory.close(CompoundFileDirectory.java:181)
> [junit] at org.apache.lucene.store.DefaultCompoundFileDirectory.close(DefaultCompoundFileDirectory.java:58)
> [junit] at org.apache.lucene.store.MockCompoundFileDirectoryWrapper.close(MockCompoundFileDirectoryWrapper.java:55)
> [junit] at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:139)
> [junit] at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3101)
> [junit] at org.apache.lucene.index.TestAddIndexes$CommitAndAddIndexes3.doBody(TestAddIndexes.java:839)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads$1.run(TestAddIndexes.java:667)
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3251) Directory#copy leaks file handles
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-3251:
------------------------------------
Attachment: LUCENE-3251.patch
attached a fix
> Directory#copy leaks file handles
> ---------------------------------
>
> Key: LUCENE-3251
> URL: https://issues.apache.org/jira/browse/LUCENE-3251
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/store
> Affects Versions: 3.0, 3.1, 3.2, 3.3, 3.4, 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: 3.4, 4.0
>
> Attachments: LUCENE-3251.patch
>
>
> Directory#copy doesn't close the target directories output stream if sourceDir.openInput(srcFile) throws an Exception. Before LUCENE-3218 Directory#copy wasn't used extensively so this wasn't likely to happen during tests. Today we had a failure on the 3.x branch that is likely caused by this bug:
> {noformat}
> [junit] Testsuite: org.apache.lucene.index.TestAddIndexes
> [junit] Testcase: testAddIndexesWithRollback(org.apache.lucene.index.TestAddIndexes): Caused an ERROR
> [junit] MockDirectoryWrapper: cannot close: there are still open files: {_co.cfs=1}
> [junit] java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open files: {_co.cfs=1}
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:483)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads.closeDir(TestAddIndexes.java:693)
> [junit] at org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:924)
> [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1277)
> [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1195)
> [junit] Caused by: java.lang.RuntimeException: unclosed IndexOutput: _co.cfs
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:410)
> [junit] at org.apache.lucene.store.MockCompoundFileDirectoryWrapper.<init>(MockCompoundFileDirectoryWrapper.java:39)
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.createCompoundOutput(MockDirectoryWrapper.java:439)
> [junit] at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:128)
> [junit] at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3101)
> [junit] at org.apache.lucene.index.TestAddIndexes$CommitAndAddIndexes3.doBody(TestAddIndexes.java:839)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads$1.run(TestAddIndexes.java:667)
> [junit]
> [junit]
> [junit] Tests run: 18, Failures: 0, Errors: 1, Time elapsed: 9.034 sec
> [junit]
> [junit] ------------- Standard Output ---------------
> [junit] java.lang.IllegalStateException: CFS has pending open files
> [junit] at org.apache.lucene.store.CompoundFileWriter.close(CompoundFileWriter.java:143)
> [junit] at org.apache.lucene.store.CompoundFileDirectory.close(CompoundFileDirectory.java:181)
> [junit] at org.apache.lucene.store.DefaultCompoundFileDirectory.close(DefaultCompoundFileDirectory.java:58)
> [junit] at org.apache.lucene.store.MockCompoundFileDirectoryWrapper.close(MockCompoundFileDirectoryWrapper.java:55)
> [junit] at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:139)
> [junit] at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3101)
> [junit] at org.apache.lucene.index.TestAddIndexes$CommitAndAddIndexes3.doBody(TestAddIndexes.java:839)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads$1.run(TestAddIndexes.java:667)
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3251) Directory#copy leaks file handles
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13056393#comment-13056393 ]
Shai Erera commented on LUCENE-3251:
------------------------------------
patch looks good.
> Directory#copy leaks file handles
> ---------------------------------
>
> Key: LUCENE-3251
> URL: https://issues.apache.org/jira/browse/LUCENE-3251
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/store
> Affects Versions: 3.0, 3.1, 3.2, 3.3, 3.4, 4.0
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: 3.4, 4.0
>
> Attachments: LUCENE-3251.patch
>
>
> Directory#copy doesn't close the target directories output stream if sourceDir.openInput(srcFile) throws an Exception. Before LUCENE-3218 Directory#copy wasn't used extensively so this wasn't likely to happen during tests. Today we had a failure on the 3.x branch that is likely caused by this bug:
> {noformat}
> [junit] Testsuite: org.apache.lucene.index.TestAddIndexes
> [junit] Testcase: testAddIndexesWithRollback(org.apache.lucene.index.TestAddIndexes): Caused an ERROR
> [junit] MockDirectoryWrapper: cannot close: there are still open files: {_co.cfs=1}
> [junit] java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open files: {_co.cfs=1}
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:483)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads.closeDir(TestAddIndexes.java:693)
> [junit] at org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:924)
> [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1277)
> [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1195)
> [junit] Caused by: java.lang.RuntimeException: unclosed IndexOutput: _co.cfs
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:410)
> [junit] at org.apache.lucene.store.MockCompoundFileDirectoryWrapper.<init>(MockCompoundFileDirectoryWrapper.java:39)
> [junit] at org.apache.lucene.store.MockDirectoryWrapper.createCompoundOutput(MockDirectoryWrapper.java:439)
> [junit] at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:128)
> [junit] at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3101)
> [junit] at org.apache.lucene.index.TestAddIndexes$CommitAndAddIndexes3.doBody(TestAddIndexes.java:839)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads$1.run(TestAddIndexes.java:667)
> [junit]
> [junit]
> [junit] Tests run: 18, Failures: 0, Errors: 1, Time elapsed: 9.034 sec
> [junit]
> [junit] ------------- Standard Output ---------------
> [junit] java.lang.IllegalStateException: CFS has pending open files
> [junit] at org.apache.lucene.store.CompoundFileWriter.close(CompoundFileWriter.java:143)
> [junit] at org.apache.lucene.store.CompoundFileDirectory.close(CompoundFileDirectory.java:181)
> [junit] at org.apache.lucene.store.DefaultCompoundFileDirectory.close(DefaultCompoundFileDirectory.java:58)
> [junit] at org.apache.lucene.store.MockCompoundFileDirectoryWrapper.close(MockCompoundFileDirectoryWrapper.java:55)
> [junit] at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:139)
> [junit] at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3101)
> [junit] at org.apache.lucene.index.TestAddIndexes$CommitAndAddIndexes3.doBody(TestAddIndexes.java:839)
> [junit] at org.apache.lucene.index.TestAddIndexes$RunAddIndexesThreads$1.run(TestAddIndexes.java:667)
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org