You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Brendan Grainger <br...@gmail.com> on 2012/06/29 01:06:22 UTC
Closing directory in LuceneTestCase causes stacktrace
Hi,
I'm trying to brush up on some of the *cough* newer APIs (we've been using 2.9.2 up until now). Anyway, I have the test below which a modified version of one of the tests in Lucene In Action, but uses LuceneTestCase as a base class.
public class MetaphoneReplacementAnaylyzerTest extends LuceneTestCase {
@Test
public void testKoolKat() throws Exception {
Analyzer analyzer = new MetaphoneReplacementAnalyzer();
IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_36, analyzer);
// Directory directory = new RAMDirectory();
Directory directory = newDirectory();
IndexWriter writer = new IndexWriter(directory, config);
Document doc = new Document();
doc.add(new Field("contents", "cool cat", Field.Store.YES, Field.Index.ANALYZED));
writer.addDocument(doc, analyzer);
writer.commit();
IndexSearcher searcher = new IndexSearcher(IndexReader.open(directory));
// IndexSearcher searcher = new IndexSearcher(IndexReader.open(writer, true));
Query q = new QueryParser(Version.LUCENE_36, "contents", analyzer).parse("kool kat");
TopDocs hits = searcher.search(q, 1);
assertEquals(1, hits.totalHits);
int docID = hits.scoreDocs[0].doc;
doc = searcher.doc(docID);
assertEquals("cool cat", doc.get("contents"));
writer.close();
searcher.close();
directory.close(); // if I comment this out, I get an assertion failure about the directory not being closed
}
}
My issue is that I get the following stack trace when I try to close the directory. I've tried variations of closing the writer instead of calling commit etc, but I think I'm just missing something fundamental here. Note that obviously if I change :
Directory directory = newDirectory();
to:
Directory directory = new RAMDirectory()
everything works, but I'd love to know how to correctly close the writer, searcher etc.
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open files: {_0.tis=1, _0.frq=1, _0.fdx=1, _0.prx=1, _0.fdt=1}
at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:548)
at com.kuripai.lucene.analysis.MetaphoneReplacementAnaylyzerTest.testKoolKat(MetaphoneReplacementAnaylyzerTest.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
...
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.RuntimeException: unclosed IndexInput: _0.prx
at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:472)
at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:497)
at org.apache.lucene.store.Directory.openInput(Directory.java:145)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:96)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:94)
at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:105)
at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:78)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:709)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:72)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:256)
at com.kuripai.lucene.analysis.MetaphoneReplacementAnaylyzerTest.testKoolKat(MetaphoneReplacementAnaylyzerTest.java:36)
... 39 more
Any insight would be greatly appreciated.
Thanks
Brendan
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Closing directory in LuceneTestCase causes stacktrace
Posted by Brendan Grainger <br...@gmail.com>.
Hi Robert,
I am trying to close the Directory on the very last line of the method which is where the exception is being thrown:
assertEquals("cool cat", doc.get("contents"));
writer.close();
searcher.close();
directory.close(); <= Crashes here with stack trace below. If I comment this out I get the exception about not closing the reader.
}
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open files: {_0.tis=1, _0.frq=1, _0.fdx=1, _0.prx=1, _0.fdt=1}
at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:548)
at com.kuripai.lucene.analysis.MetaphoneReplacementAnaylyzerTest.testKoolKat(MetaphoneReplacementAnaylyzerTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:630)
at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:536)
at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67)
at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:457)
at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74)
at org.apache.lucene.util.LuceneTestCase$SaveThreadAndTestNameRule$1.evaluate(LuceneTestCase.java:508)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:146)
at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74)
at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:36)
at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.RuntimeException: unclosed IndexInput: _0.prx
at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:472)
at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:497)
at org.apache.lucene.store.Directory.openInput(Directory.java:145)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:96)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:696)
at org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(IndexWriter.java:654)
at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:142)
at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:36)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:451)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:399)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:296)
at com.kuripai.lucene.analysis.MetaphoneReplacementAnaylyzerTest.testKoolKat(MetaphoneReplacementAnaylyzerTest.java:40)
... 39 more
Brendan Grainger
brendan.grainger@gmail.com
www.kuripai.com
On Jun 28, 2012, at 8:57 PM, Robert Muir wrote:
> On Thu, Jun 28, 2012 at 8:22 PM, Brendan Grainger
> <br...@gmail.com> wrote:
>>
>> Interestingly, if I change the *** line above to use the deprecated constructor taking just the directory it works fine:
>>
>
> its not interesting at all, its the typical contract of a java method.
> he who opens it closes it.
>
> I'll quote myself again:
>
> in this case (where IndexSearcher takes a reader that you passed in),
> closing the searcher won't actually close the underlying reader.
> you passed it in, so you should be sure to close this reader yourself.
>
>
> --
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
Re: Closing directory in LuceneTestCase causes stacktrace
Posted by Brendan Grainger <br...@gmail.com>.
Oops, I finally see it sorry. I could have sworn that this:
writer.close();
searcher.close();
directory.close();
was this!
writer.close();
searcher.close();
reader.close();
directory.close();
Sorry, totally works now.
Thanks!
On Jun 28, 2012, at 10:51 PM, Robert Muir wrote:
> That doesnt fix it, that hides your bug. Please re-read what I wrote.
> If your application code looks like this test then you are leaking
> file handles.
>
> close your indexreader!
>
>
> On Thu, Jun 28, 2012 at 10:39 PM, Brendan Grainger
> <br...@gmail.com> wrote:
>> Hi Robert,
>>
>> Looks like this fixes it:
>>
>> MockDirectoryWrapper directory = newDirectory();
>> directory.setNoDeleteOpenFile(false); // Don't emulate windows
>>
>> Thanks
>>
>> Brendan Grainger
>> brendan.grainger@gmail.com
>> www.kuripai.com
>>
>> On Jun 28, 2012, at 8:57 PM, Robert Muir wrote:
>>
>>> On Thu, Jun 28, 2012 at 8:22 PM, Brendan Grainger
>>> <br...@gmail.com> wrote:
>>>>
>>>> Interestingly, if I change the *** line above to use the deprecated constructor taking just the directory it works fine:
>>>>
>>>
>>> its not interesting at all, its the typical contract of a java method.
>>> he who opens it closes it.
>>>
>>> I'll quote myself again:
>>>
>>> in this case (where IndexSearcher takes a reader that you passed in),
>>> closing the searcher won't actually close the underlying reader.
>>> you passed it in, so you should be sure to close this reader yourself.
>>>
>>>
>>> --
>>> lucidimagination.com
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>
>
>
>
> --
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
Re: Closing directory in LuceneTestCase causes stacktrace
Posted by Robert Muir <rc...@gmail.com>.
That doesnt fix it, that hides your bug. Please re-read what I wrote.
If your application code looks like this test then you are leaking
file handles.
close your indexreader!
On Thu, Jun 28, 2012 at 10:39 PM, Brendan Grainger
<br...@gmail.com> wrote:
> Hi Robert,
>
> Looks like this fixes it:
>
> MockDirectoryWrapper directory = newDirectory();
> directory.setNoDeleteOpenFile(false); // Don't emulate windows
>
> Thanks
>
> Brendan Grainger
> brendan.grainger@gmail.com
> www.kuripai.com
>
> On Jun 28, 2012, at 8:57 PM, Robert Muir wrote:
>
>> On Thu, Jun 28, 2012 at 8:22 PM, Brendan Grainger
>> <br...@gmail.com> wrote:
>>>
>>> Interestingly, if I change the *** line above to use the deprecated constructor taking just the directory it works fine:
>>>
>>
>> its not interesting at all, its the typical contract of a java method.
>> he who opens it closes it.
>>
>> I'll quote myself again:
>>
>> in this case (where IndexSearcher takes a reader that you passed in),
>> closing the searcher won't actually close the underlying reader.
>> you passed it in, so you should be sure to close this reader yourself.
>>
>>
>> --
>> lucidimagination.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
--
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Closing directory in LuceneTestCase causes stacktrace
Posted by Brendan Grainger <br...@gmail.com>.
Hi Robert,
Looks like this fixes it:
MockDirectoryWrapper directory = newDirectory();
directory.setNoDeleteOpenFile(false); // Don't emulate windows
Thanks
Brendan Grainger
brendan.grainger@gmail.com
www.kuripai.com
On Jun 28, 2012, at 8:57 PM, Robert Muir wrote:
> On Thu, Jun 28, 2012 at 8:22 PM, Brendan Grainger
> <br...@gmail.com> wrote:
>>
>> Interestingly, if I change the *** line above to use the deprecated constructor taking just the directory it works fine:
>>
>
> its not interesting at all, its the typical contract of a java method.
> he who opens it closes it.
>
> I'll quote myself again:
>
> in this case (where IndexSearcher takes a reader that you passed in),
> closing the searcher won't actually close the underlying reader.
> you passed it in, so you should be sure to close this reader yourself.
>
>
> --
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
Re: Closing directory in LuceneTestCase causes stacktrace
Posted by Robert Muir <rc...@gmail.com>.
On Thu, Jun 28, 2012 at 8:22 PM, Brendan Grainger
<br...@gmail.com> wrote:
>
> Interestingly, if I change the *** line above to use the deprecated constructor taking just the directory it works fine:
>
its not interesting at all, its the typical contract of a java method.
he who opens it closes it.
I'll quote myself again:
in this case (where IndexSearcher takes a reader that you passed in),
closing the searcher won't actually close the underlying reader.
you passed it in, so you should be sure to close this reader yourself.
--
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Closing directory in LuceneTestCase causes stacktrace
Posted by Brendan Grainger <br...@gmail.com>.
I am closing the directory and that's the line where the exception about the open files is being thrown:
public class MetaphoneReplacementAnaylyzerTest extends LuceneTestCase {
@Test
public void testKoolKat() throws Exception {
Analyzer analyzer = new MetaphoneReplacementAnalyzer();
IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_36, analyzer);
// Directory directory = new RAMDirectory();
Directory directory = newDirectory();
IndexWriter writer = new IndexWriter(directory, config);
Document doc = new Document();
doc.add(new Field("contents", "cool cat", Field.Store.YES, Field.Index.ANALYZED));
writer.addDocument(doc, analyzer);
writer.commit();
*** IndexSearcher searcher = new IndexSearcher(IndexReader.open(directory));
Query q = new QueryParser(Version.LUCENE_36, "contents", analyzer).parse("kool kat");
TopDocs hits = searcher.search(q, 1);
assertEquals(1, hits.totalHits);
int docID = hits.scoreDocs[0].doc;
doc = searcher.doc(docID);
assertEquals("cool cat", doc.get("contents"));
writer.close();
searcher.close();
directory.close(); // <=== if I comment this out, I get an assertion failure about the directory not being closed. If I leave this in I get an error about the files being open
}
}
Interestingly, if I change the *** line above to use the deprecated constructor taking just the directory it works fine:
IndexSearcher searcher = new IndexSearcher(directory);
Brendan Grainger
brendan.grainger@gmail.com
www.kuripai.com
On Jun 28, 2012, at 7:36 PM, Robert Muir wrote:
> Hello,
>
> this part of the stacktrace:
>
>
>> Caused by: java.lang.RuntimeException: unclosed IndexInput: _0.prx
>> at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:472)
>> at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:497)
>> at org.apache.lucene.store.Directory.openInput(Directory.java:145)
>> at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:96)
>> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
>> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:94)
>> at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:105)
>> at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
>> at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:78)
>> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:709)
>> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:72)
>> at org.apache.lucene.index.IndexReader.open(IndexReader.java:256)
>> at com.kuripai.lucene.analysis.MetaphoneReplacementAnaylyzerTest.testKoolKat(MetaphoneReplacementAnaylyzerTest.java:36)
>> ... 39 more
>>
>
> is telling you where in your code you opened the un-closed object that
> you need to close.
>
> looks to me like the problem is how you open your reader:
> new IndexSearcher(IndexReader.open(directory));
>
> in this case (where IndexSearcher takes a reader that you passed in),
> closing the searcher won't actually close the underlying reader.
> you passed it in, so you should be sure to close this reader yourself.
>
> --
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
Re: Closing directory in LuceneTestCase causes stacktrace
Posted by Robert Muir <rc...@gmail.com>.
Hello,
this part of the stacktrace:
> Caused by: java.lang.RuntimeException: unclosed IndexInput: _0.prx
> at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:472)
> at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:497)
> at org.apache.lucene.store.Directory.openInput(Directory.java:145)
> at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:96)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:94)
> at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:105)
> at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
> at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:78)
> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:709)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:72)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:256)
> at com.kuripai.lucene.analysis.MetaphoneReplacementAnaylyzerTest.testKoolKat(MetaphoneReplacementAnaylyzerTest.java:36)
> ... 39 more
>
is telling you where in your code you opened the un-closed object that
you need to close.
looks to me like the problem is how you open your reader:
new IndexSearcher(IndexReader.open(directory));
in this case (where IndexSearcher takes a reader that you passed in),
closing the searcher won't actually close the underlying reader.
you passed it in, so you should be sure to close this reader yourself.
--
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org