You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Mikkel Kamstrup Erlandsen (JIRA)" <ji...@apache.org> on 2010/02/08 20:59:28 UTC
[jira] Created: (LUCENE-2255) IndexWriter.getReader() allocates
file handles
IndexWriter.getReader() allocates file handles
----------------------------------------------
Key: LUCENE-2255
URL: https://issues.apache.org/jira/browse/LUCENE-2255
Project: Lucene - Java
Issue Type: Bug
Components: Index
Environment: Ubuntu 9.10, Java 6
Reporter: Mikkel Kamstrup Erlandsen
Attachments: LuceneManyCommits.java
I am not sure if this is a "bug" or really just me not reading the Javadocs right...
The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates
file handles
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831095#action_12831095 ]
Jason Rutherglen commented on LUCENE-2255:
------------------------------------------
You're not the first to report this... I wonder if we should also do a weak ref type of dealie that auto closes the reader for users who aren't calling IR.close?
To answer the question, yes, close on IR must be called. I can't directly do anything about the javadocs so I'll leave that for Mike.
> IndexWriter.getReader() allocates file handles
> ----------------------------------------------
>
> Key: LUCENE-2255
> URL: https://issues.apache.org/jira/browse/LUCENE-2255
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Environment: Ubuntu 9.10, Java 6
> Reporter: Mikkel Kamstrup Erlandsen
> Attachments: LuceneManyCommits.java
>
>
> I am not sure if this is a "bug" or really just me not reading the Javadocs right...
> The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
> So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2255) IndexWriter.getReader() allocates
file handles
Posted by "Mikkel Kamstrup Erlandsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mikkel Kamstrup Erlandsen updated LUCENE-2255:
----------------------------------------------
Attachment: LuceneManyCommits.java
Leaks file handles via IW.getReader()
> IndexWriter.getReader() allocates file handles
> ----------------------------------------------
>
> Key: LUCENE-2255
> URL: https://issues.apache.org/jira/browse/LUCENE-2255
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Environment: Ubuntu 9.10, Java 6
> Reporter: Mikkel Kamstrup Erlandsen
> Attachments: LuceneManyCommits.java
>
>
> I am not sure if this is a "bug" or really just me not reading the Javadocs right...
> The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
> So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates
file handles
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831124#action_12831124 ]
Michael McCandless commented on LUCENE-2255:
--------------------------------------------
This has already been fixed on trunk... we added this to the javadocs a while back:
* <p>You must close the {@link IndexReader} returned by
* this method once you are done using it.</p>
While the returned reader used IndexWriter to determine which segments to open, the returned reader is fully independent of the writer. EG you can close the writer and continue using the reader. And you must close the reader.
I'll backport that fix to 3.0 & 2.9.
> IndexWriter.getReader() allocates file handles
> ----------------------------------------------
>
> Key: LUCENE-2255
> URL: https://issues.apache.org/jira/browse/LUCENE-2255
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Environment: Ubuntu 9.10, Java 6
> Reporter: Mikkel Kamstrup Erlandsen
> Attachments: LuceneManyCommits.java
>
>
> I am not sure if this is a "bug" or really just me not reading the Javadocs right...
> The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
> So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2255) IndexWriter.getReader() allocates
file handles
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-2255.
----------------------------------------
Resolution: Fixed
Fix Version/s: 3.1
3.0.1
2.9.2
> IndexWriter.getReader() allocates file handles
> ----------------------------------------------
>
> Key: LUCENE-2255
> URL: https://issues.apache.org/jira/browse/LUCENE-2255
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Environment: Ubuntu 9.10, Java 6
> Reporter: Mikkel Kamstrup Erlandsen
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: LuceneManyCommits.java
>
>
> I am not sure if this is a "bug" or really just me not reading the Javadocs right...
> The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
> So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates
file handles
Posted by "Jason Rutherglen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831116#action_12831116 ]
Jason Rutherglen commented on LUCENE-2255:
------------------------------------------
bq. What keeps you from attaching a patch for the IW javadoc?
Laziness and lack of motivation... :)
I'll post something to the issue, how's that?
> IndexWriter.getReader() allocates file handles
> ----------------------------------------------
>
> Key: LUCENE-2255
> URL: https://issues.apache.org/jira/browse/LUCENE-2255
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Environment: Ubuntu 9.10, Java 6
> Reporter: Mikkel Kamstrup Erlandsen
> Attachments: LuceneManyCommits.java
>
>
> I am not sure if this is a "bug" or really just me not reading the Javadocs right...
> The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
> So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates
file handles
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831107#action_12831107 ]
Simon Willnauer commented on LUCENE-2255:
-----------------------------------------
I see this coming up multiple times, we should document this properly in the javadoc and on the wiki. Jason, aren't you the NRT specialist here. What keeps you from attaching a patch for the IW javadoc?
simon
> IndexWriter.getReader() allocates file handles
> ----------------------------------------------
>
> Key: LUCENE-2255
> URL: https://issues.apache.org/jira/browse/LUCENE-2255
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Environment: Ubuntu 9.10, Java 6
> Reporter: Mikkel Kamstrup Erlandsen
> Attachments: LuceneManyCommits.java
>
>
> I am not sure if this is a "bug" or really just me not reading the Javadocs right...
> The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
> So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org