You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Ning Li (JIRA)" <ji...@apache.org> on 2006/10/17 09:54:35 UTC
[jira] Created: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Resources not always reclaimed in scorers after each search
-----------------------------------------------------------
Key: LUCENE-686
URL: http://issues.apache.org/jira/browse/LUCENE-686
Project: Lucene - Java
Issue Type: Bug
Components: Search
Environment: All
Reporter: Ning Li
Resources are not always reclaimed in scorers after each search.
For example, close() is not always called for term docs in TermScorer.
A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Paul Elschot (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-686?page=comments#action_12442943 ]
Paul Elschot commented on LUCENE-686:
-------------------------------------
Three things:
Is there an actual memory leak problem related to this?
In ReqExclScorer the two scorers can also be closed when they are set to null.
It's probably better to use try/finally in IndexSearcher and call close in in the finally clause,
exceptions are occasionally used to preliminary end a search, although not in the
lucene core afaik.
Regards,
Paul Elschot
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: http://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-686?page=comments#action_12444766 ]
Ning Li commented on LUCENE-686:
--------------------------------
But removing TermDocs.close() will leave IndexInput.close() in a
similar half-in/half-out situation: e.g. close() will not be called
for freqStream and skipStream in SegmentTermDocs. Yet
IndexInput.close() cannot be removed (e.g. FSIndexInput).
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: http://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Steven Parkes (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-686?page=comments#action_12444758 ]
Steven Parkes commented on LUCENE-686:
--------------------------------------
Well, this has been a nice example to drive me into some of the internals.
It seems like close() methods are around in most cases to provide determinism in operations that have externally visible side effects, e.g., commits in IndexReader.
But it doesn't seem like TermDocs has this type of behavior or is likely to in the foreseeable future? SegmentReader always keeps open handles to the appropriate files anyway.
One can certainly posit that someday a version of something will change and TermDocs.close() will do something, but wouldn't YAGNI argue for waiting until that's really the case?
I still kinda feel like this half-in/half-out (called sometimes, but not always) is kinda the worst of all worlds: it can't be relied on (not always called) and all those calls that don't do anything are (probably insignificant) wasted time. But the biggest thing is the intellectual drag, where people spend time trying to understand something that at least appears inconsistent.
If this reasoning is right, what do people think about just yanking it out?
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: http://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12478594 ]
Michael McCandless commented on LUCENE-686:
-------------------------------------------
I just committed some fixes in LUCENE-823 that could apply here. Specifically if we decide that clones should be called, then, MockRAMDirectory is setup to track that they in fact are closed (we would have to turn it on; right now it does not track clones).
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: https://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Steven Parkes (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-686?page=comments#action_12447596 ]
Steven Parkes commented on LUCENE-686:
--------------------------------------
Came across these:
http://www.gossamer-threads.com/lists/lucene/java-dev/34085#34085
http://www.gossamer-threads.com/lists/lucene/java-dev/34121#34121
Even if we don't change semantics, we could get some of this into the docs.
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: http://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-686?page=comments#action_12442987 ]
Ning Li commented on LUCENE-686:
--------------------------------
> Is there an actual memory leak problem related to this?
Right now no. For example, in FS based directories, the index inputs term docs use are clones.
Close() of cloned index inputs does not close the file descriptor. Only the origianl one does.
However, memory leak could happen to a new subclass of directory and index input, if cloned
instances require reclaiming resources. In addition, memory leak could happen to a new subclass
of scorer, if there are resources associated with the scorer which should be reclaimed when done.
> In ReqExclScorer the two scorers can also be closed when they are set to null.
Thanks for pointing this out. I'll double check all scorers and make sure close() are properly called.
> It's probably better to use try/finally in IndexSearcher and call close in in the finally clause,
> exceptions are occasionally used to preliminary end a search, although not in the
> lucene core afaik.
Will do. Thanks again!
Cheers,
Ning
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: http://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-686?page=comments#action_12444438 ]
Hoss Man commented on LUCENE-686:
---------------------------------
Quick summary of some discussion from the mailing list...
1) i replied to paul's comments in the bug indicating that while there may not be any leaks in the core code base, these changes were needed to allow people writing custom Directories or custom Scorers to avoid memory leaks.
2) paul suggested that people writing custom code can work arround this by subclassing/customizing the Directory, and all the Scorers, and the IndexSearcher
3) i suggested that made the barrier for new custom code rather high, and made a poor comparison that got us on a tangent.
4) i argued that since TermDocs had a close method, Scorers needed to call it, which ment they needed a close method which was garunteed to be called.
5) paul argued that TermDocs.close in the core right now isn't needed, and we might be better off removing it, and requiring any more complicated custom implimentations to rely on GC to clean up any resources they have (presumably using a finalize method)
6) steven_parkes then raised the point that the fundemental issue is design integrity ... we have to agree what the point of TermDocs.close is from an API standpoint, and that callers should not have to know what the concrete implimentation of hte callee is to know wether close needs to be called. Better documentation on the purpose of the method can lead to better discussion about wether it can be removed, or if the current behavior is a bug that needs fixed.
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: http://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Doron Cohen (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-686?page=comments#action_12444742 ]
Doron Cohen commented on LUCENE-686:
------------------------------------
An example of how current Lucene code relies on not having to close resoures, in PhraseQuery:
...
scorer(IndexReader reader) {
...
for (int i = 0; i < terms.size(); i++) {
TermPositions p = reader.termPositions((Term)terms.elementAt(i));
if (p == null)
return null; < - - - - change would be required here
tps[i] = p;
}
If close() has to be respected this code would need to change to close all TermPositions that were obtained just before the one that was not found.
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: http://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-686?page=all ]
Ning Li updated LUCENE-686:
---------------------------
Attachment: ScorerResourceGC.patch
A patch is attached:
- The patch is based on the lastest version from trunk.
- The patch includes a test called TestScorerResourceGJ which shows resources are not reclaimed after each search without the patch.
- The patch passes TestScorerResourcesGJ.
- The patch passes all the unit tests.
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: http://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Steven Parkes (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-686?page=comments#action_12445027 ]
Steven Parkes commented on LUCENE-686:
--------------------------------------
You're right, and that's actually at the heart of the issue.
Seems like part of what needs to be clarified is what clone() does on an IndexInput. It doesn't simply make another copy of the IndexInput. The result is neither independent of nor equivalent to the original. At least for FSIndexInput:
- not equivalent: close on a clone is different then close on the original
- not independent: close on the original impacts the behavior of existing clones
I don't see that that's going to change. The semantics may be a little bit squrrely (and maybe the name clone() isn't the best, if the result isn't a true clone?), but it's obviously not causing any problems.
I would think the best thing to do would be just document this existing behavior so it can be relied on: clones of IndexInputs do not need to be closed and cannot be relied upon if the clone source is closed.
If/once that's the documented behavior, it's okay to get rid of all the extra close()es. New implementations of IndexInput just need to abide by that behavior, however they choose to implement it.
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: http://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-686) Resources not always reclaimed in scorers after each search
Posted by Paul Elschot <pa...@xs4all.nl>.
On Wednesday 18 October 2006 06:21, Chris Hostetter wrote:
> : > : When custom Scorers and/or Directories need a close method, it can
> : > : also be provided by subclassing Scorer, IndexSearcher and Directory
>
> : > it seems like that would handicap adoption of new
Queries/Directories ...
> : > I don't know how many people would have been interested in
> : > ConstantScoreQuery or FunctionQuery if they'd been told they had to stop
> : > using IndexSearcher and start using a new
"ConstantScorableIndexSearcher"
> :
> : Iirc no (general) closing method in Scorer or IndexSearcher was needed
> : for the Constant... and Function... queries and scorers.
>
> that's kind of side stepping my point: those classes gained popularity and
> use because they were easy to adopt -- they didn't have any complex
> dependencies, and could be droped into an existing Lucene application and
> be used without any other changes. If they *had* needed maintain
> resources, and that had forced the creation of custom IndexSearchers, no
> one would have ever tried them out -- hence my point that your suggested
> workarround would handicap the creation of new more complex Scorers and
> Directories.
>
> The bottom line is:
> 1) TermDocs has a close method
> 2) anything implimenting TermDocs should be able to trust that
> close() will be called
> 3) anything iterating over a TermDocs should call close()
>
> ...ergo: TermScorer needs a close() method, right?
There is no memory leak now, and TermDocs.close() is not always
called, so I'd rather conclude that a close() method is not really
needed.
It might be nice to release resources early, but the garbage collector
can also do that later, and perhaps even more efficiently.
When it turns out that using close() during search helps performance
in some cases, I'd like to know about them.
Regards,
Paul Elschot
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-686) Resources not always reclaimed in scorers after each search
Posted by Ning Li <ni...@gmail.com>.
A new scorer that requires reclaiming resources could be used by many
other scorers such as boolean scorers and conjunction scorers. Then
those scorers should have a closing method and so do the ones use
those scorers... A general closing method would be better, wouldn't
it?
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-686) Resources not always reclaimed
in scorers after each search
Posted by Chris Hostetter <ho...@fucit.org>.
: > : When custom Scorers and/or Directories need a close method, it can
: > : also be provided by subclassing Scorer, IndexSearcher and Directory
: > it seems like that would handicap adoption of new Queries/Directories ...
: > I don't know how many people would have been interested in
: > ConstantScoreQuery or FunctionQuery if they'd been told they had to stop
: > using IndexSearcher and start using a new "ConstantScorableIndexSearcher"
:
: Iirc no (general) closing method in Scorer or IndexSearcher was needed
: for the Constant... and Function... queries and scorers.
that's kind of side stepping my point: those classes gained popularity and
use because they were easy to adopt -- they didn't have any complex
dependencies, and could be droped into an existing Lucene application and
be used without any other changes. If they *had* needed maintain
resources, and that had forced the creation of custom IndexSearchers, no
one would have ever tried them out -- hence my point that your suggested
workarround would handicap the creation of new more complex Scorers and
Directories.
The bottom line is:
1) TermDocs has a close method
2) anything implimenting TermDocs should be able to trust that
close() will be called
3) anything iterating over a TermDocs should call close()
...ergo: TermScorer needs a close() method, right?
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-686) Resources not always reclaimed in scorers after each search
Posted by Paul Elschot <pa...@xs4all.nl>.
On Tuesday 17 October 2006 21:10, Chris Hostetter wrote:
>
> : When custom Scorers and/or Directories need a close method, it can
> : also be provided by subclassing Scorer, IndexSearcher and Directory
> : in the custom code.
> : Not providing this close method in the Lucene core passes the message
> : that a working implementation is possible without it.
>
> it seems like that would handicap adoption of new Queries/Directories ...
> I don't know how many people would have been interested in
> ConstantScoreQuery or FunctionQuery if they'd been told they had to stop
> using IndexSearcher and start using a new "ConstantScorableIndexSearcher"
Iirc no (general) closing method in Scorer or IndexSearcher was needed
for the Constant... and Function... queries and scorers.
And with the code readable from an svn repository, these is no need
to stop at any class to consider modifications.
Regards,
Paul Elschot
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-686) Resources not always reclaimed
in scorers after each search
Posted by Chris Hostetter <ho...@fucit.org>.
: When custom Scorers and/or Directories need a close method, it can
: also be provided by subclassing Scorer, IndexSearcher and Directory
: in the custom code.
: Not providing this close method in the Lucene core passes the message
: that a working implementation is possible without it.
it seems like that would handicap adoption of new Queries/Directories ...
I don't know how many people would have been interested in
ConstantScoreQuery or FunctionQuery if they'd been told they had to stop
using IndexSearcher and start using a new "ConstantScorableIndexSearcher"
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-686) Resources not always reclaimed in scorers after each search
Posted by Paul Elschot <pa...@xs4all.nl>.
On Tuesday 17 October 2006 20:32, Chris Hostetter wrote:
>
> : But as long as there is no real a memory leak, what is the point of
> : adding this close functionality?
>
> I think the concern is not so much that Lucene core as is has any leaks --
> but that subclasses of core implimentations have no mechanism for safely
> managing resources. Custom Directory implimenations that rely on close()
> or custom Scorers that try to do anything complex involving external
> resources would have leaks of whatever resource they need to explicitly
> manage.
I remember being surprised that Lucene Scorers do not need a closing
method to manage their resources. The garbage collector does this.
When custom Scorers and/or Directories need a close method, it can
also be provided by subclassing Scorer, IndexSearcher and Directory
in the custom code.
Not providing this close method in the Lucene core passes the message
that a working implementation is possible without it.
Regards,
Paul Elschot
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-686) Resources not always reclaimed
in scorers after each search
Posted by Chris Hostetter <ho...@fucit.org>.
: But as long as there is no real a memory leak, what is the point of
: adding this close functionality?
I think the concern is not so much that Lucene core as is has any leaks --
but that subclasses of core implimentations have no mechanism for safely
managing resources. Custom Directory implimenations that rely on close()
or custom Scorers that try to do anything complex involving external
resources would have leaks of whatever resource they need to explicitly
manage.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-686) Resources not always reclaimed in
scorers after each search
Posted by "Paul Elschot (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-686?page=comments#action_12443014 ]
Paul Elschot commented on LUCENE-686:
-------------------------------------
In SpanTermQuery in the spans package, there is also a TermPositions that might
need to be closed.
This means that closing would have to be done via Spans in the whole spans package.
But as long as there is no real a memory leak, what is the point of
adding this close functionality?
> Resources not always reclaimed in scorers after each search
> -----------------------------------------------------------
>
> Key: LUCENE-686
> URL: http://issues.apache.org/jira/browse/LUCENE-686
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Environment: All
> Reporter: Ning Li
> Attachments: ScorerResourceGC.patch
>
>
> Resources are not always reclaimed in scorers after each search.
> For example, close() is not always called for term docs in TermScorer.
> A test will be attached to show when resources are not reclaimed.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org