You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Chris Lu (JIRA)" <ji...@apache.org> on 2008/09/13 17:48:44 UTC
[jira] Commented: (LUCENE-1383) Work around ThreadLocal's "leak"
[ https://issues.apache.org/jira/browse/LUCENE-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630784#action_12630784 ]
Chris Lu commented on LUCENE-1383:
----------------------------------
I can confirm this patch fixed the memory problem, in general. Thanks!
However, if not nitpicking, I did noticed in this patch, during the transition time between the previous index searcher/reader close and the new index searcher/reader open, there is a bump in the memory graph.
____/\____
There are 2 RAMDirectory instances in the memory for a short period of time.
In previous Lucene version without Lucene-1195, the memory graph looks more clean, with a V dip.
____ ____
\/
And in one index refresh during the same index run, there were 2 RAMDirectory in the memory for a fairly long time. But not repeatable.
So I suspect the closing is kind of delayed. And sometimes it could be missed.
Attaching the screenshot for the memory bump, and 2 the memory graphs of the begin and after of 2RAMDirecory in the memory(needed 2 because the time is fairly long)
> Work around ThreadLocal's "leak"
> --------------------------------
>
> Key: LUCENE-1383
> URL: https://issues.apache.org/jira/browse/LUCENE-1383
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 1.9, 2.0.0, 2.1, 2.2, 2.3, 2.3.1, 2.3.2
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: 2.4
>
> Attachments: LUCENE-1383.patch, ScreenHunter_01 Sep. 13 08.40.jpg
>
>
> Java's ThreadLocal is dangerous to use because it is able to take a
> surprisingly very long time to release references to the values you
> store in it. Even when a ThreadLocal instance itself is GC'd, hard
> references to the values you had stored in it are easily kept for
> quite some time later.
> While this is not technically a "memory leak", because eventually
> (when the underlying Map that stores the values cleans up its "stale"
> references) the hard reference will be cleared, and GC can proceed,
> its end behavior is not different from a memory leak in that under the
> right situation you can easily tie up far more memory than you'd
> expect, and then hit unexpected OOM error despite allocating an
> extremely large heap to your JVM.
> Lucene users have hit this many times. Here's the most recent thread:
> http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200809.mbox/%3C6e3ae6310809091157j7a9fe46bxcc31f6e63305fcdc%40mail.gmail.com%3E
> And here's another:
> http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200807.mbox/%3CF5FC94B2-E5C7-40C0-8B73-E12245B91CEE%40mikemccandless.com%3E
> And then there's LUCENE-436 and LUCENE-529 at least.
> A google search for "ThreadLocal leak" yields many compelling hits.
> Sun does this for performance reasons, but I think it's a terrible
> trap and we should work around it with Lucene.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org