You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Dan Rosher (JIRA)" <ji...@apache.org> on 2009/04/23 17:28:30 UTC
[jira] Created: (LUCENE-1609) Eliminate synchronization contention
on initial index reading in TermInfosReader ensureIndexIsRead
Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
---------------------------------------------------------------------------------------------------
Key: LUCENE-1609
URL: https://issues.apache.org/jira/browse/LUCENE-1609
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Affects Versions: 2.9
Environment: Solr
Tomcat 5.5
Ubuntu 2.6.20-17-generic
Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
Reporter: Dan Rosher
synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1609) Eliminate synchronization contention
on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1609:
---------------------------------------
Fix Version/s: 2.9
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Jed Wesley-Smith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716118#action_12716118 ]
Jed Wesley-Smith commented on LUCENE-1609:
------------------------------------------
We get hit by this too. We'd love to see a fix and we'd agree that up-front initialisation would work for us.
AFAICT there are a number of other potential subtle concurrency issues with {{TermInfosReader}}:
# lack of {{final}} on fields - a number of fields ({{directory}}, {{segment}}, {{fieldInfos}}, {{origEnum}}, {{enumerators}} etc.) are never written to after construction and should be declared {{final}} for better publication semantics
# unsafe publication of {{indexDivisor}} and {{totalIndexInterval}} these fields are not written to under lock and in a worst-case could be unstable under use.
# {{close()}} calls {{enumerators.set(null)}} which only clears the value for the calling thread.
Making the {{TermInfosReader}} more immutable would address some of these issues.
As far as the root problem goes, uncontended synchronisation is generally _very fast_, but significantly slows down once a lock becomes contended. The kind of pattern employed here (do something quite expensive but only once) is not an ideal use of synchronisation as it commonly leads to a contended lock, which remains a slow lock well after it is required\*. That being said, it isn't easy to do correctly and performantly under 1.4.
\* An alternative approach is something like this [LazyReference|http://labs.atlassian.com/source/browse/CONCURRENT/trunk/src/main/java/com/atlassian/util/concurrent/LazyReference.java?r=2242] class, although this kind of thing really requires Java5 for full value.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705138#action_12705138 ]
Michael McCandless commented on LUCENE-1609:
--------------------------------------------
Could we instead change TermInfosReader (it's package private so that's OK) to take "boolean loadIndex", and fix callers to thread down that argument?
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-1609) Eliminate synchronization contention
on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-1609:
------------------------------------------
Assignee: Michael McCandless
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1609) Eliminate synchronization contention
on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Dan Rosher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Rosher updated LUCENE-1609:
-------------------------------
Attachment: LUCENE-1609.patch
I've added the TermInfosReader object to the constructor of the IndexNotRead object so that it can be synchronised on during readIndex, and then protect indexTerms, indexInfos, indexPointer
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Reopened: (LUCENE-1609) Eliminate synchronization contention
on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reopened LUCENE-1609:
----------------------------------------
Reopening to un-deprecate getTermInfosIndexDivisor.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch, LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701994#action_12701994 ]
Uwe Schindler commented on LUCENE-1609:
---------------------------------------
You could fix this, if you put all these field into the state object, too (an abstract class instead of interface containing these variables) and cloning those on creating the new state. But then you have the mentioned problem, that one thread may exchange the state object to a IndexRead, but another one still sees the reference to the IndexNotRead object, not used any longer. As log as you not also sychronize the state object change or make it volatile in Java 1.5 it will still not work. That was, what I meant.
In my opinion, this is not fixable in any case with these type of state objects, yes?
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Attachments: LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705279#action_12705279 ]
Yonik Seeley commented on LUCENE-1609:
--------------------------------------
bq. If it's only for segment merging, couldn't we up front conditionalize the loading of the index?
Since the indexWriter now does deletes, doesn't it sometimes need the term index too?
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702379#action_12702379 ]
Yonik Seeley commented on LUCENE-1609:
--------------------------------------
Re: why the lazy load
http://www.lucidimagination.com/search/document/97e73361748808b/terminfosreader_lazy_term_index_reading#2a73aaca25d516ec
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Attachments: LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1609) Eliminate synchronization contention
on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1609:
---------------------------------------
Attachment: LUCENE-1609.patch
Attached patch. This addresses this issue and LUCENE-1718.
I added 2 new static IndexReader.open expert methods that allow you to
pass in the TermInfos index divisor. You can pass in -1 to disable
loading of the index entirely (eg, IndexWriter does this when merging
segments). I also added the param to IndexWriter.getReader, so you
can get an NRT reader w/ subsampled index terms.
This replaces the set/getTermInfosIndexDivisor methods (they are now
deprecated), ie you now must specify the divisor when opening the
reader. If these methods are called, an UnsupportedOperationException
is thrown. This is technically a break in back compat (previously you
could call it before the terms index was used, eg if no searches had
been run) but I think we should make an exception here. Very few
users make use of these expert methods, and having these users switch
to specifying the index divisor up front is a small code change in
exchange for removing all synchronization from the terms dict.
I also made all attrs in TermInfosReader final, and there is no longer
any synchronization. To handle the case in IndexWriter, where a merge
first opens a segment (which does not need the index) and then an NRT
reader (or, applyDeletes) needs to share the same pooled reader and
needs the terms index, I added a PrivateTermsDict static class to
SegmentReader. This class just wraps a no-index-loaded
TermInfosReader, which merging will use, and then can open a new
index-is-loaded TermInfosReader when/if needed.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1609) Eliminate synchronization contention
on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1609.
----------------------------------------
Resolution: Fixed
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch, LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705547#action_12705547 ]
Michael McCandless commented on LUCENE-1609:
--------------------------------------------
bq. Since the indexWriter now does deletes, doesn't it sometimes need the term index too?
True, so when IW opens the SR in order to apply deletes it should request that the terms index be loaded?
Though, because iW now internally pools open SR's (for NRT search), it's entirely possible that an SR already opened for merging is re-used when it's time to apply deletes (even in the NRT case). To handle that we could add a package private method to load the terms index, which only IW would call (we do something similar with the doc stores). IW's synchronization ensures only one thread would call the method.
It would then not be necessary to check on every Term lookup whether the index was loaded.
In general I prefer up-front initialization when possible. Ie, when I call IndexReader.open, it should do as much initializing as it can instead of deferring initialization until the first query.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701973#action_12701973 ]
Uwe Schindler commented on LUCENE-1609:
---------------------------------------
Are you sure, this works correct? If the indexState is changed in the synchronized block, another thread not synchronizing on the lock may still see the old indexState. At least, the indexState must be volatile, but this only works correct with Java 1.5 (and Lucene only needs Java 1.4 as requirement).
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Attachments: LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1609) Eliminate synchronization contention
on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1609:
---------------------------------------
Attachment: LUCENE-1609.patch
Updated patch to resolve conflicts after LUCENE-1726... I plan to commit soon.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch, LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716376#action_12716376 ]
Yonik Seeley commented on LUCENE-1609:
--------------------------------------
It would be nice to fix, but I wonder how important it is (i.e. a must fix for 2.9 vs 3.0 where we can use Java5)
Wouldn't an application that hit contention on this very fast test, simply go on to hit a different contention point after this one is removed?
A much better application fix is to simply limit the number of threads executing queries.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716593#action_12716593 ]
Michael McCandless commented on LUCENE-1609:
--------------------------------------------
I agree we could ship 2.9 without this fix (in fact I think most of
the issues marked 2.9 could be postponed until 3.x), but...
Because the ability of the JRE to downgrade an inflated lock back to a
lightweight lock seems to be very implementation/version dependent,
coupled with the fact that term info lookup is very frequently called
(especially for MultiTermQuery), makes me think this is indeed
something rather important to fix.
And a nice side effect of up-front init is that it front-loads
initialization more, ie, the work is done in IndexReader.open instead
of the first query. (Other things, like populating FieldCache,
loading norms, still happen on the first query of course...).
bq. A much better application fix is to simply limit the number of threads executing queries.
Well... if your hardware has concurrency, is it really the case that
Lucene never gets in the way? I'd love to see this confirmed
empirically. Ie you should ideally be able to run as many threads as
will saturate your hardware, and see it scale properly.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1609) Eliminate synchronization contention
on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Dan Rosher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Rosher updated LUCENE-1609:
-------------------------------
Attachment: LUCENE-1609.patch
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Attachments: LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702011#action_12702011 ]
Earwin Burrfoot commented on LUCENE-1609:
-----------------------------------------
You cannot put all these fields into state object, because you introduce state to it and it can no longer be unsafely published.
> one thread may exchange the state object to a IndexRead, but another one still sees the reference to the IndexNotRead object
Nothing terrible here, a thread hitting stale IndexNotRead synchronizes and short-circuits in the beginning of the method. The problem is that seeing proper state object doesn't guarantee seeing fields it is supposed to guard :)
Yes, it's not fixable here without volatile or proper synchronization. But I still have a feeling that lazy loading (and consequent synchronization) is not needed here at all.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Attachments: LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702404#action_12702404 ]
Michael McCandless commented on LUCENE-1609:
--------------------------------------------
If it's only for segment merging, couldn't we up front conditionalize the loading of the index?
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Attachments: LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716232#action_12716232 ]
Michael McCandless commented on LUCENE-1609:
--------------------------------------------
I agree, we need to fix this for 2.9. I'll explore the "initialize up front" approach.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701986#action_12701986 ]
Earwin Burrfoot commented on LUCENE-1609:
-----------------------------------------
The problem is not with indexState not being volatile. You can unsafely publish objects that have no internal state, or their state is consistent enough for you under any memory visibility/reordering effects. See working example of it in LUCENE-1607, Yonik's hash for interning strings.
The problem is that indexState guards indexTerms, indexInfos, indexPointers, which aren't volatile too and aren't guarded by the lock. It is possible that one thread does load these fields and then sets indexState = new IndexRead(), but another thread sees only the last write and dies with NPE.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Attachments: LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1609) Eliminate synchronization contention
on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1609.
----------------------------------------
Resolution: Fixed
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch, LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1609) Eliminate
synchronization contention on initial index reading in TermInfosReader
ensureIndexIsRead
Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701986#action_12701986 ]
Earwin Burrfoot edited comment on LUCENE-1609 at 4/23/09 9:41 AM:
------------------------------------------------------------------
The problem is not with indexState not being volatile. You can unsafely publish objects that have no internal state, or their state is consistent enough for you under any memory visibility/reordering effects. See working example of it in LUCENE-1607, Yonik's hash for interning strings.
The problem is that indexState guards indexTerms, indexInfos, indexPointers, which aren't volatile too and aren't guarded by the lock. It is possible that one thread does load these fields and then sets indexState = new IndexRead(), but another thread sees only the last write and dies with NPE.
The thing I don't get, is why do we want lazy loading here at all? Is there any usage for TermInfosReader that prevents it from initializing in constructor?
was (Author: earwin):
The problem is not with indexState not being volatile. You can unsafely publish objects that have no internal state, or their state is consistent enough for you under any memory visibility/reordering effects. See working example of it in LUCENE-1607, Yonik's hash for interning strings.
The problem is that indexState guards indexTerms, indexInfos, indexPointers, which aren't volatile too and aren't guarded by the lock. It is possible that one thread does load these fields and then sets indexState = new IndexRead(), but another thread sees only the last write and dies with NPE.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Attachments: LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1609) Eliminate synchronization
contention on initial index reading in TermInfosReader ensureIndexIsRead
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718175#action_12718175 ]
Michael McCandless commented on LUCENE-1609:
--------------------------------------------
Alas... the big problem with doing this up-front is the IndexReader.setTermIndexInterval, which relies on the fact that the index is loaded lazily.
So, maybe we need to wait until 3.0 to do this up-front.
But perhaps for this issue we should make it possible to pass in the termIndexInterval to IndexReader.open, and deprecate the current methods, and then in 3.0 we could switch to up-front loading.
> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Assignee: Michael McCandless
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org