[jira] Commented: (LUCENE-1609) Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead

["Jed Wesley-Smith (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716118#action_12716118 ] 

Jed Wesley-Smith commented on LUCENE-1609:
------------------------------------------

We get hit by this too. We'd love to see a fix and we'd agree that up-front initialisation would work for us.

AFAICT there are a number of other potential subtle concurrency issues with {{TermInfosReader}}:

# lack of {{final}} on fields - a number of fields ({{directory}}, {{segment}}, {{fieldInfos}}, {{origEnum}}, {{enumerators}} etc.) are never written to after construction and should be declared {{final}} for better publication semantics
# unsafe publication of {{indexDivisor}} and {{totalIndexInterval}} these fields are not written to under lock and in a worst-case could be unstable under use.
# {{close()}} calls {{enumerators.set(null)}} which only clears the value for the calling thread.

Making the {{TermInfosReader}} more immutable would address some of these issues.

As far as the root problem goes, uncontended synchronisation is generally _very fast_, but significantly slows down once a lock becomes contended. The kind of pattern employed here (do something quite expensive but only once) is not an ideal use of synchronisation as it commonly leads to a contended lock, which remains a slow lock well after it is required\*. That being said, it isn't easy to do correctly and performantly under 1.4. 

\* An alternative approach is something like this [LazyReference|http://labs.atlassian.com/source/browse/CONCURRENT/trunk/src/main/java/com/atlassian/util/concurrent/LazyReference.java?r=2242] class, although this kind of thing really requires Java5 for full value.

> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead 
> ---------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1609
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1609
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: 2.9
>         Environment: Solr 
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
>            Reporter: Dan Rosher
>             Fix For: 2.9
>
>         Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress test application, and later, examining the Thread dump (kill -3) , many threads are blocked on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation uses a state pattern, where only one thread can move the object from IndexNotRead state to IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the index nolonger needs a synchronized method. 
> In my particular test, this uncreased throughput at least 30 times.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org