You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Paul Cowan (JIRA)" <ji...@apache.org> on 2007/03/22 01:09:32 UTC
[jira] Updated: (LUCENE-806) Synchronization bottleneck in
FieldSortedHitQueue with many concurrent readers
[ https://issues.apache.org/jira/browse/LUCENE-806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Cowan updated LUCENE-806:
------------------------------
Attachment: lucene-806-proposed-direction.patch
Hi all,
Attached is a patch which BEGINS to address (but by no means completes) the refactoring proposal I mentioned earlier. That is, it works towards removing the logic in FieldSortedHitQueue, replacing all of the comparatorXXX() methods with individual implementations of SortComparatorSource. This lets me work towards the next step, replacing the logic in the FieldCacheImpl.Cache { createValue() } with another SCS, using some sort of 'pluggable' mechanism rather than just creating new SCSes on the fly.
Attached primarily for review/discussion, as there's a lot more to do.
> Synchronization bottleneck in FieldSortedHitQueue with many concurrent readers
> ------------------------------------------------------------------------------
>
> Key: LUCENE-806
> URL: https://issues.apache.org/jira/browse/LUCENE-806
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.0.0
> Reporter: Paul Cowan
> Priority: Minor
> Attachments: lucene-806-proposed-direction.patch, lucene-806.patch
>
>
> The below is from a post by (my colleague) Paul Smith to the java-users list:
> ---
> Hi ho peoples.
> We have an application that is internationalized, and stores data from many languages (each project has it's own index, mostly aligned with a single language, maybe 2).
> Anyway, I've noticed during some thread dumps diagnosing some performance issues, that there appears to be a _potential_ synchronization bottleneck using Locale-based sorting of Strings. I don't think this problem is the root cause of our performance problem, but I thought I'd mention it here. Here's the stack dump of a thread waiting:
> "http-1001-Processor245" daemon prio=1 tid=0x31434da0 nid=0x3744 waiting for monitor entry [0x2cd44000..0x2cd45f30]
> at java.text.RuleBasedCollator.compare(RuleBasedCollator.java)
> - waiting to lock <0x6b1e8c68> (a java.text.RuleBasedCollator)
> at org.apache.lucene.search.FieldSortedHitQueue$4.compare(FieldSortedHitQueue.java:320)
> at org.apache.lucene.search.FieldSortedHitQueue.lessThan(FieldSortedHitQueue.java:114)
> at org.apache.lucene.util.PriorityQueue.upHeap(PriorityQueue.java:120)
> at org.apache.lucene.util.PriorityQueue.put(PriorityQueue.java:47)
> at org.apache.lucene.util.PriorityQueue.insert(PriorityQueue.java:58)
> at org.apache.lucene.search.FieldSortedHitQueue.insert(FieldSortedHitQueue.java:90)
> at org.apache.lucene.search.FieldSortedHitQueue.insert(FieldSortedHitQueue.java:97)
> at org.apache.lucene.search.TopFieldDocCollector.collect(TopFieldDocCollector.java:47)
> at org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:291)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:132)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:110)
> at com.aconex.index.search.FastLocaleSortIndexSearcher.search(FastLocaleSortIndexSearcher.java:90)
> .....
> In our case we had 12 threads waiting like this, while one thread had the lock on the RuleBasedCollator. Turns out RuleBasedCollator's.compare(...) method is synchronized. I wonder if a ThreadLocal based collator would be better here... ? There doesn't appear to be a reason for other threads searching the same index to wait on this sort. Be just as easy to use their own. (Is RuleBasedCollator a "heavy" object memory wise? Wouldn't have thought so, per thread)
> Thoughts?
> ---
> I've investigated this somewhat, and agree that this is a potential problem with a series of possible workarounds. Further discussion (including proof-of-concept patch) to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org