You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org> on 2011/12/17 01:30:30 UTC
[jira] [Commented] (LUCENE-3653) Lucene Search not scalling
[ https://issues.apache.org/jira/browse/LUCENE-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171367#comment-13171367 ]
Uwe Schindler commented on LUCENE-3653:
---------------------------------------
The problems you are mentioning are no issues at all:
- VirtualMethod is only used during class instantiations and class loading and must be synchronized. There is unlikely contention at all, just because its synchronized it does not mean its slow.
- getAttributeInterfaces must be synchronized, too, as it has a reflection cache and is also only used during TokenStream instantiation. Analyzers should reuse TokenStreams so its not an issue at all. Fix your analyzers to resuse TokenStreams.
On concurency the average time increases because of eventual contention in your file system directory implementation, not because methods may be synchronized.
> Lucene Search not scalling
> --------------------------
>
> Key: LUCENE-3653
> URL: https://issues.apache.org/jira/browse/LUCENE-3653
> Project: Lucene - Java
> Issue Type: Improvement
> Reporter: Gerrit Jansen van Vuuren
>
> I've noticed that when doing thousands of searches in a single thread the average time is quite low i.e. a few milliseconds. When adding more concurrent searches doing exactly the same search the average time increases drastically.
> I've profiled the search classes and found that the whole of lucene blocks on
> org.apache.lucene.index.SegmentCoreReaders.getTermsReader
> org.apache.lucene.util.VirtualMethod
> public synchronized int getImplementationDistance
> org.apache.lucene.util.AttributeSourcew.getAttributeInterfaces
> These cause search times to increase from a few milliseconds to up to 2 seconds when doing 500 concurrent searches on the same in memory index. Note: That the index is not being updates at all, so not refresh methods are called at any stage.
> Some questions:
> Why do we need synchronization here?
> There must be a non-lockable solution for these, they basically cause lucene to be ok for single thread applications but disastrous for any concurrent implementation.
> I'll do some experiments by removing the synchronization from the methods of these classes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org