You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Karl Wettin (JIRA)" <ji...@apache.org> on 2008/04/17 18:11:21 UTC
[jira] Created: (LUCENE-1265) Identify bottleneck associated with
not pooling searchers
Identify bottleneck associated with not pooling searchers
---------------------------------------------------------
Key: LUCENE-1265
URL: https://issues.apache.org/jira/browse/LUCENE-1265
Project: Lucene - Java
Issue Type: Task
Components: Search
Reporter: Karl Wettin
There have been multiple reports suggesting synchronization problems due to great search performace boost while pooling searchers under heavy load.
I think this should be confirmed. Then identified and fixed, alternatively add such a pool to the code base.
http://www.nabble.com/Multiple-searchers-%28Was%3A-CachingWrapperFilter%3A-why-cache-per-IndexReader-%29-td14916124.html#a14997611
http://www.nabble.com/Solid-State-Drives-vs.-RAMDirectory-td16025864.html
http://wiki.apache.org/lucene-java/ImproveSearchingSpeed
Use one instance of IndexSearcher.
Share a single IndexSearcher across queries and across threads in your application.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1265) Identify bottleneck associated with
not pooling searchers
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1265.
----------------------------------------
Resolution: Duplicate
Dup of LUCENE-1329 (readOnly IndexReader) and LUCENE-753 (NIOFSDirectory), at least on non-Windows platforms.
Based on this user report it seems like sharing an IndexReader across multiple threads in fact gives better performance than thread-private IndexReader instances:
http://www.nabble.com/IndexSearcher-and-multi-threaded-performance-td20449375.html
> Identify bottleneck associated with not pooling searchers
> ---------------------------------------------------------
>
> Key: LUCENE-1265
> URL: https://issues.apache.org/jira/browse/LUCENE-1265
> Project: Lucene - Java
> Issue Type: Task
> Components: Search
> Reporter: Karl Wettin
>
> There have been multiple reports suggesting synchronization problems due to great search performace boost while pooling searchers under heavy load.
> I think this should be confirmed. Then identified and fixed, alternatively add such a pool to the code base.
> http://www.nabble.com/Multiple-searchers-%28Was%3A-CachingWrapperFilter%3A-why-cache-per-IndexReader-%29-td14916124.html#a14997611
> http://www.nabble.com/Solid-State-Drives-vs.-RAMDirectory-td16025864.html
> http://wiki.apache.org/lucene-java/ImproveSearchingSpeed
> Use one instance of IndexSearcher.
> Share a single IndexSearcher across queries and across threads in your application.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1265) Identify bottleneck associated with
not pooling searchers
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12590082#action_12590082 ]
Karl Wettin commented on LUCENE-1265:
-------------------------------------
I'll be looking in to this any year now.
> Identify bottleneck associated with not pooling searchers
> ---------------------------------------------------------
>
> Key: LUCENE-1265
> URL: https://issues.apache.org/jira/browse/LUCENE-1265
> Project: Lucene - Java
> Issue Type: Task
> Components: Search
> Reporter: Karl Wettin
>
> There have been multiple reports suggesting synchronization problems due to great search performace boost while pooling searchers under heavy load.
> I think this should be confirmed. Then identified and fixed, alternatively add such a pool to the code base.
> http://www.nabble.com/Multiple-searchers-%28Was%3A-CachingWrapperFilter%3A-why-cache-per-IndexReader-%29-td14916124.html#a14997611
> http://www.nabble.com/Solid-State-Drives-vs.-RAMDirectory-td16025864.html
> http://wiki.apache.org/lucene-java/ImproveSearchingSpeed
> Use one instance of IndexSearcher.
> Share a single IndexSearcher across queries and across threads in your application.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1265) Identify bottleneck associated with
not pooling searchers
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12590089#action_12590089 ]
Yonik Seeley commented on LUCENE-1265:
--------------------------------------
I think IndexInput is the right place to fix it (assuming the sync bottleneck is during seek+read).
> Identify bottleneck associated with not pooling searchers
> ---------------------------------------------------------
>
> Key: LUCENE-1265
> URL: https://issues.apache.org/jira/browse/LUCENE-1265
> Project: Lucene - Java
> Issue Type: Task
> Components: Search
> Reporter: Karl Wettin
>
> There have been multiple reports suggesting synchronization problems due to great search performace boost while pooling searchers under heavy load.
> I think this should be confirmed. Then identified and fixed, alternatively add such a pool to the code base.
> http://www.nabble.com/Multiple-searchers-%28Was%3A-CachingWrapperFilter%3A-why-cache-per-IndexReader-%29-td14916124.html#a14997611
> http://www.nabble.com/Solid-State-Drives-vs.-RAMDirectory-td16025864.html
> http://wiki.apache.org/lucene-java/ImproveSearchingSpeed
> Use one instance of IndexSearcher.
> Share a single IndexSearcher across queries and across threads in your application.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1265) Identify bottleneck associated with
not pooling searchers
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12590650#action_12590650 ]
Karl Wettin commented on LUCENE-1265:
-------------------------------------
Otis says:
bq. Not sure if this email got answered. That's most likely due to the synchronized isDeleted call:
bq. ./org/apache/lucene/index/SegmentReader.java: public synchronized boolean isDeleted(int n) {
> Identify bottleneck associated with not pooling searchers
> ---------------------------------------------------------
>
> Key: LUCENE-1265
> URL: https://issues.apache.org/jira/browse/LUCENE-1265
> Project: Lucene - Java
> Issue Type: Task
> Components: Search
> Reporter: Karl Wettin
>
> There have been multiple reports suggesting synchronization problems due to great search performace boost while pooling searchers under heavy load.
> I think this should be confirmed. Then identified and fixed, alternatively add such a pool to the code base.
> http://www.nabble.com/Multiple-searchers-%28Was%3A-CachingWrapperFilter%3A-why-cache-per-IndexReader-%29-td14916124.html#a14997611
> http://www.nabble.com/Solid-State-Drives-vs.-RAMDirectory-td16025864.html
> http://wiki.apache.org/lucene-java/ImproveSearchingSpeed
> Use one instance of IndexSearcher.
> Share a single IndexSearcher across queries and across threads in your application.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1265) Identify bottleneck associated with
not pooling searchers
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648048#action_12648048 ]
Mark Miller commented on LUCENE-1265:
-------------------------------------
Its looks like both this and LUCENE-1309 have been solved, no? Readonly IndexReaders and NIOFSDirectory seem to have cleared the bottleneck. (I think there is a third issue that is basically this out there somewhere too)
> Identify bottleneck associated with not pooling searchers
> ---------------------------------------------------------
>
> Key: LUCENE-1265
> URL: https://issues.apache.org/jira/browse/LUCENE-1265
> Project: Lucene - Java
> Issue Type: Task
> Components: Search
> Reporter: Karl Wettin
>
> There have been multiple reports suggesting synchronization problems due to great search performace boost while pooling searchers under heavy load.
> I think this should be confirmed. Then identified and fixed, alternatively add such a pool to the code base.
> http://www.nabble.com/Multiple-searchers-%28Was%3A-CachingWrapperFilter%3A-why-cache-per-IndexReader-%29-td14916124.html#a14997611
> http://www.nabble.com/Solid-State-Drives-vs.-RAMDirectory-td16025864.html
> http://wiki.apache.org/lucene-java/ImproveSearchingSpeed
> Use one instance of IndexSearcher.
> Share a single IndexSearcher across queries and across threads in your application.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org