You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/04/04 04:22:00 UTC
[jira] [Commented] (SOLR-16693) Use TimeLimitingBulkScorer; stop using ExitableDirectoryReader
[ https://issues.apache.org/jira/browse/SOLR-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17708204#comment-17708204 ]
ASF subversion and git services commented on SOLR-16693:
--------------------------------------------------------
Commit c9d5bcbf8098abf144c35c20b8a7663b5ab84f4b in solr's branch refs/heads/main from David Smiley
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=c9d5bcbf809 ]
SOLR-16693: timeAllowed: use TimeLimitingBulkScorer (#1450)
For timeAllowed: use more efficient TimeLimitingBulkScorer instead of ExitableDirectoryReader
Added sys prop escape hatch: solr.useExitableDirectoryReader
> Use TimeLimitingBulkScorer; stop using ExitableDirectoryReader
> --------------------------------------------------------------
>
> Key: SOLR-16693
> URL: https://issues.apache.org/jira/browse/SOLR-16693
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 9.2
> Reporter: David Smiley
> Assignee: David Smiley
> Priority: Blocker
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> While reviewing Lucene 9.5 changes coming into Solr, I noticed some changes relating to the ability to specify timeAllowed in Solr on a search query. Solr uses both ExitableDirectoryReader and TimeLimitingCollector from Lucene for this (complementary to each other). Unfortunately, changes in Lucene will make the cost of ExitableDirectoryReader wrapping happen for all queries into Solr, even those not using timeAllowed. Options to keep EDR aren't good -- fork it basically. Anecdotally, I think I've heard the overhead is not trivial and my intuition thinks likewise. Meanwhile, Lucene 9.3 added a new TimeLimitingBulkScorer which even gets first class integration into IndexSearcher which has a timeout. It's been incrementally improved, and I really like its approach, probable performance, and simplicity. It should be straightforward to integrate this into SolrIndexSearcher and also only do so for queries specifying timeAllowed. I'm not sure TimeLimitingCollector offers much value to using TLBS other than additional precision on timeAllowed at some cost to unselective queries.
> I think doing this should block Solr 9.2 using Lucene 9.5. Alternatively, someone might benchmark the state of things and see that things aren't so bad as they may seem. But that takes work too.
> [1] QueryTimeout.isTimeoutEnabled is gone: https://github.com/apache/lucene/pull/11954
> [2] TimeLimitingBulkScorer in LUCENE-10151 https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/TimeLimitingBulkScorer.java
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org