You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/08/11 10:30:00 UTC
[jira] [Commented] (LUCENE-9002) query caching leads to absurdly
slow queries
[ https://issues.apache.org/jira/browse/LUCENE-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397259#comment-17397259 ]
ASF subversion and git services commented on LUCENE-9002:
---------------------------------------------------------
Commit a9fb5a965dffd2600f7da7700e6429ebf19e14d6 in lucene's branch refs/heads/main from Julie Tibshirani
[ https://gitbox.apache.org/repos/asf?p=lucene.git;h=a9fb5a9 ]
LUCENE-10043: Decrease default LRUQueryCache#skipCacheFactor to 10 (#232)
In LUCENE-9002 we introduced logic to skip caching a clause if it would be too
expensive compared to the usual query cost. Specifically, we avoid caching a
clause if its cost is estimated to be a 250x higher than the lead iterator's.
We've found that the default of 250 is quite high and can lead to poor tail
latencies. This PR decreases it to 10 to cache more conservatively.
> query caching leads to absurdly slow queries
> --------------------------------------------
>
> Key: LUCENE-9002
> URL: https://issues.apache.org/jira/browse/LUCENE-9002
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
> Affects Versions: 7.7.2, 8.2
> Reporter: Guoqiang Jiang
> Priority: Major
> Labels: cache, performance
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
> *Description*
> We have dozens of ES clusters(based on Lucene) for metric scenarios. Most of the queries are like this: _host_ip:10.10.10.10 AND timestamp:[2019-10-01 00:00:00 TO 2019-10-05 23:59:59]_. And we frequently encounter some absurdly slow queries.
> *Solution*
> For a long time range query(e.g. 5 days), each range query will consume tens of megabytes of memory and spend hundreds of milliseconds to cache, but the benefits are not obvious. And those large cache entries will cause frequent cache eviction. So it's better to skip the caching action directly when large range query appears with a selective lead iterator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org