[jira] [Created] (LUCENE-8058) Never cache large TermInSetQuery instances

["Adrien Grand (JIRA)" <ji...@apache.org>]

Adrien Grand created LUCENE-8058:
------------------------------------

             Summary: Never cache large TermInSetQuery instances
                 Key: LUCENE-8058
                 URL: https://issues.apache.org/jira/browse/LUCENE-8058
             Project: Lucene - Core
          Issue Type: Improvement
            Reporter: Adrien Grand
             Fix For: master (8.0), 7.2


I have seen several cases in which the query cache was highly underestimating its memory usage due to the fact that it had references to large queries that ended up using more memory than the associated doc id sets.

We had a workaround for term-in-set queries by making TermInSetQuery implement Accountable, but this information is lost when it is wrapped in another query such as a BooleanQuery. So I would like to apply a safer fix that just disables caching on large TermInSetQuery instances.

I know it's a pity given that large queries are probably more expensive and thus more cache-worthy, but I see such large queries as the result of a bad design or a workaround to the fact that Lucene is not the right tool for the job, so I think that disabling caching on large term-in-set queries is the right trade-off by making the query cache safer for the majority of our users.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org