[jira] [Assigned] (SOLR-14904) Don't use documentCache for large result sets

["David Smiley (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SOLR-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Smiley reassigned SOLR-14904:
-----------------------------------

    Assignee: David Smiley

> Don't use documentCache for large result sets
> ---------------------------------------------
>
>                 Key: SOLR-14904
>                 URL: https://issues.apache.org/jira/browse/SOLR-14904
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: David Smiley
>            Assignee: David Smiley
>            Priority: Major
>
> Some users ask Solr to return many documents (high rows param), even though this is an anti-pattern.  Sometimes there is some sense to it, and even Solr itself will do it in some cases like "bin/solr export" and perhaps some streaming-expressions cases.  If there is a documentCache, these queries have a tendency to completely thrash it -- dump it and fill it with poor cache candidates.  I've even seen the cache's existence for such queries become a bottleneck of the query -- granted for the now old LRUCache and in a particularly high abuse-case.
> I propose that if the number of documents to be returned is above some fraction of the documentCache's size limit, then don't use the documentCache at all.  Maybe half size is sufficient?  Or quarter-size?  Maybe at least queryWindowSize big (thus at least 20 typically)?  I see in solrconfig a queryResultMaxDocsCached option used for the queryResultCache but it could be made to apply to populating the documentCache as well.  Code default is infinite but the default and most configs set to 200.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org