[jira] [Created] (SOLR-10634) Move calculation of some aggregations to collection phase

["Yonik Seeley (JIRA)" <ji...@apache.org>]

Yonik Seeley created SOLR-10634:
-----------------------------------

             Summary: Move calculation of some aggregations to collection phase
                 Key: SOLR-10634
                 URL: https://issues.apache.org/jira/browse/SOLR-10634
             Project: Solr
          Issue Type: Improvement
      Security Level: Public (Default Security Level. Issues are Public)
          Components: Facet Module
            Reporter: Yonik Seeley


From http://markmail.org/message/pwgnt7iqxkzcnckh

{quote}
The current code is more optimized for finding the top K buckets from
a total of N.
When one asks to return the top 10 buckets when there are potentially
millions of buckets, it makes sense to defer calculating other metrics
for those buckets until we know which ones they are.  After we
identify the top 10 buckets, we calculate the domain for that bucket
and use that to calculate the remaining metrics.

The current method is obviously much slower when one is requesting
*all* buckets.  We might as well just calculate all metrics in the
first pass rather than trying to defer them.
{quote}

So we should move aggregations from the second pass to the first pass under the following conditions:
- no limit (or a high limit compared to the number of potential buckets?)
- no sub-facets (or anything else) that will need the domain calculated anyway
- aggregation is not really memory intensive per-slot (i.e. moving some calculations from per-bucket in the second phase, to all-buckets-in-parallel in the first phase could be really bad for peak memory usage)




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org