You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Yonik Seeley (JIRA)" <ji...@apache.org> on 2017/05/11 15:06:04 UTC
[jira] [Updated] (SOLR-10634) Move calculation of some aggregations
to collection phase
[ https://issues.apache.org/jira/browse/SOLR-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated SOLR-10634:
--------------------------------
Attachment: SOLR-10634.patch
Here's a first-cut patch. Existing tests pass, but no specific tests yet to even see if it's being exercised. I don't think that limit:-1 is exercised much at all either.
> Move calculation of some aggregations to collection phase
> ---------------------------------------------------------
>
> Key: SOLR-10634
> URL: https://issues.apache.org/jira/browse/SOLR-10634
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
> Reporter: Yonik Seeley
> Attachments: SOLR-10634.patch
>
>
> From http://markmail.org/message/pwgnt7iqxkzcnckh
> {quote}
> The current code is more optimized for finding the top K buckets from
> a total of N.
> When one asks to return the top 10 buckets when there are potentially
> millions of buckets, it makes sense to defer calculating other metrics
> for those buckets until we know which ones they are. After we
> identify the top 10 buckets, we calculate the domain for that bucket
> and use that to calculate the remaining metrics.
> The current method is obviously much slower when one is requesting
> *all* buckets. We might as well just calculate all metrics in the
> first pass rather than trying to defer them.
> {quote}
> So we should move aggregations from the second pass to the first pass under the following conditions:
> - no limit (or a high limit compared to the number of potential buckets?)
> - no sub-facets (or anything else) that will need the domain calculated anyway
> - aggregation is not really memory intensive per-slot (i.e. moving some calculations from per-bucket in the second phase, to all-buckets-in-parallel in the first phase could be really bad for peak memory usage)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org