[jira] [Commented] (SOLR-8559) FCS facet performance optimization

["Dennis Gove (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15106833#comment-15106833 ] 

Dennis Gove commented on SOLR-8559:
-----------------------------------

Are you able to create a test for this specific feature? Or if not, are there existing tests covering this code I can specifically check after applying the patch?

> FCS facet performance optimization
> ----------------------------------
>
>                 Key: SOLR-8559
>                 URL: https://issues.apache.org/jira/browse/SOLR-8559
>             Project: Solr
>          Issue Type: Improvement
>          Components: faceting
>            Reporter: Keith Laban
>              Labels: optimization, performance
>         Attachments: solr-8559.patch
>
>
> While profiling a large collection (multi-sharded billions of documents), I found that a fast (5-10ms query) which had no matches would take 20-30 seconds when doing facets even when {{facet.mincount=1}}
> Profiling made it apparent that with {{facet.method=fcs}} 99% of the time was [spent here|https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/request/PerSegmentSingleValuedFaceting.java#L212].
> {{queue.udpateTop}} gets called {{numOfSegments*numTerms}}, the worst case when every term is in every segment. This formula doesn't take into account whether or not any of the terms have a positive count with respect to the docset.
> These optimizations are aimed to do two things:
> # When mincount>0 don't include segments which all terms have zero counts. This should significantly speed up processing when terms are high cardinality and the matching docset is small
> # FIXED TODO optimization: when mincount>0 move segment position the next non zero term value.
> both of these changes will minimize the number of called needed to the slow {{updateTop}} call.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org