You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Yonik Seeley (JIRA)" <ji...@apache.org> on 2010/08/27 03:44:53 UTC
[jira] Created: (SOLR-2092) Use a native priority queue to order
facet results
Use a native priority queue to order facet results
--------------------------------------------------
Key: SOLR-2092
URL: https://issues.apache.org/jira/browse/SOLR-2092
Project: Solr
Issue Type: Improvement
Reporter: Yonik Seeley
Deep paging into facets can be improved via use of a native priority queue.
Comparisons will be faster, and there will be less GC.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2092) Use a native priority queue to order
facet results
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905938#action_12905938 ]
Yonik Seeley commented on SOLR-2092:
------------------------------------
Performance results:
Faceting on a single valued field with 100K unique values.
Base doc set size = 100K docs.
facet.limit=10
8 request threads (CPU = 4 core Phenom II)
Average response time was measured externally (i.e. includes all request parsing, response writing, etc)
|facet.offset|ms trunk|ms patch
|0|18|17
|100|18|17
|1000|23|17
|10000|70|20
|100000|211|26
So as expected, most of the performance benefit is when you are paging deep, but there are also slight improvements across the board.
I plan on committing soon.
> Use a native priority queue to order facet results
> --------------------------------------------------
>
> Key: SOLR-2092
> URL: https://issues.apache.org/jira/browse/SOLR-2092
> Project: Solr
> Issue Type: Improvement
> Reporter: Yonik Seeley
> Attachments: SOLR-2092.patch, SOLR-2092.patch
>
>
> Deep paging into facets can be improved via use of a native priority queue.
> Comparisons will be faster, and there will be less GC.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (SOLR-2092) Use a native priority queue to order
facet results
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated SOLR-2092:
-------------------------------
Attachment: SOLR-2092.patch
Here's an update that adds the long pq to the single-valued fc method.
It also uses the missing count from calculated like all other counts (from the field cache).
> Use a native priority queue to order facet results
> --------------------------------------------------
>
> Key: SOLR-2092
> URL: https://issues.apache.org/jira/browse/SOLR-2092
> Project: Solr
> Issue Type: Improvement
> Reporter: Yonik Seeley
> Attachments: SOLR-2092.patch, SOLR-2092.patch
>
>
> Deep paging into facets can be improved via use of a native priority queue.
> Comparisons will be faster, and there will be less GC.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Resolved: (SOLR-2092) Use a native priority queue to order
facet results
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley resolved SOLR-2092.
--------------------------------
Fix Version/s: 4.0
Resolution: Fixed
> Use a native priority queue to order facet results
> --------------------------------------------------
>
> Key: SOLR-2092
> URL: https://issues.apache.org/jira/browse/SOLR-2092
> Project: Solr
> Issue Type: Improvement
> Reporter: Yonik Seeley
> Fix For: 4.0
>
> Attachments: SOLR-2092.patch, SOLR-2092.patch
>
>
> Deep paging into facets can be improved via use of a native priority queue.
> Comparisons will be faster, and there will be less GC.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (SOLR-2092) Use a native priority queue to order
facet results
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated SOLR-2092:
-------------------------------
Attachment: SOLR-2092.patch
Here's a draft patch with only UnInvertedField converted from using BoundedTreeSet to LongPriorityQueue so far. I haven't performance tested this yet, but I imagine ord->term lookup will remain the dominant cost of large facet.limit objects. Still, this should help cut down the maximum memory footprint too.
I also managed to reuse the internal array from the priority queue (so now both arrays used to sort term ords are reused from previously allocated arrays).
> Use a native priority queue to order facet results
> --------------------------------------------------
>
> Key: SOLR-2092
> URL: https://issues.apache.org/jira/browse/SOLR-2092
> Project: Solr
> Issue Type: Improvement
> Reporter: Yonik Seeley
> Attachments: SOLR-2092.patch
>
>
> Deep paging into facets can be improved via use of a native priority queue.
> Comparisons will be faster, and there will be less GC.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org