You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "Mark Robert Miller (Jira)" <ji...@apache.org> on 2021/09/29 16:05:00 UTC
[jira] [Commented] (SOLR-15666) Heavily reduce faceting ram usage
for many cases.
[ https://issues.apache.org/jira/browse/SOLR-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422219#comment-17422219 ]
Mark Robert Miller commented on SOLR-15666:
-------------------------------------------
Basic before and after gc picture for the very narrow faceting benchmark currently in:
Before
{noformat}
Benchmark (docCount) (facetCard) (facetCard2) (fm) (nodeCount) (numReplicas) (numShards) Mode Cnt Score Error Units
JsonFaceting.jsonFacet 500000 15000 3000 smart 2 1 4thrpt 5 13.075 ± 0.593 ops/s
JsonFaceting.jsonFacet:·async 500000 15000 3000 smart 2 1 4thrpt NaN ---
JsonFaceting.jsonFacet:·gc.alloc.rate 500000 15000 3000 smart 2 1 4thrpt 5 1233.493 ± 2651.035 MB/sec
JsonFaceting.jsonFacet:·gc.alloc.rate.norm 500000 15000 3000 smart 2 1 4thrpt 5 103587267.304 ± 222574550.680 B/op
JsonFaceting.jsonFacet:·gc.churn.G1_Eden_Space 500000 15000 3000 smart 2 1 4thrpt 5 1556.852 ± 368.148 MB/sec
JsonFaceting.jsonFacet:·gc.churn.G1_Eden_Space.norm 500000 15000 3000 smart 2 1 4thrpt 5 131303411.889 ± 30279661.952 B/op
JsonFaceting.jsonFacet:·gc.churn.G1_Old_Gen 500000 15000 3000 smart 2 1 4thrpt 5 0.330 ± 0.455 MB/sec
JsonFaceting.jsonFacet:·gc.churn.G1_Old_Gen.norm 500000 15000 3000 smart 2 1 4thrpt 5 27858.041 ± 38880.588 B/op
JsonFaceting.jsonFacet:·gc.churn.G1_Survivor_Space 500000 15000 3000 smart 2 1 4thrpt 5 2.988 ± 3.294 MB/sec
JsonFaceting.jsonFacet:·gc.churn.G1_Survivor_Space.norm 500000 15000 3000 smart 2 1 4thrpt 5 252175.421 ± 277771.041 B/op
JsonFaceting.jsonFacet:·gc.count 500000 15000 3000 smart 2 1 4thrpt 5 41.000 counts
JsonFaceting.jsonFacet:·gc.time 500000 15000 3000 smart 2 1 4thrpt 5 218.000 500000 15000 3000 smart 2 1 4thrpt 5 79.000{noformat}
After
{noformat}
Benchmark (docCount) (facetCard) (facetCard2) (fm) (nodeCount) (numReplicas) (numShards) Mode Cnt Score Error Units
JsonFaceting.jsonFacet 500000 15000 3000 smart 2 1 4thrpt 5 15.311 ± 0.385 ops/s
JsonFaceting.jsonFacet:·async 500000 15000 3000 smart 2 1 4thrpt NaN ---
JsonFaceting.jsonFacet:·gc.alloc.rate 500000 15000 3000 smart 2 1 4thrpt 5 357.791 ± 765.257 MB/sec
JsonFaceting.jsonFacet:·gc.alloc.rate.norm 500000 15000 3000 smart 2 1 4thrpt 5 25560743.208 ± 54651776.034 B/op
JsonFaceting.jsonFacet:·gc.churn.G1_Eden_Space 500000 15000 3000 smart 2 1 4thrpt 5 449.762 ± 27.717 MB/sec
JsonFaceting.jsonFacet:·gc.churn.G1_Eden_Space.norm 500000 15000 3000 smart 2 1 4thrpt 5 32363639.760 ± 2609924.751 B/op
JsonFaceting.jsonFacet:·gc.churn.G1_Old_Gen 500000 15000 3000 smart 2 1 4thrpt 5 0.061 ± 0.291 MB/sec
JsonFaceting.jsonFacet:·gc.churn.G1_Old_Gen.norm 500000 15000 3000 smart 2 1 4thrpt 5 4359.291 ± 20832.751 B/op
JsonFaceting.jsonFacet:·gc.churn.G1_Survivor_Space 500000 15000 3000 smart 2 1 4thrpt 5 0.815 ± 4.890 MB/sec
JsonFaceting.jsonFacet:·gc.churn.G1_Survivor_Space.norm 500000 15000 3000 smart 2 1 4thrpt 5 58042.222 ± 345702.114 B/op
JsonFaceting.jsonFacet:·gc.count 500000 15000 3000 smart 2 1 4thrpt 5 15.000 counts
JsonFaceting.jsonFacet:·gc.time 500000 15000 3000 smart 2 1 4thrpt 5 79.000{noformat}
> Heavily reduce faceting ram usage for many cases.
> -------------------------------------------------
>
> Key: SOLR-15666
> URL: https://issues.apache.org/jira/browse/SOLR-15666
> Project: Solr
> Issue Type: Sub-task
> Reporter: Mark Robert Miller
> Assignee: Mark Robert Miller
> Priority: Major
>
> Spinning this out of various efforts from the parallel segments search branches.
> Would require a bit more tuning and a much more extensive benchmark verification effort.
> This has performance impact as well given gc costs of faceting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org