[jira] [Commented] (LUCENE-8687) Optimise radix partitioning for points on heap

["Ignacio Vera (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16763375#comment-16763375 ] 

Ignacio Vera commented on LUCENE-8687:
--------------------------------------

I have opened a PR for this approach. Performance test shows a nice increase in indexing throughput:

||Approach||Index time (sec): Dev||Index Time (sec): Base||Index Time: Diff||Force merge time (sec): Dev||Force Merge time (sec): Base||Force Merge Time: Diff||Index size (GB): Dev||Index size (GB): Base||Index Size: Diff||Reader heap (MB): Dev||Reader heap (MB): Base||Reader heap: Diff||
|points|148.4s|178.9s|-17%|73.6s|92.2s|-20%|0.55|0.55| 0%|1.57|1.57| 0%|
|shapes|246.6s|283.7s|-13%|148.5s|168.2s|-12%|1.29|1.29| 0%|1.61|1.61| 0%|
|geo3d|180.9s|204.8s|-12%|79.8s|103.7s|-23%|0.75|0.75| 0%|1.58|1.58| 0%|

> Optimise radix partitioning for points on heap
> ----------------------------------------------
>
>                 Key: LUCENE-8687
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8687
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Ignacio Vera
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> In LUCENE-8673 it was introduced radix partitioning for merging segments. It currently works the same when you have data offline and or heap. It makes sense when data is on-heap, to not have multiple copies but perform the partitioning always in the same object, similar to what it is done with `MutablePointValues`. 
> This will allow as well to hold more points in memory.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org