You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Adrien Grand (Jira)" <ji...@apache.org> on 2022/05/18 11:11:00 UTC

[jira] [Comment Edited] (LUCENE-10574) Remove O(n^2) from TieredMergePolicy or change defaults to one that doesn't do this

    [ https://issues.apache.org/jira/browse/LUCENE-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17538746#comment-17538746 ] 

Adrien Grand edited comment on LUCENE-10574 at 5/18/22 11:10 AM:
-----------------------------------------------------------------

I used BaseMergePolicyTestCase's simulation logic to run some tests with 10k docs, flushed 3 by 3 where each doc uses 10 bytes on disk:
|| ||TieredMergePolicy's defaults||TieredMergePolicy with floor segment size = Double.MIN_VALUE||TieredMergePolicy constrained to never produce merges where the overal size of the merge is not at least 50% larger than the biggest input segment||
|Write amplification|94.0|3.6|7.7|
|Average number of segments in the index|6.0|24.4|6.7|


was (Author: jpountz):
I used BaseMergePolicyTestCase's simulation logic to run some tests with 10k docs, flushed 3 by 3 where each doc uses 10 bytes on disk:

|| || TieredMergePolicy's default || TieredMergePolicy with floo segment size = Double.MIN_VALUE || TieredMergePolicy constrained to never produce merges where the overal size of the merge is not at least 50% larger than the biggest input segment ||
|Write amplification| 94.0 | 3.6 | 7.7 |
|Average number of segments in the index| 6.0 | 24.4 | 6.7 |

> Remove O(n^2) from TieredMergePolicy or change defaults to one that doesn't do this
> -----------------------------------------------------------------------------------
>
>                 Key: LUCENE-10574
>                 URL: https://issues.apache.org/jira/browse/LUCENE-10574
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>            Priority: Major
>
> Remove {{floorSegmentBytes}} parameter, or change lucene's default to a merge policy that doesn't merge in an O(n^2) way.
> I have the feeling it might have to be the latter, as folks seem really wed to this crazy O(n^2) behavior.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org