[jira] [Created] (LUCENE-7523) UpgradeIndexMergePolicy: beyond one-off use, monster segment avoidance

["Christine Poerschke (JIRA)" <ji...@apache.org>]

Christine Poerschke created LUCENE-7523:
-------------------------------------------

             Summary: UpgradeIndexMergePolicy: beyond one-off use, monster segment avoidance
                 Key: LUCENE-7523
                 URL: https://issues.apache.org/jira/browse/LUCENE-7523
             Project: Lucene - Core
          Issue Type: Task
            Reporter: Christine Poerschke
            Priority: Minor


(Was looking at UpgradeIndexMergePolicy as part of SOLR-9648 and came up with these possibilities here, what do people think?)

Currently one probably would not configure use of the {{UpgradeIndexMergePolicy}} (UIMP) permanently since [findForcedMerges|https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/index/UpgradeIndexMergePolicy.java#L74] becomes a no-op after all segments have been upgraded.
* How about adding an optional {{fallbackToInnerAfterUpgrade}} flag? That way UIMP.findForcedMerges could fallback to its inner/delegate merge policy's findForcedMerges call after all segments have been upgraded.

Currently UIMP.findForcedMerges identifies all the segments to be upgraded and then asks its inner/delegate merge policy to come up with a MergeSpecification for those segments. If the inner/delegate merge policy does not supply a merge for all the segments to be upgraded then UIMP merges the remaining segments into _one_ new segment. That extra new segment could be quite a large 'monster' segment.
* How about adding an optional {{upgradeUnmergedSegmentsIndividually}} flag? That way UIMP.findForcedMerges could upgrade (but not merge) the remaining segments.
* Or indeed should 'upgradeUnmergedSegmentsIndividually' be the default behaviour?

Noticed whilst looking at the code:
* [UIMP.findMerges|https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/index/UpgradeIndexMergePolicy.java#L69] does not pass the mergeTrigger to the inner/delegate merge policy.
** If we can figure out why that is, let's add a comment to say why that is.
** Understanding why that is would also be needed before proceeding with beyond one-off use of UIMP.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org