[jira] [Created] (ASTERIXDB-2600) Introduce ConcurrentMergePolicy

["Chen Luo (JIRA)" <ji...@apache.org>]

Chen Luo created ASTERIXDB-2600:
-----------------------------------

             Summary: Introduce ConcurrentMergePolicy
                 Key: ASTERIXDB-2600
                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-2600
             Project: Apache AsterixDB
          Issue Type: Improvement
            Reporter: Chen Luo
            Assignee: Chen Luo


The default PrefixMergePolicy has a number of problems: it only schedules 1 merge at a time; it stops merging components when they are too large. Both problems will lead to undesirable performance behavior. This issue introduces a ConcurrentMergePolicy that fixes these problems.

It has the following four parameters:
* minComponentMergeCount: the minimum number of components to trigger a merge
* maxComponentMergeCount: the maximum number of components per merge
* maxComponentCount: the maximum number of components tolerated in total; when this number is reached, flush will be stopped
* sizeRatio: a merge is scheduled if the size of the oldest component <= sizeRatio * total size of youngest components. A larger size ratio(>1) implies fewer merges, but this leads to more components. A smaller size ratio (<1) implies more merges, but this improves query performance and space utilization.

Concurrent merges will be scheduled as well. Given a sequence of components ordered by newest to oldest, this policy first finds a longest prefix of this sequence so that no component is being merged. The merge decision will then be based on this prefix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)