You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "Ritik Raj (Jira)" <ji...@apache.org> on 2024/05/03 21:01:00 UTC

[jira] [Created] (ASTERIXDB-3394) Introducing concurrent size bound merge policy

Ritik Raj created ASTERIXDB-3394:
------------------------------------

             Summary: Introducing concurrent size bound merge policy
                 Key: ASTERIXDB-3394
                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-3394
             Project: Apache AsterixDB
          Issue Type: Improvement
          Components: STO - Storage
            Reporter: Ritik Raj


Introducing size-bound merge policy to account for cloud service providers' max blob storage file size and to prevent exceeding that specified limit.


 - Add storage config to set max mergable component size.
 - Add a new merge policy and make it our default.
 - The new merge policy will schedule merges similar to the current ConcurrentMergePolicy but it will consider the max mergable component size.
 - Update the isMergeLagging check on max components to use the "effective" meragable components which can actually be merged rather than the current total number of disk components.
 - In case of max components is reached and no merges could be scheduled, force a merge on the effective meragable components.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)