[jira] [Updated] (ASTERIXDB-3394) Introducing concurrent size bound merge policy

["Ian Maxon (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ASTERIXDB-3394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ian Maxon updated ASTERIXDB-3394:
---------------------------------
    Labels: triaged  (was: )

> Introducing concurrent size bound merge policy
> ----------------------------------------------
>
>                 Key: ASTERIXDB-3394
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-3394
>             Project: Apache AsterixDB
>          Issue Type: Improvement
>          Components: STO - Storage
>            Reporter: Ritik Raj
>            Priority: Major
>              Labels: triaged
>
> Introducing size-bound merge policy to account for cloud service providers' max blob storage file size and to prevent exceeding that specified limit.
>  - Add storage config to set max mergable component size.
>  - Add a new merge policy and make it our default.
>  - The new merge policy will schedule merges similar to the current ConcurrentMergePolicy but it will consider the max mergable component size.
>  - Update the isMergeLagging check on max components to use the "effective" meragable components which can actually be merged rather than the current total number of disk components.
>  - In case of max components is reached and no merges could be scheduled, force a merge on the effective meragable components.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)