[jira] [Updated] (FLINK-31608) Re-evaluate 'min-parallelism-increase' option

["Weijie Guo (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-31608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Weijie Guo updated FLINK-31608:
-------------------------------
    Fix Version/s: 2.0.0
                       (was: 1.20.0)

> Re-evaluate 'min-parallelism-increase' option
> ---------------------------------------------
>
>                 Key: FLINK-31608
>                 URL: https://issues.apache.org/jira/browse/FLINK-31608
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / Configuration, Runtime / Coordination
>            Reporter: Chesnay Schepler
>            Priority: Major
>             Fix For: 2.0.0
>
>
> This option was meant to prevent scale up operations where the benefit doesn't outweigh the cost, like scaling up to increase a single vertices parallelism by 1. Meanwhile, scale-down operations were always immediately executed, because they were always the result of a stopped TaskManager, causing the job to restart anyway.
> Now that users can change the requirements at will this has changed, and the expected behavior is overall undefined.
> We need to answer:
>  * should there be a dedicated option for limiting scale-down operations if the requirements were changed?
>  * should the min-parallelism-{*}increase{*} option be generalized to a min-parallelism-{*}change{*} option?
>  * How shall operations be handled that scale different vertices up or down at the same? So far the decision was made on the cumulative parallelism change, but in this case the parallelism distribution can change significantly while the cumulative change is 0.
>  * If a rescale operation was not applied due to these limits, should they be _eventually_ applied anyway (e.g., after a timeout)?
>  
> See [https://github.com/apache/flink/pull/22169#discussion_r1147447567] for the background.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)