[jira] [Updated] (FLINK-32120) Add autoscaler config option to disable parallelism key group alignment

["Gyula Fora (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-32120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gyula Fora updated FLINK-32120:
-------------------------------
    Fix Version/s: kubernetes-operator-1.10.0
                       (was: kubernetes-operator-1.9.0)

> Add autoscaler config option to disable parallelism key group alignment
> -----------------------------------------------------------------------
>
>                 Key: FLINK-32120
>                 URL: https://issues.apache.org/jira/browse/FLINK-32120
>             Project: Flink
>          Issue Type: Improvement
>          Components: Autoscaler, Kubernetes Operator
>            Reporter: Maximilian Michels
>            Assignee: Maximilian Michels
>            Priority: Major
>             Fix For: kubernetes-operator-1.10.0
>
>
> After choosing the target parallelism for a vertex, we choose a higher parallelism if that parallelism leads to evenly spreading the number of key groups. The number of key groups is derived from the max parallelism.
> The amount of actual skew we would introduce if we did not do the alignment would usually be pretty low. In fact, the data itself can have an uneven load distribution across the keys (hot keys). In this case, the key group alignment is not effective.
> For experiments, we should allow disabling the key group alignment via a configuration option.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)