You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Rui Fan (Jira)" <ji...@apache.org> on 2024/02/26 13:53:00 UTC
[jira] [Commented] (FLINK-34504) Avoid the parallelism adjustment when the upstream shuffle type doesn't have keyBy
[ https://issues.apache.org/jira/browse/FLINK-34504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17820708#comment-17820708 ]
Rui Fan commented on FLINK-34504:
---------------------------------
Merged to main(1.8.0) via: f4b04ec5f47358829ee279806d2f74b1f1641ee2
> Avoid the parallelism adjustment when the upstream shuffle type doesn't have keyBy
> ----------------------------------------------------------------------------------
>
> Key: FLINK-34504
> URL: https://issues.apache.org/jira/browse/FLINK-34504
> Project: Flink
> Issue Type: Improvement
> Components: Autoscaler
> Reporter: Rui Fan
> Assignee: Rui Fan
> Priority: Major
> Labels: pull-request-available
> Fix For: kubernetes-operator-1.8.0
>
>
> JobVertexScaler#scale has a optimization: Try to adjust the parallelism such that it divides the number of key groups without a remainder => data is evenly spread across subtasks.
>
> It's only useful when the upstream shuffle type has keyBy. We should avoid this optimization when the upstream shuffle type doesn't have keyBy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)