You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "RocMarshal (Jira)" <ji...@apache.org> on 2023/08/02 03:54:00 UTC

[jira] [Commented] (FLINK-31757) Optimize Flink un-balanced tasks scheduling

    [ https://issues.apache.org/jira/browse/FLINK-31757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750078#comment-17750078 ] 

RocMarshal commented on FLINK-31757:
------------------------------------

Hi, [~heigebupahei] Thanks for your attention. we've updated the new edition design docs in [https://docs.google.com/document/d/14WhrSNGBdcsRl3IK7CZO-RaZ5KXU2X1dWqxPEFr3iS8/edit?usp=sharing]

Would you mind having a look on the doc ? Any suggestion is appreciated~

> Optimize Flink un-balanced tasks scheduling
> -------------------------------------------
>
>                 Key: FLINK-31757
>                 URL: https://issues.apache.org/jira/browse/FLINK-31757
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Coordination
>            Reporter: RocMarshal
>            Assignee: RocMarshal
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: image-2023-04-13-08-04-04-667.png
>
>
> Supposed we have a Job with 21 {{JobVertex}}. The parallelism of vertex A is 100, and the others are 5. If each {{TaskManager}} only have one slot, then we need 100 TMs.
> There will be 5 slots with 21 sub-tasks, and the others will only have one sub-task of A. Does this mean we have to make a trade-off between wasted resources and insufficient resources?
> From a resource utilization point of view, we expect all subtasks to be evenly distributed on each TM.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)