You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "YufeiLiu (Jira)" <ji...@apache.org> on 2020/02/08 14:29:00 UTC

[jira] [Created] (FLINK-15959) Add TaskExecutor number option in FlinkYarnSessionCli

YufeiLiu created FLINK-15959:
--------------------------------

             Summary: Add TaskExecutor number option in FlinkYarnSessionCli
                 Key: FLINK-15959
                 URL: https://issues.apache.org/jira/browse/FLINK-15959
             Project: Flink
          Issue Type: New Feature
          Components: Runtime / Coordination
    Affects Versions: 1.11.0
            Reporter: YufeiLiu


Flink removed `-n` option after FLIP-6, change to ResourceManager start a new worker when required. But I think maintain a TaskExecutor number option is necessary. These workers will start immediately when ResourceManager starts and would not release even if all slots are free.
Here are some resons:
# Users actually how many resources are needed when run a single job, initialize all workers when cluster starts can speed up startup process.
#  Job schedule in  topology order,  next operator won't schedule until prior execution slot allocated. The TaskExecutors will start in several batchs in some cases, it might slow down the startup speed.
# Flink support [FLINK-12122|https://issues.apache.org/jira/browse/FLINK-12122] [Spread out tasks evenly across all available registered TaskManagers], but it will only effect if all TMs are registered. Start all TMs at begining can slove this problem.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)