You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "huweihua (Jira)" <ji...@apache.org> on 2020/02/15 04:49:00 UTC

[jira] [Created] (FLINK-16069) Create TaskDeploymentDescriptor in future.

huweihua created FLINK-16069:
--------------------------------

             Summary: Create TaskDeploymentDescriptor in future.
                 Key: FLINK-16069
                 URL: https://issues.apache.org/jira/browse/FLINK-16069
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Task
            Reporter: huweihua


The deploy of tasks will took long time when we submit a high parallelism job. And Execution#deploy run in mainThread, so it will block JobMaster process other akka messages, such as Heartbeat. The creation of TaskDeploymentDescriptor take most of time. We can put the creation in future.

For example, A job [source(8000)->sink(8000)], the total 16000 tasks from SCHEDULED to DEPLOYING took more than 1mins. This caused the heartbeat of TaskManager timeout and job never success.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)