You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "huweihua (Jira)" <ji...@apache.org> on 2020/02/18 16:08:00 UTC
[jira] [Comment Edited] (FLINK-16069) Creation of
TaskDeploymentDescriptor can block main thread for long time
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039191#comment-17039191 ]
huweihua edited comment on FLINK-16069 at 2/18/20 4:07 PM:
-----------------------------------------------------------
Hi, [~trohrmann], thanks for your time. Yes, The iteration over the input edges taking so long.
I didn't think too much about race condition, and i tried create the TaskDeploymentDescriptorFactory in the main thread, then put createDeploymentDescriptor into a future. This reduces the time taken in the main thread to 2s.
Glad to receive any suggestions.
was (Author: huwh):
Hi, [~trohrmann], thanks for your time. Yes, The iteration over the input edges taking so long.
I didn't think too much about race condition, and i tried create the TaskDeploymentDescriptorFactory in the main thread, then put createDeploymentDescriptor into a future. This reduces the time taken in the main thread to 2s.
Glad to receive any suggestions.
[有道词典|http://fanyi.youdao.com/translate?i=Sorry%20for%20not%20thinking%20about%20race%20condition.%20I%20am%20glad%20to&keyfrom=chrome]
Sorry for not t ...[详细|http://fanyi.youdao.com/translate?i=Sorry%20for%20not%20thinking%20about%20race%20condition.%20I%20am%20glad%20to&smartresult=dict&keyfrom=chrome.extension]X
对不起没有考虑竞争条件。我很高兴
> Creation of TaskDeploymentDescriptor can block main thread for long time
> ------------------------------------------------------------------------
>
> Key: FLINK-16069
> URL: https://issues.apache.org/jira/browse/FLINK-16069
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Reporter: huweihua
> Priority: Major
>
> The deploy of tasks will take long time when we submit a high parallelism job. And Execution#deploy run in mainThread, so it will block JobMaster process other akka messages, such as Heartbeat. The creation of TaskDeploymentDescriptor take most of time. We can put the creation in future.
> For example, A job [source(8000)->sink(8000)], the total 16000 tasks from SCHEDULED to DEPLOYING took more than 1mins. This caused the heartbeat of TaskManager timeout and job never success.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)