You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "chenlf (JIRA)" <ji...@apache.org> on 2018/10/16 11:58:00 UTC
[jira] [Updated] (FLINK-10564) tm costs too much time when
communicating with jm
[ https://issues.apache.org/jira/browse/FLINK-10564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
chenlf updated FLINK-10564:
---------------------------
Environment:
configs are following:
jm
high-availability zookeeper
taskmanager.heap.mb 16384
taskmanager.memory.preallocate false
taskmanager.numberOfTaskSlots 64
tm
slots 128
free slots 0-128
cpu core 40
Physical Memory 95gb
free Memory 32gb-50gb
Flink Managed Memory 22gb-35gb
Description:
it works fine until the number of tasks is above about 400.
There are 600+ tasks(each task handles billion data) running in our cluster now,and the problem is it costs too much time (even time out)when submiting/canceling/querying a task.
Recouses like memory,cpu are on normal level.
Component/s: TaskManager
JobManager
Core
Summary: tm costs too much time when communicating with jm (was: tm costs too much time to communicate jm)
> tm costs too much time when communicating with jm
> --------------------------------------------------
>
> Key: FLINK-10564
> URL: https://issues.apache.org/jira/browse/FLINK-10564
> Project: Flink
> Issue Type: Bug
> Components: Core, JobManager, TaskManager
> Environment: configs are following:
> jm
> high-availability zookeeper
> taskmanager.heap.mb 16384
> taskmanager.memory.preallocate false
> taskmanager.numberOfTaskSlots 64
> tm
> slots 128
> free slots 0-128
> cpu core 40
> Physical Memory 95gb
> free Memory 32gb-50gb
> Flink Managed Memory 22gb-35gb
> Reporter: chenlf
> Priority: Major
>
> it works fine until the number of tasks is above about 400.
> There are 600+ tasks(each task handles billion data) running in our cluster now,and the problem is it costs too much time (even time out)when submiting/canceling/querying a task.
> Recouses like memory,cpu are on normal level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)