You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Till Rohrmann (Jira)" <ji...@apache.org> on 2021/01/29 11:10:00 UTC
[jira] [Commented] (FLINK-17061) Unset process/flink memory size
from configuration once dynamic worker resource is activated.
[ https://issues.apache.org/jira/browse/FLINK-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274338#comment-17274338 ]
Till Rohrmann commented on FLINK-17061:
---------------------------------------
Is this ticket relevant for FLIP-156 [~xintongsong]?
> Unset process/flink memory size from configuration once dynamic worker resource is activated.
> ---------------------------------------------------------------------------------------------
>
> Key: FLINK-17061
> URL: https://issues.apache.org/jira/browse/FLINK-17061
> Project: Flink
> Issue Type: Task
> Components: Runtime / Configuration, Runtime / Coordination
> Affects Versions: 1.11.0
> Reporter: Xintong Song
> Priority: Major
>
> With FLINK-14106, memory of a TaskExecutor is decided in two steps on active resource managers.
> - {{SlotManager}} decides {{WorkerResourceSpec}}, including memory used by Flink tasks: task heap, task off-heap, network and managed memory.
> - {{ResourceManager}} derives {{TaskExecutorProcessSpec}} from {{WorkerResourceSpec}} and the configuration, deciding sizes of memory used by Flink framework and JVM: framework heap, framework off-heap, jvm metaspace and jvm overhead.
> This works fine for now, because both {{WorkerResourceSpec}} and {{TaskExecutorProcessSpec}} are derived from the same configurations. However, it might cause problem if later we have new {{SlotManager}} implementations that decides {{WorkerResourceSpec}} dynamically. In such cases, the process/flink sizes in configuration should be ignored, or it may easily lead to configuration conflicts.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)