You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Zhu Zhu (Jira)" <ji...@apache.org> on 2019/12/03 13:03:00 UTC
[jira] [Updated] (FLINK-15037) Introduce LimittingMemoryManager as
operator scope MemoyManager
[ https://issues.apache.org/jira/browse/FLINK-15037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhu Zhu updated FLINK-15037:
----------------------------
Description:
Current {{MemoryManager}} is slot scope component, and the operators needs to use a fraction to compute memory size/pages it can allocate and use it to reserver memory or allocate pages.
This, however, requires operators to be aware of the managed memory fraction. There is also risk that one operator may over allocate resources than it has declared and causes other operators to break.
To separate concerns, we can introduce a {{LimittingMemoryManager}} which wraps the the original MemoryManager but limit the available memory size to the fraction of total memory governed by the original one. This wrapper would be a operator scope {{MemoryManager}}.
Some more detailed discussions can be found in FLINK-14936.
cc [~azagrebin]
was:
Current {{MemoryManager}} is slot scope component, and the operators needs to use a fraction to compute memory size/pages it can allocate and use it to reserver memory or allocate pages.
This, however, requires operators to be aware of the managed memory fraction. There is also risk that one operator may over allocate resources than it has declared and causes other operators to break.
To separate concerns, we can introduce a {{LimittingMemoryManager}} which wraps the the original MemoryManager but limit the available memory size to the fraction of total memory governed by the original one. This wrapper would be a operator scope {{MemoryManager}}.
cc [~azagrebin]
> Introduce LimittingMemoryManager as operator scope MemoyManager
> ---------------------------------------------------------------
>
> Key: FLINK-15037
> URL: https://issues.apache.org/jira/browse/FLINK-15037
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.10.0
> Reporter: Zhu Zhu
> Priority: Major
>
> Current {{MemoryManager}} is slot scope component, and the operators needs to use a fraction to compute memory size/pages it can allocate and use it to reserver memory or allocate pages.
> This, however, requires operators to be aware of the managed memory fraction. There is also risk that one operator may over allocate resources than it has declared and causes other operators to break.
> To separate concerns, we can introduce a {{LimittingMemoryManager}} which wraps the the original MemoryManager but limit the available memory size to the fraction of total memory governed by the original one. This wrapper would be a operator scope {{MemoryManager}}.
> Some more detailed discussions can be found in FLINK-14936.
> cc [~azagrebin]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)