You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Greg Hogan (JIRA)" <ji...@apache.org> on 2018/03/26 15:06:00 UTC
[jira] [Commented] (FLINK-8809) Decrease maximum value of
DirectMemory at default config
[ https://issues.apache.org/jira/browse/FLINK-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413978#comment-16413978 ]
Greg Hogan commented on FLINK-8809:
-----------------------------------
As you have noted this maximum value is set only for [MaxDirectMemorySize|https://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html] which constrains the "the maximum total size (in bytes) of the New I/O (the java.nio package) direct-buffer allocations". Flink's allocation of memory segments is controlled by its configuration so there is no need to constrain this value in the JVM.
What would you choose as a reasonable default value? Requiring some users to increase this value is a [DRY|https://en.wikipedia.org/wiki/Don%27t_repeat_yourself] anti-pattern.
So I think the explanation to be: there is no harm in setting this to an essentially "infinite" value, and no benefit to setting a lower value.
> Decrease maximum value of DirectMemory at default config
> --------------------------------------------------------
>
> Key: FLINK-8809
> URL: https://issues.apache.org/jira/browse/FLINK-8809
> Project: Flink
> Issue Type: Bug
> Components: TaskManager
> Reporter: Kirill A. Korinskiy
> Priority: Major
>
> Good day!
>
> Have I can see since this [commit|https://github.com/apache/flink/commit/6c44d93d0a9da725ef8b1ad2a94889f79321db73] TaskManager uses 8,388,607 terabytes as maximum out of heap memory. I guess that not any system has so much memory and it may be a reason to kill java process by OOM Killer.
>
> I suggest to decrease this value to reasonable value by default.
>
> Right now I see only one way to overstep this hardcoded value: setup FLINK_TM_HEAP to 0, and specified heap size by hand over FLINK_ENV_JAVA_OPTS_TM.
> Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)