[jira] [Commented] (YARN-8930) CGroup-based strict container memory enforcement does not work with CGroupElasticMemoryController

["Haibo Chen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659761#comment-16659761 ] 

Haibo Chen commented on YARN-8930:
----------------------------------

The patch allows the polling-based memory check to kick in, when the strict-memory-enforcement with cgroup and elastic memory control are both enabled.

When elastic memory control is disabled and cgroup-based strict memory enforcement is enabled, the oom-killer is enabled on individual container cgroup,

the kernel would automatically invoke the oom killer to handle any container that runs out of memory limit, so check the under_oom status is unnecessary.

> CGroup-based strict container memory enforcement does not work with CGroupElasticMemoryController
> -------------------------------------------------------------------------------------------------
>
>                 Key: YARN-8930
>                 URL: https://issues.apache.org/jira/browse/YARN-8930
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 3.2.0
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>            Priority: Major
>         Attachments: YARN-8930.00.patch
>
>
> When yarn.nodemanger.resource.memory.enforced is set to true with memory cgroup turned on, (aka strict memory enforcement), containers monitor relies on the under_oom status read from the container cgroup's memory.oom_control file.
> However, when the root yarn container cgroup is under oom (e.g. when the node is overallocating iteself), the under_oom status is set for all yarn containers regardless of whether each individual container has run over its memory limit.
> What essentially happens is that whenever the root cgroup is under oom, all yarn containers are killed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org