[jira] [Resolved] (YARN-11669) [Umbrella] cgroup v2 support

["Benjamin Teke (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-11669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Benjamin Teke resolved YARN-11669.
----------------------------------
    Fix Version/s: 3.5.0
     Hadoop Flags: Reviewed
       Resolution: Fixed

> [Umbrella] cgroup v2 support
> ----------------------------
>
>                 Key: YARN-11669
>                 URL: https://issues.apache.org/jira/browse/YARN-11669
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: yarn
>            Reporter: Ferenc Erdelyi
>            Assignee: Benjamin Teke
>            Priority: Major
>             Fix For: 3.5.0
>
>
> cgroup v2 has some fundamental changes compared to v1. RHEL9, Ubuntu 22 already moved to cgroup v2 as a default, hence YARN should support it. This umbrella tracks the required work.
> [Documentation|https://docs.kernel.org/admin-guide/cgroup-v2.html]
> A way to test the newly added features:
> # Turn on cgroup v1 based on the current [documentation|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html].
> # System prerequisites:
> ## the file {{/etc/mtab}} should contain a mount path with the file system type {{cgroup2}}, by default this could be {{/sys/fs/cgroup}} on most OS's
> ## the {{cgroup.subtree_control}} file should contain the necessary controllers (update it with: {{echo "+cpu +io +memory" > cgroup.subtree_control}})
> ## either create the YARN hierarchy and give recursive access to the user running the NM on the node. The hierarchy is {{hadoop-yarn}} by default (controller by {{yarn.nodemanager.linux-container-executor.cgroups.hierarchy}}), and recursive mode is required, because as soon as the directory is created it will be filled with the controller files which YARN will try to edit.
> ### Alternatively if the NM process user has access rights on the {{/sys/fs/cgroup}} directory it'll try to create the hierarchy and update the {{cgroup.subtree_control}} file.
> # YARN configuration
> ## {{yarn.nodemanager.linux-container-executor.cgroups.mount-path}} should point to the directory where the cgroup2 structure is mounted and the {{hadoop-yarn}} hierarchy was created
> ## {{yarn.nodemanager.linux-container-executor.cgroups.v2.enabled}} should be set to {{true}}
> ## Enable a cgroup controller, like {{yarn. nodemanager. resource. cpu.enabled}}: {{true}}
> # Launch the NM and monitor the cgroup files on container launches (i.e: {{/sys/fs/cgroup/hadoop-yarn/container_id/cpu.weight}})



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org