You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2024/12/10 17:01:00 UTC

[jira] [Commented] (YARN-11743) Cgroup v2 support should fall back to v1 when there are no v2 controllers

    [ https://issues.apache.org/jira/browse/YARN-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17904583#comment-17904583 ] 

ASF GitHub Bot commented on YARN-11743:
---------------------------------------

p-szucs opened a new pull request, #7222:
URL: https://github.com/apache/hadoop/pull/7222

   Change-Id: Ia6e80c5e4273af71c42e6e5efe64c7c91769a36e
   
   <!--
     Thanks for sending a pull request!
       1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
       2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
   -->
   
   ### Description of PR
   
   
   ### How was this patch tested?
   
   
   ### For code changes:
   
   - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
   
   




> Cgroup v2 support should fall back to v1 when there are no v2 controllers
> -------------------------------------------------------------------------
>
>                 Key: YARN-11743
>                 URL: https://issues.apache.org/jira/browse/YARN-11743
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Benjamin Teke
>            Assignee: Peter Szucs
>            Priority: Major
>
> Cgroup v1/v2 mixed mode support was introduced in YARN-11692, however it does not support an edgecase where NM has the cgroup v2 support enabled (using {{yarn.nodemanager.linux-container-executor.cgroups.v2.enabled}} set to true), but there are only cgroup v1 controllers mounted. In larger clusters there is a chance that some part of the cluster is already on newer OSes with cgroup v2 as a default, and others are still using v1. 
> Currently trying to launch an NM with cgroup v2 support enabled will fail if there are no cgroup.controllers file present:
> {code:java}
> Failed to initialize controller paths! Exception: 
> java.io.IOException: No cgroup controllers file found in the directory specified: /var/lib/yarn-ce/cgroups
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsV2HandlerImpl.readControllersFile(CGroupsV2HandlerImpl.java:130)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsV2HandlerImpl.parsePreConfiguredMountPath(CGroupsV2HandlerImpl.java:101)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.AbstractCGroupsHandler.initializeControllerPaths(AbstractCGroupsHandler.java:133)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.AbstractCGroupsHandler.init(AbstractCGroupsHandler.java:107)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.AbstractCGroupsHandler.<init>(AbstractCGroupsHandler.java:103)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsV2HandlerImpl.<init>(CGroupsV2HandlerImpl.java:71)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsV2HandlerImpl.<init>(CGroupsV2HandlerImpl.java:83)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.ResourceHandlerModule.initializeCGroupV2Handler(ResourceHandlerModule.java:106)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.ResourceHandlerModule.initializeCGroupHandlers(ResourceHandlerModule.java:83)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.ResourceHandlerModule.initCGroupsCpuResourceHandler(ResourceHandlerModule.java:177)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.ResourceHandlerModule.initializeConfiguredResourceHandlerChain(ResourceHandlerModule.java:334)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.ResourceHandlerModule.getConfiguredResourceHandlerChain(ResourceHandlerModule.java:383)
> 	at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:314)
> 	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:427)
> 	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> 	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:974)
> 	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:1054)
> {code}
> Basically [readControllersFile|https://github.com/apache/hadoop/blob/950b2ff773fa828eb13bed7c3fe6b3d52c7fff18/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsV2HandlerImpl.java#L127]'s thrown error should be handled if the required controllers are mounted in v1:
> {code:java}
>   /**
>    * Parse the cgroup v2 controllers file (cgroup.controllers) to check the enabled controllers.
>    * @param cgroupPath path to the cgroup directory
>    * @return set of enabled and YARN supported controllers.
>    * @throws IOException if the file is not found or cannot be read
>    */
>   public Set<String> readControllersFile(String cgroupPath) throws IOException {
>     File cgroupControllersFile = new File(cgroupPath + Path.SEPARATOR + CGROUP_CONTROLLERS_FILE);
>     if (!cgroupControllersFile.exists()) {
>       throw new IOException("No cgroup controllers file found in the directory specified: " +
>               cgroupPath);
>     }
>     String enabledControllers = FileUtils.readFileToString(cgroupControllersFile,
>         StandardCharsets.UTF_8);
>     Set<String> validCGroups = getValidCGroups();
>     Set<String> controllerSet =
>             new HashSet<>(Arrays.asList(enabledControllers.split(" ")));
>     // Collect the valid subsystem names
>     controllerSet.retainAll(validCGroups);
>     if (controllerSet.isEmpty()) {
>       LOG.warn("The following cgroup directory doesn't contain any supported controllers: " +
>               cgroupPath);
>     }
>     return controllerSet;
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org