[jira] [Created] (YARN-9371) Better logging for initialization of cgroups in CGroupsHandlerImpl

["Szilard Nemeth (JIRA)" <ji...@apache.org>]

Szilard Nemeth created YARN-9371:
------------------------------------

             Summary: Better logging for initialization of cgroups in CGroupsHandlerImpl
                 Key: YARN-9371
                 URL: https://issues.apache.org/jira/browse/YARN-9371
             Project: Hadoop YARN
          Issue Type: Improvement
            Reporter: Szilard Nemeth


We already have YARN-9098 that dealt with (not yet merged, though) code duplication in cgroups setup code.
However, there's still room for improvement in the area of cgroups initialization, especially in CGroupsHandlerImpl.

1. org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsHandlerImpl#initializeControllerPaths: There's a decision in this method: Either cgroups are parsed from the configured mount path or from the mounted cgroups descriptor file (/proc/mounts). This decisions needs to be logged, along with some details, e.g. cgroups mount path.

2. In ResourceHandlerModule#parseConfiguredCGroupPath: Every candidate file could be logged to debug. Moreover, the returned pathSubsystemMappings could be also logged to info.

3. In CGroupsHandlerImpl#parseMtab: This method iterates over lines of the mtab file, tries to match the line and parses data of cgroups. 
We need to better have logs for: 
- If the line does not match: Print the line (I think debug level is fine for that)
- We could log the entire result map to info log.

4. In CGroupsHandlerImpl#initializeControllerPaths, initializeControllerPathsFromMtab is called. This method returns mapping of cgroups controllers (cpu, memory, devices, etc.) to file system paths. This is an important piece of information for troubleshooting, so the map needs to be logged to info.

5. In CGroupsHandlerImpl#initializeCGroupController: There's a condition for enableCGroupMount, I think it's worth to log if we are dealing with YARN-driven cgroups mount or pre-mounted controllers.

6. In CGroupsHandlerImpl#mountCGroupController: 
- existingMountPath needs to be logged for better troubleshooting
- Other debug logs could also be added, this is up to the owner of this jira.

7. In CGroupsHandlerImpl#initializePreMountedCGroupController: 
- controllerPath needs to be logged for better troubleshooting.
- Other debug logs could also be added, this is up to the owner of this jira.

 

A side-note: Implementing this Jira could be way easier after YARN-9098 is merged as that Jira eliminated lots of code duplication and made the initialization of cgroups more clear and concise.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org