[jira] [Commented] (HADOOP-12180) Move ResourceCalculatorPlugin from YARN to Common

["Chris Douglas (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14613455#comment-14613455 ] 

Chris Douglas commented on HADOOP-12180:
----------------------------------------

Summary of changes:
- Created {{o.a.h.util.SysInfo}} with the same interface as {{ResourceCalculatorPlugin}}
- Changed {{ResourceCalculatorPlugin}} to delegate to a {{SysInfo}} instance, and the OS-specific RCP impls to pass the equivalent {{SysInfoLinux}} and {{SysInfoWindows}} classes to the (now concrete) parent class
- {{CpuTimeTracker}} also got carried to Common to avoid a dep on YARN; resolved some references to the process tree utility classes
- Fixed some of the unit tests and classes to satisfy findbugs/checkstyle rules, but the impls are essentially unchanged.

This should allow the YARN {{ResourceCalculatorPlugin}} classes to be configurable as before, including those overriding the API methods of OS-specific impls (of course, any extension relying on impl details of the RCs will break). This does not implement comparable pluggability for {{SysInfo::newInstance}}, which only does the same OS check as in the current code. I also elected not to extend {{Configured}} or implement {{Configurable}} for {{SysInfo}}, as none of the impls use it. If {{SysInfo}} should check the config, use a {{ServiceLoader}}, or some other method for selecting the "correct" impl: that could be follow-on work.

The {{SysInfoLinux}} impl (and extensions in YARN-3819, YARN-3820) all scrape {{/proc}}, so maybe {{SysInfoLinuxProc}} would be more apt (some applies to Windows impl), but absent other implementations and the prenominate config... I'm inclined to ignore it.

These implementions aren't threadsafe. If we're moving them to Common (from internal classes for YARN) that may be worth fixing, but I wanted to see what others thought first. It could also be a separate JIRA.

> Move ResourceCalculatorPlugin from YARN to Common
> -------------------------------------------------
>
>                 Key: HADOOP-12180
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12180
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>         Attachments: HADOOP-12180.000.patch, HADOOP-12180.001.patch
>
>
> Some of the monitoring functions could be moved from YARN to Common for easier sharing



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)