You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-dev@hadoop.apache.org by "Nathan Roberts (JIRA)" <ji...@apache.org> on 2016/03/04 20:56:40 UTC

[jira] [Created] (YARN-4768) getAvailablePhysicalMemorySize can be inaccurate on linux

Nathan Roberts created YARN-4768:
------------------------------------

             Summary: getAvailablePhysicalMemorySize can be inaccurate on linux
                 Key: YARN-4768
                 URL: https://issues.apache.org/jira/browse/YARN-4768
             Project: Hadoop YARN
          Issue Type: Bug
          Components: nodemanager
    Affects Versions: 2.7.2, 3.0.0
         Environment: Linux
            Reporter: Nathan Roberts
            Assignee: Nathan Roberts


Algorithm currently uses "MemFree" + "Inactive" from /proc/meminfo

"Inactive" may not be a very good indication of how much memory can be readily freed because it contains both:
- Pages mapped with MAP_SHARED|MAP_ANONYMOUS (regardless of whether they're being actively accessed or not. Unclear to me why this is the case...)
- Pages mapped MAP_PRIVATE|MAP_ANONYMOUS that have not been accessed recently

Both of these types of pages probably shouldn't be considered "Available".

"Inactive(file)" would seem more accurate but it's not available in all kernel versions. To keep things simple, maybe just use "Inactive(file)" if available, otherwise fallback to "Inactive".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)