[jira] [Commented] (YARN-3122) Metrics for container's actual CPU usage

["Anubhav Dhoot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346006#comment-14346006 ] 

Anubhav Dhoot commented on YARN-3122:
-------------------------------------

Those changes look good for me.
For a sample when i run stress -c 3 using distributed shell on a 4 core machine and 8 vcores configured. This would mean 3 cores would be consumed which would map to  6 vcores. Thus PCpu would be approx 300% (similar to top) and MilliVcoresUsed would be  approx 6000 which is what we see below on the actual metrics on the NodeManager  

{noformat}
hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -debug -shell_command "stress -c 3" -jar ../share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -container_memory 350 -master_memory 350
{noformat}

{noformat}
  }, {
    "name" : "Hadoop:service=NodeManager,name=ContainerResource_container_1425421474415_0003_01_000002",
    "modelerType" : "ContainerResource_container_1425421474415_0003_01_000002",
    "tag.ContainerResource" : "container_1425421474415_0003_01_000002",
    "tag.Context" : "container",
    "tag.ContainerPid" : "10095",
    "tag.Hostname" : "anuonebox.ent.cloudera.com",
    "PMemUsageMBsNumUsage" : 23,
    "PMemUsageMBsAvgMBs" : 2.0,
    "PMemUsageMBsStdevMBs" : 0.0,
    "PMemUsageMBsIMinMBs" : 2.0,
    "PMemUsageMBsIMaxMBs" : 2.0,
    "PMemUsageMBsMinMBs" : 2.0,
    "PMemUsageMBsMaxMBs" : 2.0,
    "PCpuUsagePercentNumUsage" : 23,
    "PCpuUsagePercentAvgPercents" : 284.304347826087,
    "PCpuUsagePercentStdevPercents" : 62.196488341829514,
    "PCpuUsagePercentIMinPercents" : -1.0,
    "PCpuUsagePercentIMaxPercents" : 298.0,
    "PCpuUsagePercentMinPercents" : -1.0,
    "PCpuUsagePercentMaxPercents" : 298.0,
    "MilliVcoreUsageNumUsage" : 23,
    "MilliVcoreUsageAvgMilliVcores" : 5694.782608695651,
    "MilliVcoreUsageStdevMilliVcores" : 1245.8097752255082,
    "MilliVcoreUsageIMinMilliVcores" : -20.0,
    "MilliVcoreUsageIMaxMilliVcores" : 5971.0,
    "MilliVcoreUsageMinMilliVcores" : -20.0,
    "MilliVcoreUsageMaxMilliVcores" : 5971.0,
    "pMemLimitMBs" : 512,
    "vMemLimitMBs" : 1075,
    "vCoreLimit" : 1
  } ]
}
{noformat}

> Metrics for container's actual CPU usage
> ----------------------------------------
>
>                 Key: YARN-3122
>                 URL: https://issues.apache.org/jira/browse/YARN-3122
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 2.6.0
>            Reporter: Anubhav Dhoot
>            Assignee: Anubhav Dhoot
>         Attachments: YARN-3122.001.patch, YARN-3122.002.patch, YARN-3122.003.patch, YARN-3122.004.patch, YARN-3122.005.patch, YARN-3122.prelim.patch, YARN-3122.prelim.patch
>
>
> It would be nice to capture resource usage per container, for a variety of reasons. This JIRA is to track CPU usage. 
> YARN-2965 tracks the resource usage on the node, and the two implementations should reuse code as much as possible. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)