[jira] [Issue Comment Deleted] (MESOS-5342) CPU pinning/binding support for CgroupsCpushareIsolatorProcess

["Chris (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MESOS-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris updated MESOS-5342:
-------------------------
    Comment: was deleted

(was: implementation has been posted to review board. successfully passed 'make check'. requires use of the hwloc library to perform machine hardware topology discovery and cpu binding. updates were made to configure.ac and Makefile.am.

implementation is a new "device" under the cgroups isolator directory called "hwloc". Implementation detects topology, computes total number of cores required by the container (also checks if the container requires gpu). if the container requires gpu, the topology information is used to find the "closest" cores based on latency. If the container only requires cpu, a histogram of task assignment to cores is checked. If the histogram is "empty" (all cores have a value of 1.0) then a random core is selected and the latency matrix is used to find cores that are "closest" to the random core. The histogram is updated. If the histogram is "not empty" then a greedy submodular subset selection algorithm is used to select N cores using the latency matrix and a "per-core" cost value. The "per-core" cost value is a normalized version of the histogram divided by the number of processing units available on each core.  Greedy submodular subset selection algorithms use a "diminishing returns property" to find an optimal subset of items under a knapsack constraint.

When the list of cores is returned, a bit vector representing a cpuset is bound to the container's pid_t. When the container is cleaned up, the histogram is updated by reducing the current task counts on each core assigned to the pid_t by -1.0.)

> CPU pinning/binding support for CgroupsCpushareIsolatorProcess
> --------------------------------------------------------------
>
>                 Key: MESOS-5342
>                 URL: https://issues.apache.org/jira/browse/MESOS-5342
>             Project: Mesos
>          Issue Type: Improvement
>          Components: cgroups, containerization
>    Affects Versions: 0.28.1
>            Reporter: Chris
>
> The cgroups isolator currently lacks support for binding (also called pinning) containers to a set of cores. The GNU/Linux kernel is known to make sub-optimal core assignments for processes and threads. Poor assignments impact program performance, particularly in the case of applications requiring GPU resources. 
> Most cluster management systems from the HPC community (SLURM) provide both cgroup isolation and cpu binding. This feature would provide similar capabilities. The current interest in supporting Intel's Cache Allocation Technology will require making choices about where container's are going to run on the mesos-agent's processor(s) - this feature is a step toward developing a robust solution.
> The improvement in this JIRA ticket will handle hardware topology detection, track container-to-core utilization in a histogram, and use a mathematical optimization technique to select cores for container assignment based on latency and the container-to-core utilization histogram.
> For GPU tasks, the improvement will prioritize selection of cores based on latency between the GPU and cores in an effort to minimize copy latency.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)