mesos git commit: Add the 'resources' field back to DRFSorter::Total.

[ya...@apache.org]

Repository: mesos
Updated Branches:
  refs/heads/master 1364aa561 -> e7fcc4fbc


Add the 'resources' field back to DRFSorter::Total.

This is a partial reversal of 5a63f2a5d2a5fe0e5315a5b8f79b75b99a6a5893.
We used to have this field but it turned out to be unnecessary for the
current use cases so it was removed. We however will need it for the
ongoing work to support tasks using shared resources.

The details can be found in https://reviews.apache.org/r/45961 but in
essence, we now know of a case where the removal of resource quantities
from the sorter depends on the identity. e.g., To determine if the
quantity of a shared resource can be removed from the sorter we need to
know if there are no longer instances of **the same** (hence identity)
shared resource in the sorter.

Review: https://reviews.apache.org/r/51470/


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/e7fcc4fb
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/e7fcc4fb
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/e7fcc4fb

Branch: refs/heads/master
Commit: e7fcc4fbc09122480c8f2549a8af099769e888c0
Parents: 1364aa5
Author: Jiang Yan Xu <xu...@apple.com>
Authored: Tue Aug 30 16:24:22 2016 -0700
Committer: Jiang Yan Xu <xu...@apple.com>
Committed: Tue Aug 30 16:26:03 2016 -0700

----------------------------------------------------------------------
 src/master/allocator/sorter/drf/sorter.cpp | 10 ++++++++++
 src/master/allocator/sorter/drf/sorter.hpp |  8 +++++---
 2 files changed, 15 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/mesos/blob/e7fcc4fb/src/master/allocator/sorter/drf/sorter.cpp
----------------------------------------------------------------------
diff --git a/src/master/allocator/sorter/drf/sorter.cpp b/src/master/allocator/sorter/drf/sorter.cpp
index ac85b32..1f2954c 100644
--- a/src/master/allocator/sorter/drf/sorter.cpp
+++ b/src/master/allocator/sorter/drf/sorter.cpp
@@ -285,6 +285,7 @@ void DRFSorter::unallocated(
 void DRFSorter::add(const SlaveID& slaveId, const Resources& resources)
 {
   if (!resources.empty()) {
+    total_.resources[slaveId] += resources;
     total_.scalarQuantities += resources.createStrippedScalarQuantity();
 
     // We have to recalculate all shares when the total resources
@@ -299,6 +300,15 @@ void DRFSorter::add(const SlaveID& slaveId, const Resources& resources)
 void DRFSorter::remove(const SlaveID& slaveId, const Resources& resources)
 {
   if (!resources.empty()) {
+    CHECK(total_.resources.contains(slaveId));
+    CHECK(total_.resources[slaveId].contains(resources));
+
+    total_.resources[slaveId] -= resources;
+
+    if (total_.resources[slaveId].empty()) {
+      total_.resources.erase(slaveId);
+    }
+
     const Resources resourcesQuantity =
       resources.createStrippedScalarQuantity();
 

http://git-wip-us.apache.org/repos/asf/mesos/blob/e7fcc4fb/src/master/allocator/sorter/drf/sorter.hpp
----------------------------------------------------------------------
diff --git a/src/master/allocator/sorter/drf/sorter.hpp b/src/master/allocator/sorter/drf/sorter.hpp
index bc6bfb2..09aa685 100644
--- a/src/master/allocator/sorter/drf/sorter.hpp
+++ b/src/master/allocator/sorter/drf/sorter.hpp
@@ -150,9 +150,11 @@ private:
 
   // Total resources.
   struct Total {
-    // NOTE: We do not need to track the total resources at each
-    // slave; instead, we can safely aggregate scalar resources across
-    // slaves, which substantially improves performance (MESOS-2891).
+    hashmap<SlaveID, Resources> resources;
+
+    // NOTE: Scalars can be safely aggregated across slaves. We keep
+    // that to speed up the calculation of shares. See MESOS-2891 for
+    // the reasons why we want to do that.
     //
     // NOTE: We omit information about dynamic reservations and persistent
     // volumes here to enable resources to be aggregated across slaves