You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by jo...@apache.org on 2019/02/13 22:46:58 UTC

[mesos] branch master updated: Copied operation removal logic in agent removal code path.

This is an automated email from the ASF dual-hosted git repository.

josephwu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git


The following commit(s) were added to refs/heads/master by this push:
     new 51db083  Copied operation removal logic in agent removal code path.
51db083 is described below

commit 51db083d899ff8f444c8842b5349cc21b59995c4
Author: Joseph Wu <jo...@apache.org>
AuthorDate: Tue Feb 12 17:54:33 2019 -0800

    Copied operation removal logic in agent removal code path.
    
    There are two code paths the master can follow to remove an agent
    from its memory:
    - When the agent sends an UnregisterSlaveMessage or tries to register
      as a new agent,
    - Or, when an agent is marked unreachable/gone.
    
    The first code path did not clean up operation state completely,
    resulting in leaking some memory in the master's Framework structs.
    
    This is direct copy of the operation cleanup logic in
    `Master::__removeSlave()`.
    
    Review: https://reviews.apache.org/r/69968
---
 src/master/master.cpp | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/src/master/master.cpp b/src/master/master.cpp
index 014e0e0..8e23785 100644
--- a/src/master/master.cpp
+++ b/src/master/master.cpp
@@ -10976,6 +10976,32 @@ void Master::_removeSlave(
     removeInverseOffer(inverseOffer, true); // Rescind!
   }
 
+  // Usually, operations are removed when the framework acknowledges
+  // a terminal operation status update. However, currently only operations
+  // on registered agents can be acknowledged. Since we're about to remove
+  // this agent from the list of registered agents, clean out all outstanding
+  // operations to prevent leaks.
+  //
+  // NOTE: If the agent comes back, there will be a brief window between
+  // the `ReregisterSlaveMessage` and the first `UpdateSlaveMessage` where
+  // where the master will not be able to give correct answers to operation
+  // reconciliation requests. However, since the same thing happens during
+  // master failover, the scheduler must be able to handle this scenario
+  // anyway so we allow it to happen here.
+  foreachvalue (Operation* operation, utils::copy(slave->operations)) {
+    removeOperation(operation);
+  }
+
+  foreachvalue (
+      const Slave::ResourceProvider& provider,
+      slave->resourceProviders) {
+    foreachvalue (
+        Operation* operation,
+        utils::copy(provider.operations)) {
+      removeOperation(operation);
+    }
+  }
+
   // Remove the pending tasks from the slave.
   slave->pendingTasks.clear();