[mesos] 03/05: Added flag to control GC-ing of nested container sandboxes.

[jo...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

josephwu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git

commit 1ec85225db1b125eb513726e9adff10a2ac4291e
Author: Joseph Wu <jo...@apache.org>
AuthorDate: Mon Jul 23 12:25:26 2018 -0700

    Added flag to control GC-ing of nested container sandboxes.
    
    This adds an agent flag which enables garbage collection of finished
    nested container sandboxes.
    
    This also updates some documentation and recommends enabling this flag
    when the user uses the same default executor to launch multiple tasks.
    
    Review: https://reviews.apache.org/r/68067
---
 docs/configuration/agent.md | 13 +++++++++++++
 docs/operator-http-api.md   | 11 +++++++++++
 docs/sandbox.md             |  3 +++
 src/slave/flags.cpp         |  9 +++++++++
 src/slave/flags.hpp         |  1 +
 src/slave/http.cpp          |  1 +
 6 files changed, 38 insertions(+)

diff --git a/docs/configuration/agent.md b/docs/configuration/agent.md
index e923227..7a8df68 100644
--- a/docs/configuration/agent.md
+++ b/docs/configuration/agent.md
@@ -931,6 +931,19 @@ be a value between 0.0 and 1.0 (default: 0.1)
   </td>
 </tr>
 
+<tr id="gc_non_executor_container_sandboxes">
+  <td>
+    --[no-]gc_non_executor_container_sandboxes
+  </td>
+  <td>
+Determines whether nested container sandboxes created via the
+<code>LAUNCH_CONTAINER</code> and <code>LAUNCH_NESTED_CONTAINER</code> APIs will be
+automatically garbage collected by the agent upon termination.
+The <code>REMOVE_(NESTED_)CONTAINER</code> API is unaffected by this flag
+and can still be used. (default: false).
+  </td>
+</tr>
+
 <tr id="hadoop_home">
   <td>
     --hadoop_home=VALUE
diff --git a/docs/operator-http-api.md b/docs/operator-http-api.md
index a571683..6edf361 100644
--- a/docs/operator-http-api.md
+++ b/docs/operator-http-api.md
@@ -2953,6 +2953,10 @@ Content-Type: application/json
         "value": "0.1"
       },
       {
+        "name": "gc_non_executor_container_sandboxes",
+        "value": "false"
+      },
+      {
         "name": "help",
         "value": "false"
       },
@@ -3588,6 +3592,13 @@ This call launches a nested container. Any authorized entity,
 including the executor itself, its tasks, or the operator can use this
 API to launch a nested container.
 
+**NOTE**: Successful invocation of this API will result in some metadata
+tracked by the agent and the creation of a container sandbox under the
+parent container. The REMOVE_NESTED_CONTAINER should be used to remove
+the metadata and sandbox. If the `--gc_non_executor_container_sandboxes`
+agent flag is enabled, the agent will garbage collect the sandboxes of
+any containers launched via this API.
+
 ```
 LAUNCH_NESTED_CONTAINER HTTP Request (JSON):
 
diff --git a/docs/sandbox.md b/docs/sandbox.md
index 3b44112..c536443 100644
--- a/docs/sandbox.md
+++ b/docs/sandbox.md
@@ -157,6 +157,9 @@ Sandbox files are scheduled for garbage collection when:
 * An executor is removed or terminated.
 * A framework is removed.
 * An executor is recovered unsuccessfully during agent recovery.
+* If the `--gc_non_executor_container_sandboxes` agent flag is enabled,
+  nested container sandboxes will also be garbage collected when the
+  container exits.
 
 **NOTE:** During agent recovery, all of the executor's runs, except for the
 latest run, are scheduled for garbage collection as well.
diff --git a/src/slave/flags.cpp b/src/slave/flags.cpp
index fd53d90..ccaf650 100644
--- a/src/slave/flags.cpp
+++ b/src/slave/flags.cpp
@@ -482,6 +482,15 @@ mesos::internal::slave::Flags::Flags()
       "be a value between 0.0 and 1.0",
       GC_DISK_HEADROOM);
 
+  add(&Flags::gc_non_executor_container_sandboxes,
+      "gc_non_executor_container_sandboxes",
+      "Determines whether nested container sandboxes created via the\n"
+      "LAUNCH_CONTAINER and LAUNCH_NESTED_CONTAINER APIs will be\n"
+      "automatically garbage collected by the agent upon termination.\n"
+      "The REMOVE_(NESTED_)CONTAINER API is unaffected by this flag\n"
+      "and can still be used.",
+      false);
+
   add(&Flags::disk_watch_interval,
       "disk_watch_interval",
       "Periodic time interval (e.g., 10secs, 2mins, etc)\n"
diff --git a/src/slave/flags.hpp b/src/slave/flags.hpp
index f6db2bb..29d8b79 100644
--- a/src/slave/flags.hpp
+++ b/src/slave/flags.hpp
@@ -91,6 +91,7 @@ public:
 #endif // USE_SSL_SOCKET
   Duration gc_delay;
   double gc_disk_headroom;
+  bool gc_non_executor_container_sandboxes;
   Duration disk_watch_interval;
 
   Option<std::string> container_logger;
diff --git a/src/slave/http.cpp b/src/slave/http.cpp
index f8199af..fb92368 100644
--- a/src/slave/http.cpp
+++ b/src/slave/http.cpp
@@ -1199,6 +1199,7 @@ string Http::STATE_HELP() {
         "         \"containerizers\" : \"mesos\",",
         "         \"docker_socket\" : \"/var/run/docker.sock\",",
         "         \"gc_delay\" : \"1weeks\",",
+        "         \"gc_non_executor_container_sandboxes\" : \"false\",",
         "         \"docker_remove_delay\" : \"6hrs\",",
         "         \"port\" : \"5051\",",
         "         \"systemd_runtime_directory\" : \"/run/systemd/system\",",