You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by jp...@apache.org on 2017/11/03 17:11:18 UTC

[1/6] mesos git commit: Fixed default executor handling of nested container status.

Repository: mesos
Updated Branches:
  refs/heads/1.2.x 2c3ff9774 -> a7fb1fbce


Fixed default executor handling of nested container status.

The default executor was not handling a missing nested container
exit status correctly. It was assuming the protobuf accessor was
returning an Option rather than explicitly checking whether the
`exit_status` field was present in the message.

Added the explicit check for the `exit_status` field, and always
propagated an appropriate message into the status update, even
when the `exit_status` is absent. Added some documentation of
the `exit_status` field to the protobuf definition files.

Review: https://reviews.apache.org/r/62518/


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/1ff5c324
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/1ff5c324
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/1ff5c324

Branch: refs/heads/1.2.x
Commit: 1ff5c324c04654f6592b169a4c7fac440dd80017
Parents: 2c3ff97
Author: James Peach <jp...@apache.org>
Authored: Thu Oct 12 09:17:16 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:25:31 2017 -0700

----------------------------------------------------------------------
 include/mesos/agent/agent.proto    |  4 ++++
 include/mesos/v1/agent/agent.proto |  4 ++++
 src/launcher/default_executor.cpp  | 19 ++++++++++---------
 3 files changed, 18 insertions(+), 9 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/mesos/blob/1ff5c324/include/mesos/agent/agent.proto
----------------------------------------------------------------------
diff --git a/include/mesos/agent/agent.proto b/include/mesos/agent/agent.proto
index 775c14b..cd67895 100644
--- a/include/mesos/agent/agent.proto
+++ b/include/mesos/agent/agent.proto
@@ -314,6 +314,10 @@ message Response {
 
   // Returns termination information about the nested container.
   message WaitNestedContainer {
+    // Wait status of the lead process in the container. Note that this
+    // is the return value of `wait(2)`, so callers must use the `wait(2)`
+    // family of macros to extract whether the process exited cleanly and
+    // what the exit code was.
     optional int32 exit_status = 1;
   }
 

http://git-wip-us.apache.org/repos/asf/mesos/blob/1ff5c324/include/mesos/v1/agent/agent.proto
----------------------------------------------------------------------
diff --git a/include/mesos/v1/agent/agent.proto b/include/mesos/v1/agent/agent.proto
index a98acb7..d02f2ef 100644
--- a/include/mesos/v1/agent/agent.proto
+++ b/include/mesos/v1/agent/agent.proto
@@ -313,6 +313,10 @@ message Response {
 
   // Returns termination information about the nested container.
   message WaitNestedContainer {
+    // Wait status of the lead process in the container. Note that this
+    // is the return value of `wait(2)`, so callers must use the `wait(2)`
+    // family of macros to extract whether the process exited cleanly and
+    // what the exit code was.
     optional int32 exit_status = 1;
   }
 

http://git-wip-us.apache.org/repos/asf/mesos/blob/1ff5c324/src/launcher/default_executor.cpp
----------------------------------------------------------------------
diff --git a/src/launcher/default_executor.cpp b/src/launcher/default_executor.cpp
index d97324b..f30947e 100644
--- a/src/launcher/default_executor.cpp
+++ b/src/launcher/default_executor.cpp
@@ -641,15 +641,16 @@ protected:
     TaskState taskState;
     Option<string> message;
 
-    Option<int> status = waitResponse->wait_nested_container().exit_status();
-
-    if (status.isNone()) {
+    if (!waitResponse->wait_nested_container().has_exit_status()) {
       taskState = TASK_FAILED;
+      message = "Command terminated with unknown status";
     } else {
-      CHECK(WIFEXITED(status.get()) || WIFSIGNALED(status.get()))
-        << "Unexpected wait status " << status.get();
+      int status = waitResponse->wait_nested_container().exit_status();
+
+      CHECK(WIFEXITED(status) || WIFSIGNALED(status))
+        << "Unexpected wait status " << status;
 
-      if (WSUCCEEDED(status.get())) {
+      if (WSUCCEEDED(status)) {
         taskState = TASK_FINISHED;
       } else if (container->killing) {
         // Send TASK_KILLED if the task was killed as a result of
@@ -659,7 +660,7 @@ protected:
         taskState = TASK_FAILED;
       }
 
-      message = "Command " + WSTRINGIFY(status.get());
+      message = "Command " + WSTRINGIFY(status);
     }
 
     if (unhealthy) {
@@ -673,8 +674,8 @@ protected:
 
     LOG(INFO)
       << "Child container " << container->containerId << " of task '" << taskId
-      << "' in state " << stringify(taskState) << " terminated with status "
-      << (status.isSome() ? WSTRINGIFY(status.get()) : "unknown");
+      << "' completed in state " << stringify(taskState)
+      << ": " << message.get();
 
     // Shutdown the executor if all the active child containers have terminated.
     if (containers.empty()) {


[5/6] mesos git commit: Added workaround to unbreak the build with glibc <= 2.12.

Posted by jp...@apache.org.
Added workaround to unbreak the build with glibc <= 2.12.

In old versions of glibc, the <sys/sysmacros.h> header must be enclosed
in an `extern "C" { ... }` block. This prevents Mesos from linking on
CentOS 6.9 and likely other platforms from the same timeframe.

In more recent versions of glibc, the `extern "C" { ... }` linkage
declaration is not necessary (because the header file specifies this
itself). In this situation, there will now be two linkage declarations,
but that is fine: the inner-most linkage declaration takes precedence.

Review: https://reviews.apache.org/r/59154


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/180fc8b3
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/180fc8b3
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/180fc8b3

Branch: refs/heads/1.2.x
Commit: 180fc8b3e69ca15b8beffcb4d564c31852661a72
Parents: c6cf956
Author: Neil Conway <ne...@gmail.com>
Authored: Wed May 10 11:06:24 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:48:36 2017 -0700

----------------------------------------------------------------------
 src/linux/fs.cpp                                         | 6 ++++++
 src/slave/containerizer/mesos/isolators/gpu/isolator.cpp | 6 ++++++
 src/tests/containerizer/fs_tests.cpp                     | 6 ++++++
 3 files changed, 18 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/mesos/blob/180fc8b3/src/linux/fs.cpp
----------------------------------------------------------------------
diff --git a/src/linux/fs.cpp b/src/linux/fs.cpp
index c0b8cc3..52dcab6 100644
--- a/src/linux/fs.cpp
+++ b/src/linux/fs.cpp
@@ -23,7 +23,13 @@
 #include <linux/limits.h>
 #include <linux/unistd.h>
 
+// This header include must be enclosed in an `extern "C"` block to
+// workaround a bug in glibc <= 2.12 (see MESOS-7378).
+//
+// TODO(neilc): Remove this when we no longer support glibc <= 2.12.
+extern "C" {
 #include <sys/sysmacros.h>
+}
 
 #include <list>
 #include <set>

http://git-wip-us.apache.org/repos/asf/mesos/blob/180fc8b3/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp b/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
index 3fd1495..25636b5 100644
--- a/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
+++ b/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
@@ -16,7 +16,13 @@
 
 #include <stdint.h>
 
+// This header include must be enclosed in an `extern "C"` block to
+// workaround a bug in glibc <= 2.12 (see MESOS-7378).
+//
+// TODO(neilc): Remove this when we no longer support glibc <= 2.12.
+extern "C" {
 #include <sys/sysmacros.h>
+}
 
 #include <algorithm>
 #include <list>

http://git-wip-us.apache.org/repos/asf/mesos/blob/180fc8b3/src/tests/containerizer/fs_tests.cpp
----------------------------------------------------------------------
diff --git a/src/tests/containerizer/fs_tests.cpp b/src/tests/containerizer/fs_tests.cpp
index 00f3483..a8b93ba 100644
--- a/src/tests/containerizer/fs_tests.cpp
+++ b/src/tests/containerizer/fs_tests.cpp
@@ -18,7 +18,13 @@
 
 #include <gmock/gmock.h>
 
+// This header include must be enclosed in an `extern "C"` block to
+// workaround a bug in glibc <= 2.12 (see MESOS-7378).
+//
+// TODO(neilc): Remove this when we no longer support glibc <= 2.12.
+extern "C" {
 #include <sys/sysmacros.h>
+}
 
 #include <stout/foreach.hpp>
 #include <stout/gtest.hpp>


[2/6] mesos git commit: Added MESOS-8080 to the 1.2.3 CHANGELOG.

Posted by jp...@apache.org.
Added MESOS-8080 to the 1.2.3 CHANGELOG.


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/386d9f64
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/386d9f64
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/386d9f64

Branch: refs/heads/1.2.x
Commit: 386d9f64c4550554388e16c41460c32ad122e2a9
Parents: 1ff5c32
Author: James Peach <jp...@apache.org>
Authored: Fri Nov 3 09:27:08 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:27:08 2017 -0700

----------------------------------------------------------------------
 CHANGELOG | 1 +
 1 file changed, 1 insertion(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/mesos/blob/386d9f64/CHANGELOG
----------------------------------------------------------------------
diff --git a/CHANGELOG b/CHANGELOG
index fcd3983..2ae7fb3 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -14,6 +14,7 @@ All Issues:
   * [MESOS-7909] - Ordering dependency between 'linux/capabilities' and 'docker/runtime' isolator.
   * [MESOS-7926] - Abnormal termination of default executor can cause MesosContainerizer::destroy to fail.
   * [MESOS-7934] - OOM due to LibeventSSLSocket send incorrectly returning 0 after shutdown.
+  * [MESOS-8080] - The default executor does not propagate missing task exit status correctly.
   * [MESOS-8135] - Masters can lose track of tasks' executor IDs.
 
 


[4/6] mesos git commit: Added MESOS-7365 to the 1.2.3 CHANGELOG.

Posted by jp...@apache.org.
Added MESOS-7365 to the 1.2.3 CHANGELOG.


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/c6cf956e
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/c6cf956e
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/c6cf956e

Branch: refs/heads/1.2.x
Commit: c6cf956e4ef28b4f98073842ba3629889c17a37b
Parents: 83d7aaf
Author: James Peach <jp...@apache.org>
Authored: Fri Nov 3 09:43:05 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:43:05 2017 -0700

----------------------------------------------------------------------
 CHANGELOG | 1 +
 1 file changed, 1 insertion(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/mesos/blob/c6cf956e/CHANGELOG
----------------------------------------------------------------------
diff --git a/CHANGELOG b/CHANGELOG
index 2ae7fb3..369e733 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -6,6 +6,7 @@ All Issues:
 ** Bug
   * [MESOS-6743] - Docker executor hangs forever if `docker stop` fails.
   * [MESOS-6950] - Launching two tasks with the same Docker image simultaneously may cause a staging dir never cleaned up.
+  * [MESOS-7365] - Compile error with recent glibc.
   * [MESOS-7652] - Docker image with universal containerizer does not work if WORKDIR is missing in the rootfs.
   * [MESOS-7858] - Launching a nested container with namespace/pid isolation, with glibc < 2.25, may deadlock the LinuxLauncher and MesosContainerizer.
   * [MESOS-7863] - Agent may drop pending kill task status updates.


[3/6] mesos git commit: Fixed compilation error on Linux with recent glibc.

Posted by jp...@apache.org.
Fixed compilation error on Linux with recent glibc.

Recent versions of glibc deprecate the implicit inclusion of
<sys/sysmacros.h> via <sys/types.h>, so we should now include
<sys/sysmacros.h> directly when necessary.

Review: https://reviews.apache.org/r/58245


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/83d7aafb
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/83d7aafb
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/83d7aafb

Branch: refs/heads/1.2.x
Commit: 83d7aafbb1756534ae6c0efdce484d298b9694bd
Parents: 386d9f6
Author: Neil Conway <ne...@gmail.com>
Authored: Thu Apr 6 10:43:32 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:41:12 2017 -0700

----------------------------------------------------------------------
 src/linux/fs.cpp                                         | 2 ++
 src/slave/containerizer/mesos/isolators/gpu/isolator.cpp | 2 ++
 src/tests/containerizer/fs_tests.cpp                     | 2 ++
 3 files changed, 6 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/mesos/blob/83d7aafb/src/linux/fs.cpp
----------------------------------------------------------------------
diff --git a/src/linux/fs.cpp b/src/linux/fs.cpp
index 16c4583..c0b8cc3 100644
--- a/src/linux/fs.cpp
+++ b/src/linux/fs.cpp
@@ -23,6 +23,8 @@
 #include <linux/limits.h>
 #include <linux/unistd.h>
 
+#include <sys/sysmacros.h>
+
 #include <list>
 #include <set>
 #include <utility>

http://git-wip-us.apache.org/repos/asf/mesos/blob/83d7aafb/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp b/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
index d4ab539..3fd1495 100644
--- a/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
+++ b/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
@@ -16,6 +16,8 @@
 
 #include <stdint.h>
 
+#include <sys/sysmacros.h>
+
 #include <algorithm>
 #include <list>
 #include <map>

http://git-wip-us.apache.org/repos/asf/mesos/blob/83d7aafb/src/tests/containerizer/fs_tests.cpp
----------------------------------------------------------------------
diff --git a/src/tests/containerizer/fs_tests.cpp b/src/tests/containerizer/fs_tests.cpp
index f726fb0..00f3483 100644
--- a/src/tests/containerizer/fs_tests.cpp
+++ b/src/tests/containerizer/fs_tests.cpp
@@ -18,6 +18,8 @@
 
 #include <gmock/gmock.h>
 
+#include <sys/sysmacros.h>
+
 #include <stout/foreach.hpp>
 #include <stout/gtest.hpp>
 #include <stout/hashset.hpp>


[6/6] mesos git commit: Added MESOS-7378 to the 1.2.3 CHANGELOG.

Posted by jp...@apache.org.
Added MESOS-7378 to the 1.2.3 CHANGELOG.


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/a7fb1fbc
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/a7fb1fbc
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/a7fb1fbc

Branch: refs/heads/1.2.x
Commit: a7fb1fbce652512a61f42b31f7298960fa52d758
Parents: 180fc8b
Author: James Peach <jp...@apache.org>
Authored: Fri Nov 3 09:50:46 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:50:46 2017 -0700

----------------------------------------------------------------------
 CHANGELOG | 1 +
 1 file changed, 1 insertion(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/mesos/blob/a7fb1fbc/CHANGELOG
----------------------------------------------------------------------
diff --git a/CHANGELOG b/CHANGELOG
index 369e733..67d543e 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -7,6 +7,7 @@ All Issues:
   * [MESOS-6743] - Docker executor hangs forever if `docker stop` fails.
   * [MESOS-6950] - Launching two tasks with the same Docker image simultaneously may cause a staging dir never cleaned up.
   * [MESOS-7365] - Compile error with recent glibc.
+  * [MESOS-7378] - Build failure with glibc 2.12.
   * [MESOS-7652] - Docker image with universal containerizer does not work if WORKDIR is missing in the rootfs.
   * [MESOS-7858] - Launching a nested container with namespace/pid isolation, with glibc < 2.25, may deadlock the LinuxLauncher and MesosContainerizer.
   * [MESOS-7863] - Agent may drop pending kill task status updates.