You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by jp...@apache.org on 2017/11/03 17:11:18 UTC
[1/6] mesos git commit: Fixed default executor handling of nested
container status.
Repository: mesos
Updated Branches:
refs/heads/1.2.x 2c3ff9774 -> a7fb1fbce
Fixed default executor handling of nested container status.
The default executor was not handling a missing nested container
exit status correctly. It was assuming the protobuf accessor was
returning an Option rather than explicitly checking whether the
`exit_status` field was present in the message.
Added the explicit check for the `exit_status` field, and always
propagated an appropriate message into the status update, even
when the `exit_status` is absent. Added some documentation of
the `exit_status` field to the protobuf definition files.
Review: https://reviews.apache.org/r/62518/
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/1ff5c324
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/1ff5c324
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/1ff5c324
Branch: refs/heads/1.2.x
Commit: 1ff5c324c04654f6592b169a4c7fac440dd80017
Parents: 2c3ff97
Author: James Peach <jp...@apache.org>
Authored: Thu Oct 12 09:17:16 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:25:31 2017 -0700
----------------------------------------------------------------------
include/mesos/agent/agent.proto | 4 ++++
include/mesos/v1/agent/agent.proto | 4 ++++
src/launcher/default_executor.cpp | 19 ++++++++++---------
3 files changed, 18 insertions(+), 9 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/1ff5c324/include/mesos/agent/agent.proto
----------------------------------------------------------------------
diff --git a/include/mesos/agent/agent.proto b/include/mesos/agent/agent.proto
index 775c14b..cd67895 100644
--- a/include/mesos/agent/agent.proto
+++ b/include/mesos/agent/agent.proto
@@ -314,6 +314,10 @@ message Response {
// Returns termination information about the nested container.
message WaitNestedContainer {
+ // Wait status of the lead process in the container. Note that this
+ // is the return value of `wait(2)`, so callers must use the `wait(2)`
+ // family of macros to extract whether the process exited cleanly and
+ // what the exit code was.
optional int32 exit_status = 1;
}
http://git-wip-us.apache.org/repos/asf/mesos/blob/1ff5c324/include/mesos/v1/agent/agent.proto
----------------------------------------------------------------------
diff --git a/include/mesos/v1/agent/agent.proto b/include/mesos/v1/agent/agent.proto
index a98acb7..d02f2ef 100644
--- a/include/mesos/v1/agent/agent.proto
+++ b/include/mesos/v1/agent/agent.proto
@@ -313,6 +313,10 @@ message Response {
// Returns termination information about the nested container.
message WaitNestedContainer {
+ // Wait status of the lead process in the container. Note that this
+ // is the return value of `wait(2)`, so callers must use the `wait(2)`
+ // family of macros to extract whether the process exited cleanly and
+ // what the exit code was.
optional int32 exit_status = 1;
}
http://git-wip-us.apache.org/repos/asf/mesos/blob/1ff5c324/src/launcher/default_executor.cpp
----------------------------------------------------------------------
diff --git a/src/launcher/default_executor.cpp b/src/launcher/default_executor.cpp
index d97324b..f30947e 100644
--- a/src/launcher/default_executor.cpp
+++ b/src/launcher/default_executor.cpp
@@ -641,15 +641,16 @@ protected:
TaskState taskState;
Option<string> message;
- Option<int> status = waitResponse->wait_nested_container().exit_status();
-
- if (status.isNone()) {
+ if (!waitResponse->wait_nested_container().has_exit_status()) {
taskState = TASK_FAILED;
+ message = "Command terminated with unknown status";
} else {
- CHECK(WIFEXITED(status.get()) || WIFSIGNALED(status.get()))
- << "Unexpected wait status " << status.get();
+ int status = waitResponse->wait_nested_container().exit_status();
+
+ CHECK(WIFEXITED(status) || WIFSIGNALED(status))
+ << "Unexpected wait status " << status;
- if (WSUCCEEDED(status.get())) {
+ if (WSUCCEEDED(status)) {
taskState = TASK_FINISHED;
} else if (container->killing) {
// Send TASK_KILLED if the task was killed as a result of
@@ -659,7 +660,7 @@ protected:
taskState = TASK_FAILED;
}
- message = "Command " + WSTRINGIFY(status.get());
+ message = "Command " + WSTRINGIFY(status);
}
if (unhealthy) {
@@ -673,8 +674,8 @@ protected:
LOG(INFO)
<< "Child container " << container->containerId << " of task '" << taskId
- << "' in state " << stringify(taskState) << " terminated with status "
- << (status.isSome() ? WSTRINGIFY(status.get()) : "unknown");
+ << "' completed in state " << stringify(taskState)
+ << ": " << message.get();
// Shutdown the executor if all the active child containers have terminated.
if (containers.empty()) {
[5/6] mesos git commit: Added workaround to unbreak the build with
glibc <= 2.12.
Posted by jp...@apache.org.
Added workaround to unbreak the build with glibc <= 2.12.
In old versions of glibc, the <sys/sysmacros.h> header must be enclosed
in an `extern "C" { ... }` block. This prevents Mesos from linking on
CentOS 6.9 and likely other platforms from the same timeframe.
In more recent versions of glibc, the `extern "C" { ... }` linkage
declaration is not necessary (because the header file specifies this
itself). In this situation, there will now be two linkage declarations,
but that is fine: the inner-most linkage declaration takes precedence.
Review: https://reviews.apache.org/r/59154
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/180fc8b3
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/180fc8b3
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/180fc8b3
Branch: refs/heads/1.2.x
Commit: 180fc8b3e69ca15b8beffcb4d564c31852661a72
Parents: c6cf956
Author: Neil Conway <ne...@gmail.com>
Authored: Wed May 10 11:06:24 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:48:36 2017 -0700
----------------------------------------------------------------------
src/linux/fs.cpp | 6 ++++++
src/slave/containerizer/mesos/isolators/gpu/isolator.cpp | 6 ++++++
src/tests/containerizer/fs_tests.cpp | 6 ++++++
3 files changed, 18 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/180fc8b3/src/linux/fs.cpp
----------------------------------------------------------------------
diff --git a/src/linux/fs.cpp b/src/linux/fs.cpp
index c0b8cc3..52dcab6 100644
--- a/src/linux/fs.cpp
+++ b/src/linux/fs.cpp
@@ -23,7 +23,13 @@
#include <linux/limits.h>
#include <linux/unistd.h>
+// This header include must be enclosed in an `extern "C"` block to
+// workaround a bug in glibc <= 2.12 (see MESOS-7378).
+//
+// TODO(neilc): Remove this when we no longer support glibc <= 2.12.
+extern "C" {
#include <sys/sysmacros.h>
+}
#include <list>
#include <set>
http://git-wip-us.apache.org/repos/asf/mesos/blob/180fc8b3/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp b/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
index 3fd1495..25636b5 100644
--- a/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
+++ b/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
@@ -16,7 +16,13 @@
#include <stdint.h>
+// This header include must be enclosed in an `extern "C"` block to
+// workaround a bug in glibc <= 2.12 (see MESOS-7378).
+//
+// TODO(neilc): Remove this when we no longer support glibc <= 2.12.
+extern "C" {
#include <sys/sysmacros.h>
+}
#include <algorithm>
#include <list>
http://git-wip-us.apache.org/repos/asf/mesos/blob/180fc8b3/src/tests/containerizer/fs_tests.cpp
----------------------------------------------------------------------
diff --git a/src/tests/containerizer/fs_tests.cpp b/src/tests/containerizer/fs_tests.cpp
index 00f3483..a8b93ba 100644
--- a/src/tests/containerizer/fs_tests.cpp
+++ b/src/tests/containerizer/fs_tests.cpp
@@ -18,7 +18,13 @@
#include <gmock/gmock.h>
+// This header include must be enclosed in an `extern "C"` block to
+// workaround a bug in glibc <= 2.12 (see MESOS-7378).
+//
+// TODO(neilc): Remove this when we no longer support glibc <= 2.12.
+extern "C" {
#include <sys/sysmacros.h>
+}
#include <stout/foreach.hpp>
#include <stout/gtest.hpp>
[2/6] mesos git commit: Added MESOS-8080 to the 1.2.3 CHANGELOG.
Posted by jp...@apache.org.
Added MESOS-8080 to the 1.2.3 CHANGELOG.
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/386d9f64
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/386d9f64
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/386d9f64
Branch: refs/heads/1.2.x
Commit: 386d9f64c4550554388e16c41460c32ad122e2a9
Parents: 1ff5c32
Author: James Peach <jp...@apache.org>
Authored: Fri Nov 3 09:27:08 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:27:08 2017 -0700
----------------------------------------------------------------------
CHANGELOG | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/386d9f64/CHANGELOG
----------------------------------------------------------------------
diff --git a/CHANGELOG b/CHANGELOG
index fcd3983..2ae7fb3 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -14,6 +14,7 @@ All Issues:
* [MESOS-7909] - Ordering dependency between 'linux/capabilities' and 'docker/runtime' isolator.
* [MESOS-7926] - Abnormal termination of default executor can cause MesosContainerizer::destroy to fail.
* [MESOS-7934] - OOM due to LibeventSSLSocket send incorrectly returning 0 after shutdown.
+ * [MESOS-8080] - The default executor does not propagate missing task exit status correctly.
* [MESOS-8135] - Masters can lose track of tasks' executor IDs.
[4/6] mesos git commit: Added MESOS-7365 to the 1.2.3 CHANGELOG.
Posted by jp...@apache.org.
Added MESOS-7365 to the 1.2.3 CHANGELOG.
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/c6cf956e
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/c6cf956e
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/c6cf956e
Branch: refs/heads/1.2.x
Commit: c6cf956e4ef28b4f98073842ba3629889c17a37b
Parents: 83d7aaf
Author: James Peach <jp...@apache.org>
Authored: Fri Nov 3 09:43:05 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:43:05 2017 -0700
----------------------------------------------------------------------
CHANGELOG | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/c6cf956e/CHANGELOG
----------------------------------------------------------------------
diff --git a/CHANGELOG b/CHANGELOG
index 2ae7fb3..369e733 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -6,6 +6,7 @@ All Issues:
** Bug
* [MESOS-6743] - Docker executor hangs forever if `docker stop` fails.
* [MESOS-6950] - Launching two tasks with the same Docker image simultaneously may cause a staging dir never cleaned up.
+ * [MESOS-7365] - Compile error with recent glibc.
* [MESOS-7652] - Docker image with universal containerizer does not work if WORKDIR is missing in the rootfs.
* [MESOS-7858] - Launching a nested container with namespace/pid isolation, with glibc < 2.25, may deadlock the LinuxLauncher and MesosContainerizer.
* [MESOS-7863] - Agent may drop pending kill task status updates.
[3/6] mesos git commit: Fixed compilation error on Linux with recent
glibc.
Posted by jp...@apache.org.
Fixed compilation error on Linux with recent glibc.
Recent versions of glibc deprecate the implicit inclusion of
<sys/sysmacros.h> via <sys/types.h>, so we should now include
<sys/sysmacros.h> directly when necessary.
Review: https://reviews.apache.org/r/58245
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/83d7aafb
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/83d7aafb
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/83d7aafb
Branch: refs/heads/1.2.x
Commit: 83d7aafbb1756534ae6c0efdce484d298b9694bd
Parents: 386d9f6
Author: Neil Conway <ne...@gmail.com>
Authored: Thu Apr 6 10:43:32 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:41:12 2017 -0700
----------------------------------------------------------------------
src/linux/fs.cpp | 2 ++
src/slave/containerizer/mesos/isolators/gpu/isolator.cpp | 2 ++
src/tests/containerizer/fs_tests.cpp | 2 ++
3 files changed, 6 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/83d7aafb/src/linux/fs.cpp
----------------------------------------------------------------------
diff --git a/src/linux/fs.cpp b/src/linux/fs.cpp
index 16c4583..c0b8cc3 100644
--- a/src/linux/fs.cpp
+++ b/src/linux/fs.cpp
@@ -23,6 +23,8 @@
#include <linux/limits.h>
#include <linux/unistd.h>
+#include <sys/sysmacros.h>
+
#include <list>
#include <set>
#include <utility>
http://git-wip-us.apache.org/repos/asf/mesos/blob/83d7aafb/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp b/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
index d4ab539..3fd1495 100644
--- a/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
+++ b/src/slave/containerizer/mesos/isolators/gpu/isolator.cpp
@@ -16,6 +16,8 @@
#include <stdint.h>
+#include <sys/sysmacros.h>
+
#include <algorithm>
#include <list>
#include <map>
http://git-wip-us.apache.org/repos/asf/mesos/blob/83d7aafb/src/tests/containerizer/fs_tests.cpp
----------------------------------------------------------------------
diff --git a/src/tests/containerizer/fs_tests.cpp b/src/tests/containerizer/fs_tests.cpp
index f726fb0..00f3483 100644
--- a/src/tests/containerizer/fs_tests.cpp
+++ b/src/tests/containerizer/fs_tests.cpp
@@ -18,6 +18,8 @@
#include <gmock/gmock.h>
+#include <sys/sysmacros.h>
+
#include <stout/foreach.hpp>
#include <stout/gtest.hpp>
#include <stout/hashset.hpp>
[6/6] mesos git commit: Added MESOS-7378 to the 1.2.3 CHANGELOG.
Posted by jp...@apache.org.
Added MESOS-7378 to the 1.2.3 CHANGELOG.
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/a7fb1fbc
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/a7fb1fbc
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/a7fb1fbc
Branch: refs/heads/1.2.x
Commit: a7fb1fbce652512a61f42b31f7298960fa52d758
Parents: 180fc8b
Author: James Peach <jp...@apache.org>
Authored: Fri Nov 3 09:50:46 2017 -0700
Committer: James Peach <jp...@apache.org>
Committed: Fri Nov 3 09:50:46 2017 -0700
----------------------------------------------------------------------
CHANGELOG | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/a7fb1fbc/CHANGELOG
----------------------------------------------------------------------
diff --git a/CHANGELOG b/CHANGELOG
index 369e733..67d543e 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -7,6 +7,7 @@ All Issues:
* [MESOS-6743] - Docker executor hangs forever if `docker stop` fails.
* [MESOS-6950] - Launching two tasks with the same Docker image simultaneously may cause a staging dir never cleaned up.
* [MESOS-7365] - Compile error with recent glibc.
+ * [MESOS-7378] - Build failure with glibc 2.12.
* [MESOS-7652] - Docker image with universal containerizer does not work if WORKDIR is missing in the rootfs.
* [MESOS-7858] - Launching a nested container with namespace/pid isolation, with glibc < 2.25, may deadlock the LinuxLauncher and MesosContainerizer.
* [MESOS-7863] - Agent may drop pending kill task status updates.