You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by be...@apache.org on 2019/04/26 14:58:42 UTC
[mesos] branch 1.8.x updated (1e2258f -> acefa90)
This is an automated email from the ASF dual-hosted git repository.
bennoe pushed a change to branch 1.8.x
in repository https://gitbox.apache.org/repos/asf/mesos.git.
from 1e2258f Added MESOS-9619 to the 1.8.0 CHANGELOG.
new 5789193 Relaxed protobuf union validation strictness.
new acefa90 Added MESOS-9740 to the 1.8.0 CHANGELOG.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGELOG | 1 +
src/common/validation.cpp | 9 ++++++++-
src/master/validation.cpp | 8 +++++++-
src/tests/master_validation_tests.cpp | 19 +++----------------
4 files changed, 19 insertions(+), 18 deletions(-)
[mesos] 02/02: Added MESOS-9740 to the 1.8.0 CHANGELOG.
Posted by be...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bennoe pushed a commit to branch 1.8.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit acefa90695a32f8e8d6361f8192a6522aeaadbb9
Author: Benno Evers <be...@mesosphere.com>
AuthorDate: Fri Apr 26 16:46:00 2019 +0200
Added MESOS-9740 to the 1.8.0 CHANGELOG.
---
CHANGELOG | 1 +
1 file changed, 1 insertion(+)
diff --git a/CHANGELOG b/CHANGELOG
index 12aeedb..6ee8ad2 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -275,6 +275,7 @@ All Resolved Issues:
* [MESOS-9727] - Heartbeat calls from executor to agent are reported as errors.
* [MESOS-9729] - Unpublishing a volume that is failed to publish crashes the agent with CSI v1.
* [MESOS-9733] - Random sorter generates non-uniform result for hierarchical roles.
+ * [MESOS-9740] - Invalid protobuf unions in ExecutorInfo::ContainerInfo will prevent agents from reregistering with 1.8+ masters
** Epic
* [MESOS-8054] - Feedback for operations
[mesos] 01/02: Relaxed protobuf union validation strictness.
Posted by be...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bennoe pushed a commit to branch 1.8.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 578919306623530515ae27a13e34047f8efac838
Author: Joseph Wu <jo...@mesosphere.io>
AuthorDate: Fri Apr 26 13:51:21 2019 +0200
Relaxed protobuf union validation strictness.
As part of MESOS-6874, the master validates protobuf unions passed as
part of an ExecutorInfo::ContainerInfo. This prevents a task from
specifying, for example, a ContainerInfo::MESOS, but filling
out the docker field (which is then ignored by the agent).
This validation change is actually an API change, because previously
runnable ExecutorInfo's and TaskInfo's will now fail validation.
This has two visible effects on clusters:
* Agents running containers with invalid protobuf unions will not
be able to reregister with the master.
* Existing frameworks will not be able to re-launch the same tasks
that were working before a Mesos master upgrade.
This changes the validation to print a warning instead. Where possible,
the warning will provide some information to indicate which task
or executor is sending the invalid protobuf.
Review: https://reviews.apache.org/r/70546/
---
src/common/validation.cpp | 9 ++++++++-
src/master/validation.cpp | 8 +++++++-
src/tests/master_validation_tests.cpp | 19 +++----------------
3 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/src/common/validation.cpp b/src/common/validation.cpp
index 458f225..14a8c7b 100644
--- a/src/common/validation.cpp
+++ b/src/common/validation.cpp
@@ -268,7 +268,14 @@ Option<Error> validateContainerInfo(const ContainerInfo& containerInfo)
{
Option<Error> unionError = protobuf::validateProtobufUnion(containerInfo);
if (unionError.isSome()) {
- return unionError;
+ // TODO(josephw): There is not enough information in this function to
+ // print an actionable warning. Readers of this warning will not know
+ // which container (task or executor) has this problem. Printing
+ // the entire ContainerInfo is the best we can do.
+ LOG(WARNING)
+ << "Invalid protobuf union detected in the given ContainerInfo ("
+ << containerInfo.DebugString()
+ << "): " << unionError.get();
}
foreach (const Volume& volume, containerInfo.volumes()) {
diff --git a/src/master/validation.cpp b/src/master/validation.cpp
index d7f210f..9fb0850 100644
--- a/src/master/validation.cpp
+++ b/src/master/validation.cpp
@@ -1001,7 +1001,13 @@ Option<Error> validateType(const ExecutorInfo& executor)
Option<Error> unionError =
protobuf::validateProtobufUnion(executor.container());
if (unionError.isSome()) {
- return unionError;
+ LOG(WARNING)
+ << "Executor " << executor.executor_id()
+ // NOTE: Validation of FrameworkID is done after this validation.
+ << " of framework '"
+ << (executor.has_framework_id() ? executor.framework_id().value() : "")
+ << "' has an invalid protobuf union: "
+ << unionError.get();
}
}
switch (executor.type()) {
diff --git a/src/tests/master_validation_tests.cpp b/src/tests/master_validation_tests.cpp
index 400ad68..3897e71 100644
--- a/src/tests/master_validation_tests.cpp
+++ b/src/tests/master_validation_tests.cpp
@@ -3259,7 +3259,9 @@ TEST_F(TaskValidationTest, TaskMissingDockerInfo)
// This test verifies that a task that has `ContainerInfo` set as MESOS
// but has a `DockerInfo` is rejected during `TaskInfo` validation.
-TEST_F(TaskValidationTest, TaskMesosTypeWithDockerInfo)
+// TODO(josephw): Enable this regression test when we officially deprecate
+// this invalid protobuf. See MESOS-6874 and MESOS-9740.
+TEST_F(TaskValidationTest, DISABLED_TaskMesosTypeWithDockerInfo)
{
Try<Owned<cluster::Master>> master = StartMaster();
ASSERT_SOME(master);
@@ -3466,21 +3468,6 @@ TEST_F(ExecutorValidationTest, ExecutorType)
"'ExecutorInfo.container.mesos.image' must not be set for "
"'DEFAULT' executor"));
}
- {
- // Invalid protobuf union in ContainerInfo.
- executorInfo.set_type(ExecutorInfo::CUSTOM);
- executorInfo.mutable_command();
- executorInfo.mutable_container()->set_type(ContainerInfo::DOCKER);
- executorInfo.mutable_container()->mutable_mesos();
-
- Option<Error> error = ::executor::internal::validateType(executorInfo);
-
- EXPECT_SOME(error);
- EXPECT_TRUE(strings::contains(
- error->message,
- "Protobuf union `mesos.ContainerInfo` with `Type == DOCKER` "
- "should not have the field `mesos` set."));
- }
}