You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by ji...@apache.org on 2018/08/07 00:04:30 UTC
[mesos] branch 1.4.x updated (a723132 -> 3f917b7)
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a change to branch 1.4.x
in repository https://gitbox.apache.org/repos/asf/mesos.git.
from a723132 Added MESOS-9125 to the 1.4.2 CHANGELOG.
new 29c4b5c Fixed the iptables deadlock in CNI port mapper plugin.
new 3f917b7 Added MESOS-9127 to 1.4.2 CHANGELOG.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGELOG | 1 +
.../cni/plugins/port_mapper/port_mapper.cpp | 32 ++++++++++++++++++----
2 files changed, 28 insertions(+), 5 deletions(-)
[mesos] 01/02: Fixed the iptables deadlock in CNI port mapper
plugin.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.4.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 29c4b5c3ad16e3e5c1e2ca8022699faf46cd2d7e
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Wed Aug 1 21:51:08 2018 -0700
Fixed the iptables deadlock in CNI port mapper plugin.
It is possible that the port mapping cleanup command will cause iptables
to deadlock if there are a lot of entires in the iptables, because the
`sed` won't process the next line while executing `iptables -w -t nat -D
...`. But the executing of `iptables -w -t nat -D ...` might get stuck
if the first command `iptables -w -t nat -S <TAG>` didn't finish
(because the xtables lock is not released). The first command might not
finish if it has a lot of output, filling the pipe that `sed` hasn't had
a chance to process yet. See more details in MESOS-9127.
This patch fixed the issue by writing the commands to a file and then
executing them.
Review: https://reviews.apache.org/r/68158/
---
.../cni/plugins/port_mapper/port_mapper.cpp | 32 ++++++++++++++++++----
1 file changed, 27 insertions(+), 5 deletions(-)
diff --git a/src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp b/src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
index 22e05de..78c23cf 100644
--- a/src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
+++ b/src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
@@ -351,16 +351,38 @@ Try<Nothing> PortMapper::addPortMapping(
Try<Nothing> PortMapper::delPortMapping()
{
+ // The iptables command searches for the DNAT rules with tag
+ // "container_id: <CNI_CONTAINERID>", and if it exists goes ahead
+ // and deletes it.
+ //
+ // NOTE: We use a temp file here, instead of letting `sed` directly
+ // executing the iptables commands because otherwise, it is possible
+ // that the port mapping cleanup command will cause iptables to
+ // deadlock if there are a lot of entires in the iptables, because
+ // the `sed` won't process the next line while executing `iptables
+ // -w -t nat -D ...`. But the executing of `iptables -w -t nat -D
+ // ...` might get stuck if the first command `iptables -w -t nat -S
+ // <TAG>` didn't finish (because the xtables lock is not released).
+ // The first command might not finish if it has a lot of output,
+ // filling the pipe that `sed` hasn't had a chance to process yet.
+ // See details in MESOS-9127.
string script = strings::format(
R"~(
#!/bin/sh
- exec 1>&2
set -x
+ set -e
+
+ FILE=$(mktemp)
+
+ cleanup() {
+ rm -f "$FILE"
+ }
+
+ trap cleanup EXIT
- # The iptables command searches for the DNAT rules with tag
- # "container_id: <CNI_CONTAINERID>", and if it exists goes ahead
- # and deletes it.
- iptables -w -t nat -S %s | sed "/%s/ s/-A/iptables -w -t nat -D/e")~",
+ iptables -w -t nat -S %s | sed -n "/%s/ s/-A/iptables -w -t nat -D/p" > $FILE
+ sh $FILE
+ )~",
chain,
getIptablesRuleTag()).get();
[mesos] 02/02: Added MESOS-9127 to 1.4.2 CHANGELOG.
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jieyu pushed a commit to branch 1.4.x
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 3f917b7f1675e0ba22133a00c614988b0dc30f9a
Author: Jie Yu <yu...@gmail.com>
AuthorDate: Mon Aug 6 17:02:53 2018 -0700
Added MESOS-9127 to 1.4.2 CHANGELOG.
---
CHANGELOG | 1 +
1 file changed, 1 insertion(+)
diff --git a/CHANGELOG b/CHANGELOG
index 42dde34..ec49cd2 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -48,6 +48,7 @@ Release Notes - Mesos - Version 1.4.2 (WIP)
* [MESOS-8987] - Master asks agent to shutdown upon auth errors.
* [MESOS-9049] - Agent GC could unmount a dangling persistent volume multiple times.
* [MESOS-9125] - Port mapper CNI plugin might fail with "Resource temporarily unavailable"
+ * [MESOS-9127] - Port mapper CNI plugin might deadlock iptables on the agent.
Release Notes - Mesos - Version 1.4.1