You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by al...@apache.org on 2016/12/22 14:12:57 UTC
[1/2] mesos git commit: Documentation for the port-mapper CNI plugin.
Repository: mesos
Updated Branches:
refs/heads/1.1.x 45e6b4451 -> 213bac632
Documentation for the port-mapper CNI plugin.
Review: https://reviews.apache.org/r/53015/
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/9b76ef13
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/9b76ef13
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/9b76ef13
Branch: refs/heads/1.1.x
Commit: 9b76ef13abdc46265dfdccfcdc4aa085a729189b
Parents: 45e6b44
Author: Avinash sridharan <av...@mesosphere.io>
Authored: Wed Nov 2 23:03:24 2016 -0700
Committer: Alexander Rukletsov <al...@apache.org>
Committed: Thu Dec 22 15:09:54 2016 +0100
----------------------------------------------------------------------
docs/cni.md | 126 ++++++++++++++++++++++++++++++++++++++++++++-----------
1 file changed, 102 insertions(+), 24 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/9b76ef13/docs/cni.md
----------------------------------------------------------------------
diff --git a/docs/cni.md b/docs/cni.md
index 2f7143c..70a2db6 100644
--- a/docs/cni.md
+++ b/docs/cni.md
@@ -22,13 +22,13 @@ default.
- [Configuring CNI networks](#configuring-cni-networks)
- [Attaching containers to CNI networks](#attaching-containers-to-cni-networks)
- [Accessing container network namespace](#accessing-container-network-namespace)
- - [Mesos meta-data to CNI plugins](#mesos-meta-data-to-cni-plugins)
+ - [Passing network labels and port-mapping information to CNI plugins](#mesos-meta-data-to-cni-plugins)
- [Adding/Deleting/Modifying CNI networks](#adding-modifying-deleting)
- [Networking Recipes](#networking-recipes)
- [A bridge network](#a-bridge-network)
+ - [A port-mapper plugin for CNI networks](#a-port-mapper-plugin)
- [A Calico network](#a-calico-network)
- [A Weave network](#a-weave-network)
-- [Limitations](#limitations)
### <a name="motivation"></a>Motivation
@@ -91,7 +91,7 @@ specifies two flags at Agent startup as follows:
sudo mesos-slave --master=<master IP> --ip=<Agent IP>
--work_dir=/var/lib/mesos
--network_cni_config_dir=<location of CNI configs>
- --network_cni_plugins_dir=<location of CNI plugsin>
+ --network_cni_plugins_dir=<search path for CNI plugins>
```
Note that the `network/cni` isolator learns all the available networks
@@ -132,7 +132,7 @@ will need to attach the container to a CNI network (such as
bridge/macvlan) that, in turn, is attached to the host network.
```
-#### <a name="mesos-meta-data-to-cni-plugins"></a>Mesos meta-data to CNI plugins
+#### <a name="network-labels-and-port-mapping">Passing network labels and port-mapping information to CNI plugins</a>
When invoking CNI plugins (e.g., with command ADD), the isolator will
pass on some Mesos meta-data to the plugins by specifying the `args`
@@ -156,19 +156,24 @@ instance:
{ "key" : "app", "value" : "myapp" },
{ "key" : "env", "value" : "prod" }
]
- }
+ },
+ "port_mappings" : [
+ { "host_port" : 8080, "container_port" : 80 },
+ { "host_port" : 8081, "container_port" : 443 }
+ ]
}
}
}
}
```
-It is important to note that `labels` within the `NetworkInfo` is set
-by frameworks launching the container, and the isolator passses on
-this information to the CNI plugins. As per the spec, it is the
-prerogative of the CNI plugins to use this meta-data information to
-enforce domain specific policies while attaching containers to a CNI
-network.
+It is important to note that `labels` or `port_mappings` within the
+`NetworkInfo` is set by frameworks launching the container, and the
+isolator passses on this information to the CNI plugins. As per the
+spec, it is the prerogative of the CNI plugins to use this meta-data
+information as they see fit while attaching/detaching containers to a CNI
+network. E.g., CNI plugins could use `labels` to enforce domain
+specific policies, or `port_mappings` to implement NAT rules.
#### <a name="accessing-container-network-namespace"></a>Accessing container network namespace
@@ -377,6 +382,92 @@ default via 192.168.0.1 dev eth0
192.168.0.0/16 dev eth0 proto kernel scope link src 192.168.0.2
```
+#### <a name="a-port-mapper-plugin">A port-mapper plugin for CNI networks</a>
+
+For private, isolated, networks such as a bridge network where the IP
+address of a container is not routeable from outside the host it
+becomes imperative to provide containers with DNAT capabilities so
+that services running on the container can be exposed outside the host
+on which the container is running.
+
+Unfortunately, there is no CNI plugin available in the
+[containernetworking/cni](https://github.com/containernetwork/cni)
+repository that provides port-mapping functionality.
+Hence, we have developed a port-mapper CNI plugin that resides
+within the Mesos code base called the `mesos-cni-port-mapper`. The
+`mesos-cni-port-mapper` is designed to work with any other CNI plugin
+that requires DNAT capabilities. One of the most obvious being the
+`bridge` CNI plugin.
+
+We explain the operational semantics of the `mesos-cni-port-mapper`
+plugin by taking an example CNI configuration that allows the
+`mesos-cni-port-mapper` to provide DNAT functionality to the `bridge`
+plugin.
+
+```
+{
+ "name" : "port-mapper-test",
+ "type" : "mesos-cni-port-mapper",
+ "excludeDevices" : ["mesos-cni0"],
+ "chain": "MESOS-TEST-PORT-MAPPER",
+ "delegate": {
+ "type": "bridge",
+ "bridge": "mesos-cni0",
+ "isGateway": true,
+ "ipMasq": true,
+ "ipam": {
+ "type": "host-local",
+ "subnet": "192.168.0.0/16",
+ "routes": [
+ { "dst":
+ "0.0.0.0/0" }
+ ]
+ }
+ }
+}
+```
+
+For the CNI configuration above, apart from the parameters that the
+`mesos-cni-port-mapper` plugin accepts, the important point to note in
+the CNI configuration of the plugin is the "delegate" field. The
+"delegate" field allows the `mesos-cni-port-mapper` to wrap the CNI
+configuration of any other CNI plugin, and allows the plugin to
+provide DNAT capabilities to any CNI network. In this specific case
+the `mesos-cni-port-mapper` is providing DNAT capabilities to
+containers running on the bridge network `mesos-cni0`. The parameters
+that the `mesos-cni-port-mapper` accepts are listed below:
+
+* ***name*** : Name of the CNI network.
+* ***type*** : Name of the port-mapper CNI plugin.
+* ***chain*** : The chain in which the iptables DNAT rule
+will be added in the NAT table. This allows the operator to group
+DNAT rules for a given CNI network under its own chain, allowing for
+better management of the iptables rules.
+* ***excludeDevices***: These are a list of ingress devices on which the
+DNAT rule should not be applied.
+* ***delegate*** : This is a JSON dict that holds the CNI JSON configuration
+of a CNI plugin that the port-mapper plugin is expected to invoke.
+
+The `mesos-cni-port-mapper` relies heavily on `iptables` to provide
+the DNAT capabilities to a CNI network. In order for the port-mapper
+plugin to function properly we have certain minimum
+**requirements for iptables** as listed below:
+
+* ***iptables 1.4.20 or higher***: This because we need to use the -w option
+of iptables in order to allow atomic writes to iptables.
+* ***Require the xt_comments module of iptables***: We use the comments
+module to tag iptables rules belonging to a container. These tags are used as a
+key while deleting iptables rules when the specific container is deleted.
+
+Finally, while the CNI configuration of the port-mapper plugin tells
+the plugin as to how and where to install the `iptables` rules, and which
+CNI plugin to "delegate" the attachment/detachment of the container, the
+port-mapping information itself is learned by looking at the
+`NetworkInfo` set in the `args` field of the CNI configuration passed
+by Mesos to the port-mapper plugin. Please refer to the "[Passing
+network labels and port-mapping information to CNI
+plugins](#mesos-meta-data-to-cni-plugins)" section for more details.
+
#### <a name="a-calico-network">A Calico network</a>
[Calico](https://projectcalico.org/) provides 3rd-party CNI plugin
@@ -412,16 +503,3 @@ maintain, even when there are multiple hops.
For more information on setting up and using Weave CNI, see [Weave's
CNI
documentation](https://www.weave.works/docs/net/latest/cni-plugin/)
-
-### <a name="limitations"></a>Limitations
-
-Currently the `network/cni` isolator does not provide any port mapping
-capabilities. Therefore if operators are running services on networks
-that are not addressable from outside the Agent host, the operators
-will need to run proxies/gateways for the services on the host network
-to direct traffic to their services. We plan to address this
-limitation by having a CNI plugin, within the Mesos repository, that
-provides port mapping functionality and can be used with any other CNI
-plugin, such as the CNI bridge plugin. We are tracking this effort
-through
-[MESOS-6014](https://issues.apache.org/jira/browse/MESOS-6014).
[2/2] mesos git commit: Added MESOS-6411 to CHANGELOG for 1.1.1.
Posted by al...@apache.org.
Added MESOS-6411 to CHANGELOG for 1.1.1.
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/213bac63
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/213bac63
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/213bac63
Branch: refs/heads/1.1.x
Commit: 213bac632853d4492800f0b02c9f606fdd590f84
Parents: 9b76ef1
Author: Alexander Rukletsov <al...@apache.org>
Authored: Thu Dec 22 15:12:13 2016 +0100
Committer: Alexander Rukletsov <al...@apache.org>
Committed: Thu Dec 22 15:12:13 2016 +0100
----------------------------------------------------------------------
CHANGELOG | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/213bac63/CHANGELOG
----------------------------------------------------------------------
diff --git a/CHANGELOG b/CHANGELOG
index ea757ea..d4024b5 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -7,6 +7,7 @@ All Issues:
* [MESOS-6002] - The whiteout file cannot be removed correctly using aufs backend.
* [MESOS-6142] - Frameworks may RESERVE for an arbitrary role.
* [MESOS-6360] - The handling of whiteout files in provisioner is not correct.
+ * [MESOS-6411] - Add documentation for CNI port-mapper plugin.
* [MESOS-6526] - `mesos-containerizer launch --environment` exposes executor env vars in `ps`.
* [MESOS-6621] - SSL downgrade path will CHECK-fail when using both temporary and persistent sockets