You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Marcus Sorensen <sh...@gmail.com> on 2017/03/20 16:50:52 UTC
CNI: Mesos containers need to access Mesos agent
http://mesos.apache.org/documentation/latest/cni/
"For Mesos, the executors launched as containers need to register with the Agent in order for a task to be successfully launched. Hence, it is imperative that the Agent IP is reachable from the container IP and vice versa. "
Can anyone shed some light on this requirement for me? We'd like to understand the purpose of this to determine if we can work around it or find some means of securing it. We are really focusing on network security and isolation in our CNI design, we'd prefer to maintain network isolation between the Mesos containers and hosts.
In particular, if we have to work around it, I'm wondering if there'd be any opportunity for the CNI plugin to open access to the port for just a short period until registration, then firewall it off and what the behavior might be if there is not continual access. Or perhaps we add a link local interface of some sort and a route, such that individual containers can reach their agent but the Mesos container networks don't need to be generally open to the Mesos host networks.
Re: CNI: Mesos containers need to access Mesos agent
Posted by tommy xiao <xi...@gmail.com>.
+1
2017-03-21 7:16 GMT+08:00 Marcus Sorensen <sh...@gmail.com>:
>
> Thanks. Good to see it is already being tracked.
>
> On 2017-03-20 14:39 (-0600), Avinash Sridharan <av...@mesosphere.io>
> wrote:
> > Hi Marcus,
> > The reason we need connectivity from the container's network namespace
> to
> > the host network namespace is that the Mesos executor running in the
> > container's network namespace needs to register back with the agent in
> > order to send TASK updates about the container to the agent. Without this
> > connectivity the agent will not know if the container has started
> > successfully and will simply kill the container, failing the container
> > launch.
> >
> > I know this is a restriction on some virtual networking solutions, and
> > going forward the right solution would be to support agent/executor
> > communication over domain sockets:
> > https://issues.apache.org/jira/browse/MESOS-6240
> >
> > We still need to figure out when that can be accomplished.
> >
> > In terms of the work arounds, if you can open communication to port 5051
> > between the host network namespace and the container's network namespace
> it
> > should just work.
> >
> > On Mon, Mar 20, 2017 at 9:50 AM, Marcus Sorensen <sh...@gmail.com>
> > wrote:
> >
> > > http://mesos.apache.org/documentation/latest/cni/
> > >
> > > "For Mesos, the executors launched as containers need to register with
> the
> > > Agent in order for a task to be successfully launched. Hence, it is
> > > imperative that the Agent IP is reachable from the container IP and
> vice
> > > versa. "
> > >
> > > Can anyone shed some light on this requirement for me? We'd like to
> > > understand the purpose of this to determine if we can work around it or
> > > find some means of securing it. We are really focusing on network
> security
> > > and isolation in our CNI design, we'd prefer to maintain network
> isolation
> > > between the Mesos containers and hosts.
> > >
> > > In particular, if we have to work around it, I'm wondering if there'd
> be
> > > any opportunity for the CNI plugin to open access to the port for just
> a
> > > short period until registration, then firewall it off and what the
> behavior
> > > might be if there is not continual access. Or perhaps we add a link
> local
> > > interface of some sort and a route, such that individual containers can
> > > reach their agent but the Mesos container networks don't need to be
> > > generally open to the Mesos host networks.
> > >
> >
> >
> >
> > --
> > Avinash Sridharan, Mesosphere
> > +1 (323) 702 5245 <(323)%20702-5245>
> >
>
--
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com
Re: CNI: Mesos containers need to access Mesos agent
Posted by Marcus Sorensen <sh...@gmail.com>.
Thanks. Good to see it is already being tracked.
On 2017-03-20 14:39 (-0600), Avinash Sridharan <av...@mesosphere.io> wrote:
> Hi Marcus,
> The reason we need connectivity from the container's network namespace to
> the host network namespace is that the Mesos executor running in the
> container's network namespace needs to register back with the agent in
> order to send TASK updates about the container to the agent. Without this
> connectivity the agent will not know if the container has started
> successfully and will simply kill the container, failing the container
> launch.
>
> I know this is a restriction on some virtual networking solutions, and
> going forward the right solution would be to support agent/executor
> communication over domain sockets:
> https://issues.apache.org/jira/browse/MESOS-6240
>
> We still need to figure out when that can be accomplished.
>
> In terms of the work arounds, if you can open communication to port 5051
> between the host network namespace and the container's network namespace it
> should just work.
>
> On Mon, Mar 20, 2017 at 9:50 AM, Marcus Sorensen <sh...@gmail.com>
> wrote:
>
> > http://mesos.apache.org/documentation/latest/cni/
> >
> > "For Mesos, the executors launched as containers need to register with the
> > Agent in order for a task to be successfully launched. Hence, it is
> > imperative that the Agent IP is reachable from the container IP and vice
> > versa. "
> >
> > Can anyone shed some light on this requirement for me? We'd like to
> > understand the purpose of this to determine if we can work around it or
> > find some means of securing it. We are really focusing on network security
> > and isolation in our CNI design, we'd prefer to maintain network isolation
> > between the Mesos containers and hosts.
> >
> > In particular, if we have to work around it, I'm wondering if there'd be
> > any opportunity for the CNI plugin to open access to the port for just a
> > short period until registration, then firewall it off and what the behavior
> > might be if there is not continual access. Or perhaps we add a link local
> > interface of some sort and a route, such that individual containers can
> > reach their agent but the Mesos container networks don't need to be
> > generally open to the Mesos host networks.
> >
>
>
>
> --
> Avinash Sridharan, Mesosphere
> +1 (323) 702 5245 <(323)%20702-5245>
>
Re: CNI: Mesos containers need to access Mesos agent
Posted by Avinash Sridharan <av...@mesosphere.io>.
Hi Marcus,
The reason we need connectivity from the container's network namespace to
the host network namespace is that the Mesos executor running in the
container's network namespace needs to register back with the agent in
order to send TASK updates about the container to the agent. Without this
connectivity the agent will not know if the container has started
successfully and will simply kill the container, failing the container
launch.
I know this is a restriction on some virtual networking solutions, and
going forward the right solution would be to support agent/executor
communication over domain sockets:
https://issues.apache.org/jira/browse/MESOS-6240
We still need to figure out when that can be accomplished.
In terms of the work arounds, if you can open communication to port 5051
between the host network namespace and the container's network namespace it
should just work.
On Mon, Mar 20, 2017 at 9:50 AM, Marcus Sorensen <sh...@gmail.com>
wrote:
> http://mesos.apache.org/documentation/latest/cni/
>
> "For Mesos, the executors launched as containers need to register with the
> Agent in order for a task to be successfully launched. Hence, it is
> imperative that the Agent IP is reachable from the container IP and vice
> versa. "
>
> Can anyone shed some light on this requirement for me? We'd like to
> understand the purpose of this to determine if we can work around it or
> find some means of securing it. We are really focusing on network security
> and isolation in our CNI design, we'd prefer to maintain network isolation
> between the Mesos containers and hosts.
>
> In particular, if we have to work around it, I'm wondering if there'd be
> any opportunity for the CNI plugin to open access to the port for just a
> short period until registration, then firewall it off and what the behavior
> might be if there is not continual access. Or perhaps we add a link local
> interface of some sort and a route, such that individual containers can
> reach their agent but the Mesos container networks don't need to be
> generally open to the Mesos host networks.
>
--
Avinash Sridharan, Mesosphere
+1 (323) 702 5245 <(323)%20702-5245>