Re: Enabling 'network' namespace for custom network isolators

[Kapil Arya <ka...@mesosphere.io>]

Hi All,

I have now filed a Jira ticket (MESOS-2884)[1] and created a couple of
RRs[2] along the lines of the suggestions provided by Ian and Jie.

There is one difference though. Instead of creating a LinuxIsolator class,
I added the `int namespaces()` directly to Isolator class. By default, it
returns '0' and thus allows for a simplified approach. By creating
LinuxIsolator, we would then have a complicated lookup logic in
MesosContainerizer to first figure out the subclass type and then call
namespaces() on it.

>From here on, I think we can continue the discussion on the Jira ticket if
no one has any objections.

Best,
Kapil

[1]. https://issues.apache.org/jira/browse/MESOS-2884
[2a]. https://reviews.apache.org/r/35585/
[2b]. https://reviews.apache.org/r/35586/

On Mon, May 11, 2015 at 7:42 PM, Niklas Nielsen <ni...@mesosphere.io>
wrote:

> (inlined)
>
> On 11 May 2015 at 14:30, Kapil Arya <ka...@mesosphere.io> wrote:
>
> > On Mon, May 11, 2015 at 4:58 PM, Jie Yu <yu...@gmail.com> wrote:
> >
> > > >
> > > > Yes. The simplest (cleanest?) way that I see would be to refactor the
> > > > launcher to take the desired flags when executing the executor, i.e.,
> > > > (Linux)Launcher::fork() takes the namespace flags. The launcher would
> > be
> > > > directed which namespaces to create by the caller, not inferring them
> > > > itself from any flags: the MesosContainerizer in turn would determine
> > > this
> > > > based on the isolators it was using for the container (querying
> them).
> > > This
> > > > also facilitates the MesosContainerizer having different isolators,
> and
> > > > thus namespaces, for different containers.
> > >
> > >
> > > +1
> > >
> > > For instance, the isolator could have an interface 'int namespaces()'
> > which
> > > specifies the namespaces needed. The launcher can just query that and
> > pass
> > > them to the linux launcher.
> > >
> > > Since currently, the launcher and isolator interfaces are designed for
> > both
> > > mac and linux and namespace does not make sense on Mac, we probably
> need
> > a
> > > LinuxIsolator interface (inherit from Isolator) and a LinuxLauncher
> > > interface (inherit from Launcher).
> > >
> >
> > This is a good idea. I think this will keep things pretty clean and
> > readable within Mesos and for the Isolators.
> >
> >
> > > - Jie
> > >
> > > On Mon, May 11, 2015 at 1:29 PM, Ian Downes
> <idownes@twitter.com.invalid
> > >
> > > wrote:
> > >
> > > > >
> > > > > TLDR: We want to use a custom network isolator, but there is no way
> > to
> > > > > enable the 'network' namespace from within an isolator module.
> > > > >
> > > > >
> > > > > We are working on creating a custom network isolator as a Mesos
> > module.
> > > > > However, the way Mesos Slave is setup, there is no way to enable
> > > > 'network'
> > > > > namespace for the executor without enabling the 'port-mapping'
> > > isolator.
> > > > > This is due to the fact that the LinuxLauncher looks at the
> > > '--isolation'
> > > > > flag to figure out the list of namespaces to be enabled. The same
> > > problem
> > > > > would exist if one were to write a custom pid or filesystem
> isolation
> > > > > module.
> > > > >
> > > >
> > > > Curious, are these going to be open source and added to the codebase
> or
> > > > will they be proprietary modules? What specifically is lacking in the
> > > > existing network and pid isolators? Could we extend those?
> >
>
> We will be bringing in a dependency, experimental work with Calico, and
> wanted to be flexible in how we call out to our tools.
>
>
> > > >
> > > > So, there are a couple of question:
> > > > >
> > > > > 1. With the current Mesos source code, is there a way to specify
> the
> > > > > 'network/port_mapping' isolator in a way that it doesn't do the
> > actual
> > > > work
> > > > > of mapping the ports (e.g., without specifying any port-mapping
> > > specific
> > > > > flags)? If this works, we can just specify this isolator on the
> slave
> > > > > command line and it would force the LinuxLauncher to create a new
> > > network
> > > > > namespace.
> > > >
> > > >
> > > > No, as written they're coupled.
> > > >
> > > > 2. Is it reasonable to have a separate mechanism to specify what
> > > namespaces
> > > > > should be created/enabled for an executor if we don't want to use
> the
> > > > > in-built isolators such as pid and port-mapping?
> > > >
> > > >
> > > > Yes. The simplest (cleanest?) way that I see would be to refactor the
> > > > launcher to take the desired flags when executing the executor, i.e.,
> > > > (Linux)Launcher::fork() takes the namespace flags. The launcher would
> > be
> > > > directed which namespaces to create by the caller, not inferring them
> > > > itself from any flags: the MesosContainerizer in turn would determine
> > > this
> > > > based on the isolators it was using for the container (querying
> them).
> > > This
> > > > also facilitates the MesosContainerizer having different isolators,
> and
> > > > thus namespaces, for different containers.
> > > >
> > > > WRT (2), one potential mechanism is to introduce a new flag,
> > > '--namespace'.
> > > > > The downside of creating such a low-level flag is that it provides
> > > little
> > > > > to no value to the end users. The end users shouldn't be concerned
> > > about
> > > > > which namespaces to enable and so on
> >
> > >
> > > >
> > > > That seems unduly onerous to the user and almost as rigid.
> >
>
> No matter if you refactor the way we tell the launcher to set the namespace
> flags, we need to refactor the way it is provided by the user to the slave.
> (Don't get me wrong - I do love the decoupling of the slave flags from the
> launcher :)
>
> I suggested that we create 'namespaces/network' instead of the
> --namespaces, which would be equivalent to the namespaces/pid
> isolator+launcher code.
>
> Thoughts?
>
>
> > > >
> > > >
> > > > > Another alternative is to create a decorator hook for the
> > > LinuxLauncher,
> > > > > which can force the LinuxLauncher to enable certain namespaces
> > without
> > > > > having to look at the '--isolation' flag. The downside here is that
> > the
> > > > > decorator will be literally setting up a few bits and nothing more.
> > > > >
> > > >
> > > > I don't think there's a need for a decorator hook, just refactoring
> to
> > > pass
> > > > in through fork() is sufficient?
> > > >
> > > > Are there any other alternatives for a better and cleaner design
> (both
> > > long
> > > > > term and short term)?
> > > > >
> > > >
> > > > Happy to chat further about this.
> > > >
> > > > ian
> > > >
> > >
> >
>