custom(ized) conteinerization

[Alex Glikson <GL...@il.ibm.com>]

Triggered by the thread on potential deprecation of external 
containerizer, I wonder what would make sense to do to address the 
following set of requirements:
1. I need resource isolation for individual tasks (mainly for QoS 
reasons), so having container per task seems reasonable
2. I have rather advanced networking requirements, not easily addressable 
with default mesos containerizer or docker
3. Some of the tasks are stateful, so I would really prefer that Mesos 
doesn't kill them, pretty much ever (unless triggered by the framework)

It seems that having my own containerizer would be a reasonable approach. 
But given some of the requirements above, I am trying to figure out 
whether I would at all need to implement "usage" and "update" (and maybe 
even 'destroy', unless it is invoked as part of killTask received from the 
framework?).

Moreover, the isolation mechanism I have in mind does use the same Linux 
features as docker/mesos containerizers (cgroups, namespaces, etc), but in 
a somewhat different manner. So, I wonder whether I can use more than one 
containerizer on the same host -- e.g., to run tasks of my framework on 
the same host as tasks of , say, Marathon+docker (and if yes, how can I 
check whether they will work together). If mixing containerizers in the 
same host is not recommended, is there an easy way to dynamically decide 
which slaves are 'allocated' to which 'type' of resources (e.g., some sort 
of entire-host allocation policy)?

Some thoughts/advice would be really helpful, before we actually spend 
time implementing a new containerizer, one way or another.

Thanks!
Alex

P.S. Disclaimer: I am new to Mesos, so maybe some (or all) of the above 
doesn't make much sense, so bear with me..

Re: custom(ized) conteinerization

[Benjamin Mahler <be...@gmail.com>]

Alex, yes, you need to figure out if they can conflict.

Ideally implementations avoid conflicting as much as possible but in some
cases it may not be possible to avoid.

On Sat, Oct 10, 2015 at 12:31 AM, Alex Glikson <GL...@il.ibm.com> wrote:

> Timothy Chen <tn...@gmail.com> wrote on 10/10/2015 09:38:38 AM:
> >
> > As Ben said containierizers don't conflict since the framework
> > specifies which containerizer the task should be launched from.
> >
> > And isolators are not configurable per task but per slave, so all
> > tasks will use the same isolators in the same slave.
>
> Tasks of the same framework -- sure. But what about tasks/containers of
> different frameworks running on the same host? For example, containerizer
> A might assume that all the network traffic should go via bridge X, while
> containerizer B might assume OVS+vxlan. In such a case the two are likely
> to be incompatible (maybe even cluster-wide, but most certainly within a
> host). I assume that conflicts are also possible with other resources
> (e.g., incompatible ways of using cgroups).
>
> Regards,
> Alex
>
> >
> > Tim
> >
> > > On Oct 9, 2015, at 11:22 PM, Alex Glikson <GL...@il.ibm.com> wrote:
> > >
> > > Thanks!
> > >
> > >> You can mix containerizers, although they should not conflict with
> each
> > >> other.
> > >
> > > How would I know whether they conflict? For example, the docker one
> and
> > > the default mesos one with certain configuration of isolators etc?
> > >
> > > Thanks,
> > > Alex
> > >
> > >
> > > Benjamin Mahler <be...@gmail.com> wrote on 10/10/2015
> 03:51:56
> > > AM:
> > >
> > >> From: Benjamin Mahler <be...@gmail.com>
> > >> To: dev <de...@mesos.apache.org>
> > >> Date: 10/10/2015 03:52 AM
> > >> Subject: Re: custom(ized) conteinerization
> > >>
> > >> (1) is something that has come up before. The containerizer deals
> with a
> > >> variable sized container, the semantics of a task are defined by the
> > >> executor, there is no way for the containerizer to understand the
> > > meaning
> > >> of a task currently. Some tasks are "special" in that they don't have
> an
> > >> executor (e.g. command task, docker task, etc), and in this case they
> > > will
> > >> be isolated individually. The main approach that has been discussed
> to
> > > my
> > >> knowledge is to have the executor leverage the mesos containerizer to
> > >> create nested containers:
> > > https://issues.apache.org/jira/browse/MESOS-3333
> > >>
> > >> For (2) you should implement a custom network _isolator_ that the
> mesos
> > >> containerizer can use.
> > >>
> > >> With respect to (3), for regular resources the policy used in Mesos
> is
> > > that
> > >> Mesos will never make decisions to kill things, it must be triggered
> by
> > > the
> > >> framework or an operator. So this requirement should be met already.
> For
> > >> revocable resources, Mesos may destroy containers, but the framework
> > > will
> > >> be aware of that when deciding to use them. If they are stateful, you
> > >> should use reservations and store the data in a volume.
> > >>
> > >> You can mix containerizers, although they should not conflict with
> each
> > >> other.
> > >>
> > >>> On Fri, Oct 9, 2015 at 3:04 AM, Alex Glikson <GL...@il.ibm.com>
> wrote:
> > >>>
> > >>> Triggered by the thread on potential deprecation of external
> > >>> containerizer, I wonder what would make sense to do to address the
> > >>> following set of requirements:
> > >>> 1. I need resource isolation for individual tasks (mainly for QoS
> > >>> reasons), so having container per task seems reasonable
> > >>> 2. I have rather advanced networking requirements, not easily
> > > addressable
> > >>> with default mesos containerizer or docker
> > >>> 3. Some of the tasks are stateful, so I would really prefer that
> Mesos
> > >>> doesn't kill them, pretty much ever (unless triggered by the
> > > framework)
> > >>>
> > >>> It seems that having my own containerizer would be a reasonable
> > > approach.
> > >>> But given some of the requirements above, I am trying to figure out
> > >>> whether I would at all need to implement "usage" and "update" (and
> > > maybe
> > >>> even 'destroy', unless it is invoked as part of killTask received
> from
> > > the
> > >>> framework?).
> > >>>
> > >>> Moreover, the isolation mechanism I have in mind does use the same
> > > Linux
> > >>> features as docker/mesos containerizers (cgroups, namespaces, etc),
> > > but in
> > >>> a somewhat different manner. So, I wonder whether I can use more
> than
> > > one
> > >>> containerizer on the same host -- e.g., to run tasks of my framework
> > > on
> > >>> the same host as tasks of , say, Marathon+docker (and if yes, how
> can
> > > I
> > >>> check whether they will work together). If mixing containerizers in
> > > the
> > >>> same host is not recommended, is there an easy way to dynamically
> > > decide
> > >>> which slaves are 'allocated' to which 'type' of resources (e.g.,
> some
> > > sort
> > >>> of entire-host allocation policy)?
> > >>>
> > >>> Some thoughts/advice would be really helpful, before we actually
> spend
> > >>> time implementing a new containerizer, one way or another.
> > >>>
> > >>> Thanks!
> > >>> Alex
> > >>>
> > >>> P.S. Disclaimer: I am new to Mesos, so maybe some (or all) of the
> > > above
> > >>> doesn't make much sense, so bear with me..
> > >
> > >
> >
>
>
>

Re: custom(ized) conteinerization

[Alex Glikson <GL...@il.ibm.com>]

Timothy Chen <tn...@gmail.com> wrote on 10/10/2015 09:38:38 AM:
> 
> As Ben said containierizers don't conflict since the framework 
> specifies which containerizer the task should be launched from.
> 
> And isolators are not configurable per task but per slave, so all 
> tasks will use the same isolators in the same slave.

Tasks of the same framework -- sure. But what about tasks/containers of 
different frameworks running on the same host? For example, containerizer 
A might assume that all the network traffic should go via bridge X, while 
containerizer B might assume OVS+vxlan. In such a case the two are likely 
to be incompatible (maybe even cluster-wide, but most certainly within a 
host). I assume that conflicts are also possible with other resources 
(e.g., incompatible ways of using cgroups).

Regards,
Alex

> 
> Tim
> 
> > On Oct 9, 2015, at 11:22 PM, Alex Glikson <GL...@il.ibm.com> wrote:
> > 
> > Thanks!
> > 
> >> You can mix containerizers, although they should not conflict with 
each
> >> other.
> > 
> > How would I know whether they conflict? For example, the docker one 
and 
> > the default mesos one with certain configuration of isolators etc?
> > 
> > Thanks,
> > Alex
> > 
> > 
> > Benjamin Mahler <be...@gmail.com> wrote on 10/10/2015 
03:51:56 
> > AM:
> > 
> >> From: Benjamin Mahler <be...@gmail.com>
> >> To: dev <de...@mesos.apache.org>
> >> Date: 10/10/2015 03:52 AM
> >> Subject: Re: custom(ized) conteinerization
> >> 
> >> (1) is something that has come up before. The containerizer deals 
with a
> >> variable sized container, the semantics of a task are defined by the
> >> executor, there is no way for the containerizer to understand the
> > meaning
> >> of a task currently. Some tasks are "special" in that they don't have 
an
> >> executor (e.g. command task, docker task, etc), and in this case they
> > will
> >> be isolated individually. The main approach that has been discussed 
to
> > my
> >> knowledge is to have the executor leverage the mesos containerizer to
> >> create nested containers:
> > https://issues.apache.org/jira/browse/MESOS-3333
> >> 
> >> For (2) you should implement a custom network _isolator_ that the 
mesos
> >> containerizer can use.
> >> 
> >> With respect to (3), for regular resources the policy used in Mesos 
is
> > that
> >> Mesos will never make decisions to kill things, it must be triggered 
by
> > the
> >> framework or an operator. So this requirement should be met already. 
For
> >> revocable resources, Mesos may destroy containers, but the framework
> > will
> >> be aware of that when deciding to use them. If they are stateful, you
> >> should use reservations and store the data in a volume.
> >> 
> >> You can mix containerizers, although they should not conflict with 
each
> >> other.
> >> 
> >>> On Fri, Oct 9, 2015 at 3:04 AM, Alex Glikson <GL...@il.ibm.com> 
wrote:
> >>> 
> >>> Triggered by the thread on potential deprecation of external
> >>> containerizer, I wonder what would make sense to do to address the
> >>> following set of requirements:
> >>> 1. I need resource isolation for individual tasks (mainly for QoS
> >>> reasons), so having container per task seems reasonable
> >>> 2. I have rather advanced networking requirements, not easily
> > addressable
> >>> with default mesos containerizer or docker
> >>> 3. Some of the tasks are stateful, so I would really prefer that 
Mesos
> >>> doesn't kill them, pretty much ever (unless triggered by the
> > framework)
> >>> 
> >>> It seems that having my own containerizer would be a reasonable
> > approach.
> >>> But given some of the requirements above, I am trying to figure out
> >>> whether I would at all need to implement "usage" and "update" (and
> > maybe
> >>> even 'destroy', unless it is invoked as part of killTask received 
from
> > the
> >>> framework?).
> >>> 
> >>> Moreover, the isolation mechanism I have in mind does use the same
> > Linux
> >>> features as docker/mesos containerizers (cgroups, namespaces, etc),
> > but in
> >>> a somewhat different manner. So, I wonder whether I can use more 
than
> > one
> >>> containerizer on the same host -- e.g., to run tasks of my framework
> > on
> >>> the same host as tasks of , say, Marathon+docker (and if yes, how 
can
> > I
> >>> check whether they will work together). If mixing containerizers in
> > the
> >>> same host is not recommended, is there an easy way to dynamically
> > decide
> >>> which slaves are 'allocated' to which 'type' of resources (e.g., 
some
> > sort
> >>> of entire-host allocation policy)?
> >>> 
> >>> Some thoughts/advice would be really helpful, before we actually 
spend
> >>> time implementing a new containerizer, one way or another.
> >>> 
> >>> Thanks!
> >>> Alex
> >>> 
> >>> P.S. Disclaimer: I am new to Mesos, so maybe some (or all) of the
> > above
> >>> doesn't make much sense, so bear with me..
> > 
> > 
> 

Re: custom(ized) conteinerization

[Kapil Arya <ka...@mesosphere.io>]

On Sat, Oct 10, 2015 at 2:38 AM, Timothy Chen <tn...@gmail.com> wrote:

> As Ben said containierizers don't conflict since the framework specifies
> which containerizer the task should be launched from.
>
> And isolators are not configurable per task but per slave, so all tasks
> will use the same isolators in the same slave.
>

There is a Jira (MESOS-3361
<https://issues.apache.org/jira/browse/MESOS-3361>) to allow slaves to
pick/enable Isolators per-task. That in combination with isolator
capabilities (MESOS-3362 <https://issues.apache.org/jira/browse/MESOS-3362>)
can help the framework to specify custom isolators.

Alternatively, as Ben suggested, one can write a custom network-isolator
that looks at TaskInfo/ExecutorInfo to decide what "kind" of
network-isolation to enable for the current task.



> Tim
>
> > On Oct 9, 2015, at 11:22 PM, Alex Glikson <GL...@il.ibm.com> wrote:
> >
> > Thanks!
> >
> >> You can mix containerizers, although they should not conflict with each
> >> other.
> >
> > How would I know whether they conflict? For example, the docker one and
> > the default mesos one with certain configuration of isolators etc?
> >
> > Thanks,
> > Alex
> >
> >
> > Benjamin Mahler <be...@gmail.com> wrote on 10/10/2015 03:51:56
> > AM:
> >
> >> From: Benjamin Mahler <be...@gmail.com>
> >> To: dev <de...@mesos.apache.org>
> >> Date: 10/10/2015 03:52 AM
> >> Subject: Re: custom(ized) conteinerization
> >>
> >> (1) is something that has come up before. The containerizer deals with a
> >> variable sized container, the semantics of a task are defined by the
> >> executor, there is no way for the containerizer to understand the
> > meaning
> >> of a task currently. Some tasks are "special" in that they don't have an
> >> executor (e.g. command task, docker task, etc), and in this case they
> > will
> >> be isolated individually. The main approach that has been discussed to
> > my
> >> knowledge is to have the executor leverage the mesos containerizer to
> >> create nested containers:
> > https://issues.apache.org/jira/browse/MESOS-3333
> >>
> >> For (2) you should implement a custom network _isolator_ that the mesos
> >> containerizer can use.
> >>
> >> With respect to (3), for regular resources the policy used in Mesos is
> > that
> >> Mesos will never make decisions to kill things, it must be triggered by
> > the
> >> framework or an operator. So this requirement should be met already. For
> >> revocable resources, Mesos may destroy containers, but the framework
> > will
> >> be aware of that when deciding to use them. If they are stateful, you
> >> should use reservations and store the data in a volume.
> >>
> >> You can mix containerizers, although they should not conflict with each
> >> other.
> >>
> >>> On Fri, Oct 9, 2015 at 3:04 AM, Alex Glikson <GL...@il.ibm.com>
> wrote:
> >>>
> >>> Triggered by the thread on potential deprecation of external
> >>> containerizer, I wonder what would make sense to do to address the
> >>> following set of requirements:
> >>> 1. I need resource isolation for individual tasks (mainly for QoS
> >>> reasons), so having container per task seems reasonable
> >>> 2. I have rather advanced networking requirements, not easily
> > addressable
> >>> with default mesos containerizer or docker
> >>> 3. Some of the tasks are stateful, so I would really prefer that Mesos
> >>> doesn't kill them, pretty much ever (unless triggered by the
> > framework)
> >>>
> >>> It seems that having my own containerizer would be a reasonable
> > approach.
> >>> But given some of the requirements above, I am trying to figure out
> >>> whether I would at all need to implement "usage" and "update" (and
> > maybe
> >>> even 'destroy', unless it is invoked as part of killTask received from
> > the
> >>> framework?).
> >>>
> >>> Moreover, the isolation mechanism I have in mind does use the same
> > Linux
> >>> features as docker/mesos containerizers (cgroups, namespaces, etc),
> > but in
> >>> a somewhat different manner. So, I wonder whether I can use more than
> > one
> >>> containerizer on the same host -- e.g., to run tasks of my framework
> > on
> >>> the same host as tasks of , say, Marathon+docker (and if yes, how can
> > I
> >>> check whether they will work together). If mixing containerizers in
> > the
> >>> same host is not recommended, is there an easy way to dynamically
> > decide
> >>> which slaves are 'allocated' to which 'type' of resources (e.g., some
> > sort
> >>> of entire-host allocation policy)?
> >>>
> >>> Some thoughts/advice would be really helpful, before we actually spend
> >>> time implementing a new containerizer, one way or another.
> >>>
> >>> Thanks!
> >>> Alex
> >>>
> >>> P.S. Disclaimer: I am new to Mesos, so maybe some (or all) of the
> > above
> >>> doesn't make much sense, so bear with me..
> >
> >
>

Re: custom(ized) conteinerization

[Timothy Chen <tn...@gmail.com>]

As Ben said containierizers don't conflict since the framework specifies which containerizer the task should be launched from.

And isolators are not configurable per task but per slave, so all tasks will use the same isolators in the same slave.

Tim

> On Oct 9, 2015, at 11:22 PM, Alex Glikson <GL...@il.ibm.com> wrote:
> 
> Thanks!
> 
>> You can mix containerizers, although they should not conflict with each
>> other.
> 
> How would I know whether they conflict? For example, the docker one and 
> the default mesos one with certain configuration of isolators etc?
> 
> Thanks,
> Alex
> 
> 
> Benjamin Mahler <be...@gmail.com> wrote on 10/10/2015 03:51:56 
> AM:
> 
>> From: Benjamin Mahler <be...@gmail.com>
>> To: dev <de...@mesos.apache.org>
>> Date: 10/10/2015 03:52 AM
>> Subject: Re: custom(ized) conteinerization
>> 
>> (1) is something that has come up before. The containerizer deals with a
>> variable sized container, the semantics of a task are defined by the
>> executor, there is no way for the containerizer to understand the
> meaning
>> of a task currently. Some tasks are "special" in that they don't have an
>> executor (e.g. command task, docker task, etc), and in this case they
> will
>> be isolated individually. The main approach that has been discussed to
> my
>> knowledge is to have the executor leverage the mesos containerizer to
>> create nested containers:
> https://issues.apache.org/jira/browse/MESOS-3333
>> 
>> For (2) you should implement a custom network _isolator_ that the mesos
>> containerizer can use.
>> 
>> With respect to (3), for regular resources the policy used in Mesos is
> that
>> Mesos will never make decisions to kill things, it must be triggered by
> the
>> framework or an operator. So this requirement should be met already. For
>> revocable resources, Mesos may destroy containers, but the framework
> will
>> be aware of that when deciding to use them. If they are stateful, you
>> should use reservations and store the data in a volume.
>> 
>> You can mix containerizers, although they should not conflict with each
>> other.
>> 
>>> On Fri, Oct 9, 2015 at 3:04 AM, Alex Glikson <GL...@il.ibm.com> wrote:
>>> 
>>> Triggered by the thread on potential deprecation of external
>>> containerizer, I wonder what would make sense to do to address the
>>> following set of requirements:
>>> 1. I need resource isolation for individual tasks (mainly for QoS
>>> reasons), so having container per task seems reasonable
>>> 2. I have rather advanced networking requirements, not easily
> addressable
>>> with default mesos containerizer or docker
>>> 3. Some of the tasks are stateful, so I would really prefer that Mesos
>>> doesn't kill them, pretty much ever (unless triggered by the
> framework)
>>> 
>>> It seems that having my own containerizer would be a reasonable
> approach.
>>> But given some of the requirements above, I am trying to figure out
>>> whether I would at all need to implement "usage" and "update" (and
> maybe
>>> even 'destroy', unless it is invoked as part of killTask received from
> the
>>> framework?).
>>> 
>>> Moreover, the isolation mechanism I have in mind does use the same
> Linux
>>> features as docker/mesos containerizers (cgroups, namespaces, etc),
> but in
>>> a somewhat different manner. So, I wonder whether I can use more than
> one
>>> containerizer on the same host -- e.g., to run tasks of my framework
> on
>>> the same host as tasks of , say, Marathon+docker (and if yes, how can
> I
>>> check whether they will work together). If mixing containerizers in
> the
>>> same host is not recommended, is there an easy way to dynamically
> decide
>>> which slaves are 'allocated' to which 'type' of resources (e.g., some
> sort
>>> of entire-host allocation policy)?
>>> 
>>> Some thoughts/advice would be really helpful, before we actually spend
>>> time implementing a new containerizer, one way or another.
>>> 
>>> Thanks!
>>> Alex
>>> 
>>> P.S. Disclaimer: I am new to Mesos, so maybe some (or all) of the
> above
>>> doesn't make much sense, so bear with me..
> 
> 

Re: custom(ized) conteinerization

[Alex Glikson <GL...@il.ibm.com>]

Thanks!

> You can mix containerizers, although they should not conflict with each
> other.

How would I know whether they conflict? For example, the docker one and 
the default mesos one with certain configuration of isolators etc?

Thanks,
Alex


Benjamin Mahler <be...@gmail.com> wrote on 10/10/2015 03:51:56 
AM:

> From: Benjamin Mahler <be...@gmail.com>
> To: dev <de...@mesos.apache.org>
> Date: 10/10/2015 03:52 AM
> Subject: Re: custom(ized) conteinerization
> 
> (1) is something that has come up before. The containerizer deals with a
> variable sized container, the semantics of a task are defined by the
> executor, there is no way for the containerizer to understand the 
meaning
> of a task currently. Some tasks are "special" in that they don't have an
> executor (e.g. command task, docker task, etc), and in this case they 
will
> be isolated individually. The main approach that has been discussed to 
my
> knowledge is to have the executor leverage the mesos containerizer to
> create nested containers: 
https://issues.apache.org/jira/browse/MESOS-3333
> 
> For (2) you should implement a custom network _isolator_ that the mesos
> containerizer can use.
> 
> With respect to (3), for regular resources the policy used in Mesos is 
that
> Mesos will never make decisions to kill things, it must be triggered by 
the
> framework or an operator. So this requirement should be met already. For
> revocable resources, Mesos may destroy containers, but the framework 
will
> be aware of that when deciding to use them. If they are stateful, you
> should use reservations and store the data in a volume.
> 
> You can mix containerizers, although they should not conflict with each
> other.
> 
> On Fri, Oct 9, 2015 at 3:04 AM, Alex Glikson <GL...@il.ibm.com> wrote:
> 
> > Triggered by the thread on potential deprecation of external
> > containerizer, I wonder what would make sense to do to address the
> > following set of requirements:
> > 1. I need resource isolation for individual tasks (mainly for QoS
> > reasons), so having container per task seems reasonable
> > 2. I have rather advanced networking requirements, not easily 
addressable
> > with default mesos containerizer or docker
> > 3. Some of the tasks are stateful, so I would really prefer that Mesos
> > doesn't kill them, pretty much ever (unless triggered by the 
framework)
> >
> > It seems that having my own containerizer would be a reasonable 
approach.
> > But given some of the requirements above, I am trying to figure out
> > whether I would at all need to implement "usage" and "update" (and 
maybe
> > even 'destroy', unless it is invoked as part of killTask received from 
the
> > framework?).
> >
> > Moreover, the isolation mechanism I have in mind does use the same 
Linux
> > features as docker/mesos containerizers (cgroups, namespaces, etc), 
but in
> > a somewhat different manner. So, I wonder whether I can use more than 
one
> > containerizer on the same host -- e.g., to run tasks of my framework 
on
> > the same host as tasks of , say, Marathon+docker (and if yes, how can 
I
> > check whether they will work together). If mixing containerizers in 
the
> > same host is not recommended, is there an easy way to dynamically 
decide
> > which slaves are 'allocated' to which 'type' of resources (e.g., some 
sort
> > of entire-host allocation policy)?
> >
> > Some thoughts/advice would be really helpful, before we actually spend
> > time implementing a new containerizer, one way or another.
> >
> > Thanks!
> > Alex
> >
> > P.S. Disclaimer: I am new to Mesos, so maybe some (or all) of the 
above
> > doesn't make much sense, so bear with me..
> >
> >
> >

Re: custom(ized) conteinerization

[Benjamin Mahler <be...@gmail.com>]

(1) is something that has come up before. The containerizer deals with a
variable sized container, the semantics of a task are defined by the
executor, there is no way for the containerizer to understand the meaning
of a task currently. Some tasks are "special" in that they don't have an
executor (e.g. command task, docker task, etc), and in this case they will
be isolated individually. The main approach that has been discussed to my
knowledge is to have the executor leverage the mesos containerizer to
create nested containers: https://issues.apache.org/jira/browse/MESOS-3333

For (2) you should implement a custom network _isolator_ that the mesos
containerizer can use.

With respect to (3), for regular resources the policy used in Mesos is that
Mesos will never make decisions to kill things, it must be triggered by the
framework or an operator. So this requirement should be met already. For
revocable resources, Mesos may destroy containers, but the framework will
be aware of that when deciding to use them. If they are stateful, you
should use reservations and store the data in a volume.

You can mix containerizers, although they should not conflict with each
other.

On Fri, Oct 9, 2015 at 3:04 AM, Alex Glikson <GL...@il.ibm.com> wrote:

> Triggered by the thread on potential deprecation of external
> containerizer, I wonder what would make sense to do to address the
> following set of requirements:
> 1. I need resource isolation for individual tasks (mainly for QoS
> reasons), so having container per task seems reasonable
> 2. I have rather advanced networking requirements, not easily addressable
> with default mesos containerizer or docker
> 3. Some of the tasks are stateful, so I would really prefer that Mesos
> doesn't kill them, pretty much ever (unless triggered by the framework)
>
> It seems that having my own containerizer would be a reasonable approach.
> But given some of the requirements above, I am trying to figure out
> whether I would at all need to implement "usage" and "update" (and maybe
> even 'destroy', unless it is invoked as part of killTask received from the
> framework?).
>
> Moreover, the isolation mechanism I have in mind does use the same Linux
> features as docker/mesos containerizers (cgroups, namespaces, etc), but in
> a somewhat different manner. So, I wonder whether I can use more than one
> containerizer on the same host -- e.g., to run tasks of my framework on
> the same host as tasks of , say, Marathon+docker (and if yes, how can I
> check whether they will work together). If mixing containerizers in the
> same host is not recommended, is there an easy way to dynamically decide
> which slaves are 'allocated' to which 'type' of resources (e.g., some sort
> of entire-host allocation policy)?
>
> Some thoughts/advice would be really helpful, before we actually spend
> time implementing a new containerizer, one way or another.
>
> Thanks!
> Alex
>
> P.S. Disclaimer: I am new to Mesos, so maybe some (or all) of the above
> doesn't make much sense, so bear with me..
>
>
>