process isolation

[Paul Mackles <pa...@loopr.com>]

Hi - I just wanted to confirm my understanding of something... with process
isolation, Mesos will not do anything if a given executor exceeds its
resource allocation. In other words, if I accept a resource with 1GB of
memory and then my executor uses 3GB, Mesos won't detect that the process
exceeded its allocation and kill the process. For that, you need to enable
cgroups at which point allocation limits are enforced by the OS. Did I get
that right?

-- 
Thanks,
Paul

Re: process isolation

[Sam Taha <ta...@gmail.com>]

See comments from Ben Mahler on related question about isolation and using
cgroups with and without cpu subsystems and cfs enforced:

If using process isolation nothing is enforced.

If using cgroups isolation:

  with no subsystems: nothing is enforced.

  with the 'cpu' subsystem: this will enforce your cgroup gets at least 0.5
cpu resources. Think of this as a lower bound guarantee on your cpu quota.
You can use more if the system is otherwise idle.

  with the 'cpu' subsystem and with --cgroups_enable_cfs: this will enforce
your application gets at least 0.5 cpu resources and no more than 0.5 cpu
resources. This adds an upper bound on your cpu consumption.

So, fractions make sense and are definitely supported. Let me know if you
need more clarification here.

You may be interested in reading about cpu.shares:
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpu.html

Ben


On Mon, Sep 30, 2013 at 1:20 PM, Sam Taha <ta...@gmail.com> wrote:

> What does it mean to request fractiional CPU resources like 0.5 or 1.5. Is
> this meaningful when making requests and how does it relate to isolation levels.
> For example, you can't really pin 1.5 cores in a cgroup.
>
> Should I really bother letting users/jobs request less than 1 cpu?
>
> Thanks,
> Sam Taha
>
> http://www.grandlogic.com
>
>
>

On Mon, Oct 21, 2013 at 10:17 AM, Vinod Kone <vi...@gmail.com> wrote:

> Yes.
>
> @vinodkone
> Sent from my mobile
>
> > On Oct 21, 2013, at 5:05 AM, Paul Mackles <pa...@loopr.com> wrote:
> >
> > Hi - I just wanted to confirm my understanding of something... with
> process isolation, Mesos will not do anything if a given executor exceeds
> its resource allocation. In other words, if I accept a resource with 1GB of
> memory and then my executor uses 3GB, Mesos won't detect that the process
> exceeded its allocation and kill the process. For that, you need to enable
> cgroups at which point allocation limits are enforced by the OS. Did I get
> that right?
> >
> > --
> > Thanks,
> > Paul
>

Re: process isolation

[Vinod Kone <vi...@gmail.com>]

Yes. 

@vinodkone
Sent from my mobile 

> On Oct 21, 2013, at 5:05 AM, Paul Mackles <pa...@loopr.com> wrote:
> 
> Hi - I just wanted to confirm my understanding of something... with process isolation, Mesos will not do anything if a given executor exceeds its resource allocation. In other words, if I accept a resource with 1GB of memory and then my executor uses 3GB, Mesos won't detect that the process exceeded its allocation and kill the process. For that, you need to enable cgroups at which point allocation limits are enforced by the OS. Did I get that right?
> 
> -- 
> Thanks,
> Paul