Controlling Resources Allocated to a Given Task

[Sammy Steele <sa...@stanford.edu>]

Hi,

I am trying to determine the impact of various CPU and Memory allocations
on the performance of a mesos task. I don't think I am accurately
controlling how a framework limits the resources used by executors.

I am using the test_framework.py in the python example and modifying the
associated test_executor to run my task. To change the resources allocated,
I have been changing the parameters of TASK_CPUS and TASK_MEM in
test_framework.py. When a task is actively running, the Mem and CPU
recorded on the Mesos framework website corresponds exactly to the values
specified by those parameters. However,  I am running a program with 4
threads and strangely I see no change in performance when running on 1 CPU
with 32 MB vs. 8 CPU with 512 MB. This has lead me to wonder if changing
those parameters does not actually limit the amount of CPU/ Memory
available to a given task. Is there another way I should go about
controlling resources? Thanks!

Sammy

Re: Controlling Resources Allocated to a Given Task

[Vinod Kone <vi...@gmail.com>]

How are you launching the slaves? By default the slave doesn't do any
resource isolation.

You should enable cgroups (only available on linux) for this to work.

./bin/mesos-slave.sh --isolation='cgroups/cpu,cgroups/mem'

Note that 'cpu' isolation by default is a lower bound. To set it as an
upper bound add "--cgroups_enable_cfs" to the slave flags.


For more options: './bin/mesos-slave.sh --help'

HTH,


On Mon, Jul 14, 2014 at 4:49 PM, Sammy Steele <sa...@stanford.edu>
wrote:

> Hi,
>
> I am trying to determine the impact of various CPU and Memory allocations
> on the performance of a mesos task. I don't think I am accurately
> controlling how a framework limits the resources used by executors.
>
> I am using the test_framework.py in the python example and modifying the
> associated test_executor to run my task. To change the resources allocated,
> I have been changing the parameters of TASK_CPUS and TASK_MEM in
> test_framework.py. When a task is actively running, the Mem and CPU
> recorded on the Mesos framework website corresponds exactly to the values
> specified by those parameters. However,  I am running a program with 4
> threads and strangely I see no change in performance when running on 1 CPU
> with 32 MB vs. 8 CPU with 512 MB. This has lead me to wonder if changing
> those parameters does not actually limit the amount of CPU/ Memory
> available to a given task. Is there another way I should go about
> controlling resources? Thanks!
>
> Sammy
>