You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by "Clarke, Trevor" <tc...@ball.com> on 2015/09/23 15:33:38 UTC
how does resource allocation work with docker?
When a framework accepts an offer and starts a docker task, how does mesos enforce the task's allocated cpus? Is CPU share used and scaled appropriately? Are cpu sets explicitly specified to limit execution to the allocated cpus? Is there a way for a process in the docker container to determine how many cpus it has been allocated? (so it can adjust fan-out size, etc. for optimal execution)
This message and any enclosures are intended only for the addressee. Please
notify the sender by email if you are not the intended recipient. If you are
not the intended recipient, you may not use, copy, disclose, or distribute this
message or its contents or enclosures to any other person and any such actions
may be unlawful. Ball reserves the right to monitor and review all messages
and enclosures sent to or from this email address.
Re: how does resource allocation work with docker?
Posted by Guangya Liu <gy...@gmail.com>.
Hi Clarke,
Yes, you are right, the mesos is now using 1024 * CPUS as cpushare, you can
refer to
https://github.com/apache/mesos/blob/master/src/docker/docker.cpp#L394 for
detail.
Regarding to cpus, I think that by default the container have same number
of cores/cpus as the docker server, not sure if you can adjust your task
based on the cpu/core number on the docker server.
Thanks,
Guangya
On Wed, Sep 23, 2015 at 11:02 PM, Clarke, Trevor <tc...@ball.com> wrote:
> I think you misunderstand. I'm not looking to change the container
> resource allocation, I'm looking to have the program in the container make
> adjustments (changing static buffer allocations based on memory limits or
> adjusting the number of threads based on cpu allocation). But I need a way
> to communicate the resource limits to the app in the container.
>
>
> ------------------------------
> *From:* haosdent [haosdent@gmail.com]
> *Sent:* Wednesday, September 23, 2015 10:56 AM
> *To:* user@mesos.apache.org
> *Subject:* RE: how does resource allocation work with docker?
>
> >Is there a way for a process in the docker container to determine how
> many cpus it has been allocated?
> Because docker limit the resource before start the container, I think
> there isn't a way to change container resource usage after it has already
> run. You only could allocate resource before launch a container so far.
> On Sep 23, 2015 10:31 PM, "Clarke, Trevor" <tc...@ball.com> wrote:
>
>> That's just the scheduler...I did locate the appropriate docker run call
>> and it appears that mesos cpu offer * 1024 is used as the cpu share so
>> there's no hard enforcement of total cpus but the scheduler block is scaled
>> to the number of requested cpus. It also appears there's no way to
>> determine the cpu share from within docker without explicitly passing an
>> environment variable, etc.
>> ------------------------------
>> *From:* Guangya Liu [gyliu513@gmail.com]
>> *Sent:* Wednesday, September 23, 2015 9:41 AM
>> *To:* user@mesos.apache.org
>> *Subject:* Re: how does resource allocation work with docker?
>>
>> Hi Clarke,
>>
>> You can take a look at the framework for docker here
>> https://github.com/apache/mesos/blob/master/src/examples/docker_no_executor_framework.cpp
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_mesos_blob_master_src_examples_docker-5Fno-5Fexecutor-5Fframework.cpp&d=AwMFaQ&c=jF7FvYH6t0RX1HrEjVCgHQ&r=BZAdW7eZ7BA-TVm8CsncxQ&m=myFl_koVz_iyPrvAE5EQBW4Cmn4ocS6C5KCd07WzvjA&s=PWUrWtXV7HgaC4GxJ0HstRpy_b9dqvCptTljtTzQrx0&e=>
>>
>> All of the resource limitations for a docker task should be defined in
>> the framework.
>>
>> Thanks,
>>
>> Guangya
>>
>> On Wed, Sep 23, 2015 at 9:33 PM, Clarke, Trevor <tc...@ball.com> wrote:
>>
>>> When a framework accepts an offer and starts a docker task, how does
>>> mesos enforce the task's allocated cpus? Is CPU share used and scaled
>>> appropriately? Are cpu sets explicitly specified to limit execution to the
>>> allocated cpus? Is there a way for a process in the docker container to
>>> determine how many cpus it has been allocated? (so it can adjust fan-out
>>> size, etc. for optimal execution)
>>>
>>>
>>> This message and any enclosures are intended only for the addressee.
>>> Please
>>> notify the sender by email if you are not the intended recipient. If you
>>> are
>>> not the intended recipient, you may not use, copy, disclose, or
>>> distribute this
>>> message or its contents or enclosures to any other person and any such
>>> actions
>>> may be unlawful. Ball reserves the right to monitor and review all
>>> messages
>>> and enclosures sent to or from this email address.
>>>
>>
>>
>>
>> This message and any enclosures are intended only for the addressee.
>> Please
>> notify the sender by email if you are not the intended recipient. If you
>> are
>> not the intended recipient, you may not use, copy, disclose, or
>> distribute this
>> message or its contents or enclosures to any other person and any such
>> actions
>> may be unlawful. Ball reserves the right to monitor and review all
>> messages
>> and enclosures sent to or from this email address.
>>
>
>
> This message and any enclosures are intended only for the addressee.
> Please
> notify the sender by email if you are not the intended recipient. If you
> are
> not the intended recipient, you may not use, copy, disclose, or distribute
> this
> message or its contents or enclosures to any other person and any such
> actions
> may be unlawful. Ball reserves the right to monitor and review all
> messages
> and enclosures sent to or from this email address.
>
RE: how does resource allocation work with docker?
Posted by "Clarke, Trevor" <tc...@ball.com>.
I think you misunderstand. I'm not looking to change the container resource allocation, I'm looking to have the program in the container make adjustments (changing static buffer allocations based on memory limits or adjusting the number of threads based on cpu allocation). But I need a way to communicate the resource limits to the app in the container.
________________________________
From: haosdent [haosdent@gmail.com]
Sent: Wednesday, September 23, 2015 10:56 AM
To: user@mesos.apache.org
Subject: RE: how does resource allocation work with docker?
>Is there a way for a process in the docker container to determine how many cpus it has been allocated?
Because docker limit the resource before start the container, I think there isn't a way to change container resource usage after it has already run. You only could allocate resource before launch a container so far.
On Sep 23, 2015 10:31 PM, "Clarke, Trevor" <tc...@ball.com>> wrote:
That's just the scheduler...I did locate the appropriate docker run call and it appears that mesos cpu offer * 1024 is used as the cpu share so there's no hard enforcement of total cpus but the scheduler block is scaled to the number of requested cpus. It also appears there's no way to determine the cpu share from within docker without explicitly passing an environment variable, etc.
________________________________
From: Guangya Liu [gyliu513@gmail.com<ma...@gmail.com>]
Sent: Wednesday, September 23, 2015 9:41 AM
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: Re: how does resource allocation work with docker?
Hi Clarke,
You can take a look at the framework for docker here https://github.com/apache/mesos/blob/master/src/examples/docker_no_executor_framework.cpp<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_mesos_blob_master_src_examples_docker-5Fno-5Fexecutor-5Fframework.cpp&d=AwMFaQ&c=jF7FvYH6t0RX1HrEjVCgHQ&r=BZAdW7eZ7BA-TVm8CsncxQ&m=myFl_koVz_iyPrvAE5EQBW4Cmn4ocS6C5KCd07WzvjA&s=PWUrWtXV7HgaC4GxJ0HstRpy_b9dqvCptTljtTzQrx0&e=>
All of the resource limitations for a docker task should be defined in the framework.
Thanks,
Guangya
On Wed, Sep 23, 2015 at 9:33 PM, Clarke, Trevor <tc...@ball.com>> wrote:
When a framework accepts an offer and starts a docker task, how does mesos enforce the task's allocated cpus? Is CPU share used and scaled appropriately? Are cpu sets explicitly specified to limit execution to the allocated cpus? Is there a way for a process in the docker container to determine how many cpus it has been allocated? (so it can adjust fan-out size, etc. for optimal execution)
This message and any enclosures are intended only for the addressee. Please
notify the sender by email if you are not the intended recipient. If you are
not the intended recipient, you may not use, copy, disclose, or distribute this
message or its contents or enclosures to any other person and any such actions
may be unlawful. Ball reserves the right to monitor and review all messages
and enclosures sent to or from this email address.
This message and any enclosures are intended only for the addressee. Please
notify the sender by email if you are not the intended recipient. If you are
not the intended recipient, you may not use, copy, disclose, or distribute this
message or its contents or enclosures to any other person and any such actions
may be unlawful. Ball reserves the right to monitor and review all messages
and enclosures sent to or from this email address.
This message and any enclosures are intended only for the addressee. Please
notify the sender by email if you are not the intended recipient. If you are
not the intended recipient, you may not use, copy, disclose, or distribute this
message or its contents or enclosures to any other person and any such actions
may be unlawful. Ball reserves the right to monitor and review all messages
and enclosures sent to or from this email address.
RE: how does resource allocation work with docker?
Posted by haosdent <ha...@gmail.com>.
>Is there a way for a process in the docker container to determine how
many cpus it has been allocated?
Because docker limit the resource before start the container, I think there
isn't a way to change container resource usage after it has already run.
You only could allocate resource before launch a container so far.
On Sep 23, 2015 10:31 PM, "Clarke, Trevor" <tc...@ball.com> wrote:
> That's just the scheduler...I did locate the appropriate docker run call
> and it appears that mesos cpu offer * 1024 is used as the cpu share so
> there's no hard enforcement of total cpus but the scheduler block is scaled
> to the number of requested cpus. It also appears there's no way to
> determine the cpu share from within docker without explicitly passing an
> environment variable, etc.
> ------------------------------
> *From:* Guangya Liu [gyliu513@gmail.com]
> *Sent:* Wednesday, September 23, 2015 9:41 AM
> *To:* user@mesos.apache.org
> *Subject:* Re: how does resource allocation work with docker?
>
> Hi Clarke,
>
> You can take a look at the framework for docker here
> https://github.com/apache/mesos/blob/master/src/examples/docker_no_executor_framework.cpp
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_mesos_blob_master_src_examples_docker-5Fno-5Fexecutor-5Fframework.cpp&d=AwMFaQ&c=jF7FvYH6t0RX1HrEjVCgHQ&r=BZAdW7eZ7BA-TVm8CsncxQ&m=myFl_koVz_iyPrvAE5EQBW4Cmn4ocS6C5KCd07WzvjA&s=PWUrWtXV7HgaC4GxJ0HstRpy_b9dqvCptTljtTzQrx0&e=>
>
> All of the resource limitations for a docker task should be defined in the
> framework.
>
> Thanks,
>
> Guangya
>
> On Wed, Sep 23, 2015 at 9:33 PM, Clarke, Trevor <tc...@ball.com> wrote:
>
>> When a framework accepts an offer and starts a docker task, how does
>> mesos enforce the task's allocated cpus? Is CPU share used and scaled
>> appropriately? Are cpu sets explicitly specified to limit execution to the
>> allocated cpus? Is there a way for a process in the docker container to
>> determine how many cpus it has been allocated? (so it can adjust fan-out
>> size, etc. for optimal execution)
>>
>>
>> This message and any enclosures are intended only for the addressee.
>> Please
>> notify the sender by email if you are not the intended recipient. If you
>> are
>> not the intended recipient, you may not use, copy, disclose, or
>> distribute this
>> message or its contents or enclosures to any other person and any such
>> actions
>> may be unlawful. Ball reserves the right to monitor and review all
>> messages
>> and enclosures sent to or from this email address.
>>
>
>
>
> This message and any enclosures are intended only for the addressee.
> Please
> notify the sender by email if you are not the intended recipient. If you
> are
> not the intended recipient, you may not use, copy, disclose, or distribute
> this
> message or its contents or enclosures to any other person and any such
> actions
> may be unlawful. Ball reserves the right to monitor and review all
> messages
> and enclosures sent to or from this email address.
>
RE: how does resource allocation work with docker?
Posted by "Clarke, Trevor" <tc...@ball.com>.
That's just the scheduler...I did locate the appropriate docker run call and it appears that mesos cpu offer * 1024 is used as the cpu share so there's no hard enforcement of total cpus but the scheduler block is scaled to the number of requested cpus. It also appears there's no way to determine the cpu share from within docker without explicitly passing an environment variable, etc.
________________________________
From: Guangya Liu [gyliu513@gmail.com]
Sent: Wednesday, September 23, 2015 9:41 AM
To: user@mesos.apache.org
Subject: Re: how does resource allocation work with docker?
Hi Clarke,
You can take a look at the framework for docker here https://github.com/apache/mesos/blob/master/src/examples/docker_no_executor_framework.cpp<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_mesos_blob_master_src_examples_docker-5Fno-5Fexecutor-5Fframework.cpp&d=AwMFaQ&c=jF7FvYH6t0RX1HrEjVCgHQ&r=BZAdW7eZ7BA-TVm8CsncxQ&m=myFl_koVz_iyPrvAE5EQBW4Cmn4ocS6C5KCd07WzvjA&s=PWUrWtXV7HgaC4GxJ0HstRpy_b9dqvCptTljtTzQrx0&e=>
All of the resource limitations for a docker task should be defined in the framework.
Thanks,
Guangya
On Wed, Sep 23, 2015 at 9:33 PM, Clarke, Trevor <tc...@ball.com>> wrote:
When a framework accepts an offer and starts a docker task, how does mesos enforce the task's allocated cpus? Is CPU share used and scaled appropriately? Are cpu sets explicitly specified to limit execution to the allocated cpus? Is there a way for a process in the docker container to determine how many cpus it has been allocated? (so it can adjust fan-out size, etc. for optimal execution)
This message and any enclosures are intended only for the addressee. Please
notify the sender by email if you are not the intended recipient. If you are
not the intended recipient, you may not use, copy, disclose, or distribute this
message or its contents or enclosures to any other person and any such actions
may be unlawful. Ball reserves the right to monitor and review all messages
and enclosures sent to or from this email address.
This message and any enclosures are intended only for the addressee. Please
notify the sender by email if you are not the intended recipient. If you are
not the intended recipient, you may not use, copy, disclose, or distribute this
message or its contents or enclosures to any other person and any such actions
may be unlawful. Ball reserves the right to monitor and review all messages
and enclosures sent to or from this email address.
Re: how does resource allocation work with docker?
Posted by Guangya Liu <gy...@gmail.com>.
Hi Clarke,
You can take a look at the framework for docker here
https://github.com/apache/mesos/blob/master/src/examples/docker_no_executor_framework.cpp
All of the resource limitations for a docker task should be defined in the
framework.
Thanks,
Guangya
On Wed, Sep 23, 2015 at 9:33 PM, Clarke, Trevor <tc...@ball.com> wrote:
> When a framework accepts an offer and starts a docker task, how does mesos
> enforce the task's allocated cpus? Is CPU share used and scaled
> appropriately? Are cpu sets explicitly specified to limit execution to the
> allocated cpus? Is there a way for a process in the docker container to
> determine how many cpus it has been allocated? (so it can adjust fan-out
> size, etc. for optimal execution)
>
>
> This message and any enclosures are intended only for the addressee.
> Please
> notify the sender by email if you are not the intended recipient. If you
> are
> not the intended recipient, you may not use, copy, disclose, or distribute
> this
> message or its contents or enclosures to any other person and any such
> actions
> may be unlawful. Ball reserves the right to monitor and review all
> messages
> and enclosures sent to or from this email address.
>