You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@mesos.apache.org by Varun Gupta <va...@uber.com> on 2018/08/06 21:28:14 UTC

Understand fixed resource estimator to get oversubscribe resources

Hi,

I was reading the code
<https://github.com/apache/mesos/blob/master/src/slave/resource_estimators/fixed.cpp#L57>
for fixed resource estimator and want to understand the behavior for it.

For example, CPU is the only revocable resource for the Mesos Cluster. I
have one machine with 24 CPUs as non-revocable, and correspondingly 24
CPU's are revocable resources (hardcoded for fixed resource estimator).

Below are two scenario's, I want to confirm what is the actual behavior or
it is completely different.

1) Static behavior, were on 24 CPU revocable and 24 CPU non-revocable, 12
CPU non-revocable task is launched with 100% utilization. In this scenario,
resources offered 12 CPU non-revocable + 24 CPU revocable resources.

2) Same as above scenario, and reading the document
<https://docs.google.com/document/d/1r1WCHgmPJp5wbrqSZLsgtxPNj3sULfHrSFmxp2GyPTo/edit#>
for Revocable resources for Aurora. *"A revocable CPU resource value will
include unallocated + unused resource"*. For above scenario, fixed resource
estimator should return 12 CPU non-revocable and 12 CPU revocable (as 12
CPU non-revocable are running on 100% utilization).
Following up this scenario, suppose 12 CPU non-revocable task is launched
and of that 8 CPU are effectively used. Thereby, offer generated will be 12
CPU non-revocable and 12 (available) + 4 (usage slack) CPU revocable
resources.

Thanks,
Varun

Re: Understand fixed resource estimator to get oversubscribe resources

Posted by Varun Gupta <va...@uber.com>.

Thanks Benjamin!
On Fri, Aug 10, 2018 at 1:01 PM Benjamin Mahler <bm...@apache.org> wrote:

> The fixed resource estimator provides a fixed size revocable pool: if you
> tell it to create a 24 cpu revocable pool, there will be a 24 cpu revocable
> pool. It is not looking at utilization slack.
>
> On Mon, Aug 6, 2018 at 2:28 PM, Varun Gupta <va...@uber.com> wrote:
>
>> Hi,
>>
>> I was reading the code
>> <https://github.com/apache/mesos/blob/master/src/slave/resource_estimators/fixed.cpp#L57>
>> for fixed resource estimator and want to understand the behavior for it.
>>
>> For example, CPU is the only revocable resource for the Mesos Cluster. I
>> have one machine with 24 CPUs as non-revocable, and correspondingly 24
>> CPU's are revocable resources (hardcoded for fixed resource estimator).
>>
>> Below are two scenario's, I want to confirm what is the actual behavior
>> or it is completely different.
>>
>> 1) Static behavior, were on 24 CPU revocable and 24 CPU non-revocable, 12
>> CPU non-revocable task is launched with 100% utilization. In this scenario,
>> resources offered 12 CPU non-revocable + 24 CPU revocable resources.
>>
>> 2) Same as above scenario, and reading the document
>> <https://docs.google.com/document/d/1r1WCHgmPJp5wbrqSZLsgtxPNj3sULfHrSFmxp2GyPTo/edit#>
>> for Revocable resources for Aurora. *"A revocable CPU resource value
>> will include unallocated + unused resource"*. For above scenario, fixed
>> resource estimator should return 12 CPU non-revocable and 12 CPU revocable
>> (as 12 CPU non-revocable are running on 100% utilization).
>> Following up this scenario, suppose 12 CPU non-revocable task is launched
>> and of that 8 CPU are effectively used. Thereby, offer generated will be 12
>> CPU non-revocable and 12 (available) + 4 (usage slack) CPU revocable
>> resources.
>>
>> Thanks,
>> Varun
>>
>>
>

Re: Understand fixed resource estimator to get oversubscribe resources

Posted by Benjamin Mahler <bm...@apache.org>.

The fixed resource estimator provides a fixed size revocable pool: if you
tell it to create a 24 cpu revocable pool, there will be a 24 cpu revocable
pool. It is not looking at utilization slack.

On Mon, Aug 6, 2018 at 2:28 PM, Varun Gupta <va...@uber.com> wrote:

> Hi,
>
> I was reading the code
> <https://github.com/apache/mesos/blob/master/src/slave/resource_estimators/fixed.cpp#L57>
> for fixed resource estimator and want to understand the behavior for it.
>
> For example, CPU is the only revocable resource for the Mesos Cluster. I
> have one machine with 24 CPUs as non-revocable, and correspondingly 24
> CPU's are revocable resources (hardcoded for fixed resource estimator).
>
> Below are two scenario's, I want to confirm what is the actual behavior or
> it is completely different.
>
> 1) Static behavior, were on 24 CPU revocable and 24 CPU non-revocable, 12
> CPU non-revocable task is launched with 100% utilization. In this scenario,
> resources offered 12 CPU non-revocable + 24 CPU revocable resources.
>
> 2) Same as above scenario, and reading the document
> <https://docs.google.com/document/d/1r1WCHgmPJp5wbrqSZLsgtxPNj3sULfHrSFmxp2GyPTo/edit#>
> for Revocable resources for Aurora. *"A revocable CPU resource value will
> include unallocated + unused resource"*. For above scenario, fixed
> resource estimator should return 12 CPU non-revocable and 12 CPU revocable
> (as 12 CPU non-revocable are running on 100% utilization).
> Following up this scenario, suppose 12 CPU non-revocable task is launched
> and of that 8 CPU are effectively used. Thereby, offer generated will be 12
> CPU non-revocable and 12 (available) + 4 (usage slack) CPU revocable
> resources.
>
> Thanks,
> Varun
>
>