You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "Greg Mann (JIRA)" <ji...@apache.org> on 2018/07/02 19:56:00 UTC
[jira] [Commented] (MESOS-8935) Quota limit "chopping" can lead to
cpu-only and memory-only offers.
[ https://issues.apache.org/jira/browse/MESOS-8935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530369#comment-16530369 ]
Greg Mann commented on MESOS-8935:
----------------------------------
Backports:
1.6.x:
{code}
commit 0587245b66ad3f2209c66a67211d987d2abdd371
Author: Meng Zhu <mz...@mesosphere.io>
Date: Wed Jun 20 17:00:03 2018 -0700
Added a master flag to configure minimum allocatable resources.
This patch adds a new master flag `min_allocatable_resources`.
It specifies one or more resource quantities that define the
minimum allocatable resources for the allocator. The allocator
will only offer resources that contain at least one of the
specified resource quantities.
For example, the setting `disk:1000|cpus:1;mem:32` means that
the allocator will only allocate resources when they contain
1000MB of disk, or when they contain both 1 cpu and 32MB of
memory.
The default value for this new flag is such that it maintains
previous default behavior.
Also fixed all related tests and updated documentation.
Review: https://reviews.apache.org/r/67513/
commit ccb24bf3f7e098723179ee4b595aa99e3a0869e4
Author: Meng Zhu <mz...@mesosphere.io>
Date: Thu Jun 28 08:33:00 2018 -0700
Added a resource utility `isScalarQuantity`.
`isScalarQuantity()` checks if a `Resources` object
is "pure" scalar quantity i.e. its `Resource`(s) only has
name, type (set to scalar) and scalar fields set.
Also added tests.
Review: https://reviews.apache.org/r/67516/
commit a615f36d9f10c92eaa4be95978987976dfc085e8
Author: Meng Zhu <mz...@mesosphere.io>
Date: Thu Jun 28 08:32:47 2018 -0700
Fixed a bug in `createStrippedScalarQuantity()`.
This patch fixes `createStrippedScalarQuantity()` by
stripping the revocable field in resources.
Also added a test.
Review: https://reviews.apache.org/r/67510/
{code}
1.5.x:
{code}
commit 2e16cdb16ee9cc4162fad8b3957d69b9af7dbd8b
Author: Meng Zhu <mz...@mesosphere.io>
Date: Wed Jun 20 17:00:03 2018 -0700
Added a master flag to configure minimum allocatable resources.
This patch adds a new master flag `min_allocatable_resources`.
It specifies one or more resource quantities that define the
minimum allocatable resources for the allocator. The allocator
will only offer resources that contain at least one of the
specified resource quantities.
For example, the setting `disk:1000|cpus:1;mem:32` means that
the allocator will only allocate resources when they contain
1000MB of disk, or when they contain both 1 cpu and 32MB of
memory.
The default value for this new flag is such that it maintains
previous default behavior.
Also fixed all related tests and updated documentation.
Review: https://reviews.apache.org/r/67513/
commit be077099b4dfcc1f82fe7f5ed222567eeb0c082c
Author: Meng Zhu <mz...@mesosphere.io>
Date: Wed Jun 20 16:59:58 2018 -0700
Added a resource utility `isScalarQuantity`.
`isScalarQuantity()` checks if a `Resources` object
is a "pure" scalar quantity; i.e., its resources only have
name, type (set to scalar) and scalar fields set.
Also added tests.
Review: https://reviews.apache.org/r/67516/
commit 7a19d085c8aead7693c5d6212dbba7db771e60f6
Author: Meng Zhu <mz...@mesosphere.io>
Date: Wed Jun 20 16:59:54 2018 -0700
Fixed a bug in `createStrippedScalarQuantity()`.
This patch fixes `createStrippedScalarQuantity()` by
stripping the revocable field in resources.
Also added a test.
Review: https://reviews.apache.org/r/67510/
commit c9efa4048be540f6cee47c012a7637ffed5e203e
Author: Benjamin Mahler <bm...@apache.org>
Date: Mon Feb 5 13:32:37 2018 -0800
Introduced a CHECK_NOTERROR macro.
Review: https://reviews.apache.org/r/65514
{code}
1.4.x:
{code}
commit 9cba3aa6dd571f8b92b46261d2e1256b0c47e338 (1.4.x-allocatable)
Author: Meng Zhu <mz...@mesosphere.io>
Date: Wed Jun 20 17:00:03 2018 -0700
Added a master flag to configure minimum allocatable resources.
This patch adds a new master flag `min_allocatable_resources`.
It specifies one or more resource quantities that define the
minimum allocatable resources for the allocator. The allocator
will only offer resources that contain at least one of the
specified resource quantities.
For example, the setting `disk:1000|cpus:1;mem:32` means that
the allocator will only allocate resources when they contain
1000MB of disk, or when they contain both 1 cpu and 32MB of
memory.
The default value for this new flag is such that it maintains
previous default behavior.
Also fixed all related tests and updated documentation.
Review: https://reviews.apache.org/r/67513/
commit c45b4bdd50ee2fc5db7e3c2274ef2938a8999c22
Author: Meng Zhu <mz...@mesosphere.io>
Date: Wed Jun 20 16:59:58 2018 -0700
Added a resource utility `isScalarQuantity`.
`isScalarQuantity()` checks if a `Resources` object
is a "pure" scalar quantity; i.e., its resources only have
name, type (set to scalar) and scalar fields set.
Also added tests.
Review: https://reviews.apache.org/r/67516/
commit 450651bd115292edf4d42cab3367ba58017b4354
Author: Meng Zhu <mz...@mesosphere.io>
Date: Wed Jun 20 16:59:54 2018 -0700
Fixed a bug in `createStrippedScalarQuantity()`.
This patch fixes `createStrippedScalarQuantity()` by
stripping the revocable field in resources.
Also added a test.
Review: https://reviews.apache.org/r/67510/
commit 8911db363509c50fbeaf2813cd06f16cf829541d
Author: Benjamin Mahler <bm...@apache.org>
Date: Mon Feb 5 13:32:37 2018 -0800
Introduced a CHECK_NOTERROR macro.
Review: https://reviews.apache.org/r/65514
{code}
> Quota limit "chopping" can lead to cpu-only and memory-only offers.
> -------------------------------------------------------------------
>
> Key: MESOS-8935
> URL: https://issues.apache.org/jira/browse/MESOS-8935
> Project: Mesos
> Issue Type: Bug
> Components: allocation
> Affects Versions: 1.4.2, 1.5.0, 1.5.1, 1.6.0
> Reporter: Benjamin Mahler
> Assignee: Meng Zhu
> Priority: Major
> Fix For: 1.4.2, 1.5.2, 1.7.0, 1.6.1
>
>
> When we allocate resources to a role, we'll "chop" the available resources of the agent up to the quota limit for the role (per MESOS-7099). This prevents the role from exceeding its quota limit.
> This has the unintended consequence of creating cpu-only and memory-only offers.
> Consider agents with 10 cpus and 100 GB mem and roles with quota guarantee/limit of 5 cpus, 10 GB mem. The following allocations will occur:
> agent 1:
> r1 -> 5 cpus 10GB mem
> r2 -> 5 cpus 10GB mem
> r3 -> 0 cpus 10GB mem (quota allocates even if it can make progress towards a single resource and MESOS-1688 allows this)
> r4 -> 0 cpus 10GB mem
> ...
> r10 -> 0 cpus 10GB mem
> agent 2:
> r3 -> 5 cpus 0GB mem (r3 is already at its 10GB mem limit)
> r4 -> 5 cpus 0GB mem
> r11 -> 0 cpus 10GB mem
> ...
> r20 -> 0 cpus 10GB mem
> Here, roles 3-20 receive memory only and cpu only offers. This gets further exacerbated if DRF chooses the same ordering between roles across cycles.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)