You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "Greg Mann (JIRA)" <ji...@apache.org> on 2018/07/03 17:11:00 UTC
[jira] [Commented] (MESOS-8986) `slave.available()` in the
allocator is expensive and drags down allocation performance.
[ https://issues.apache.org/jira/browse/MESOS-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531697#comment-16531697 ]
Greg Mann commented on MESOS-8986:
----------------------------------
Backports:
1.6.x:
{code}
commit 4e064011038d1afcb60e3374aa94dd01ac88f6b9
Author: Meng Zhu <mz...@mesosphere.io>
Date: Thu Jun 21 09:09:39 2018 -0700
Modified `createStrippedScalarQuantity()` to clear all metadata fields.
Currently `createStrippedScalarQuantity()` strips resource meta-data
and transforms dynamic reservations into a static reservation.
However, no current code depends on the reservations in resources
returned by this helper function. This leads to boilerplate code
around call sites and performance overhead.
This patch updates the function to clear all reservation information.
Review: https://reviews.apache.org/r/67615/
commit 0d50c45bf19061e4c978641db1f7f9f99088dbae
Author: Meng Zhu <mz...@mesosphere.io>
Date: Thu Jun 21 09:09:36 2018 -0700
Refactored `struct Slave` in the allocator for better performance.
This patch refactors the `struct Slave` in the allocator.
In particular, it addresses the slowness of computing
agents' available resources. Instead of calculating them
every time on the fly, this patch "denormalizes" the agent
available resources by updating and persisting the field
each time an agent's allocated or total resources change.
Review: https://reviews.apache.org/r/67561/
{code}
1.5.x:
{code}
commit e14a05fc0135697a41fd4c5ec4237ac195240736
Author: Meng Zhu <mz...@mesosphere.io>
Date: Thu Jun 21 09:09:39 2018 -0700
Modified `createStrippedScalarQuantity()` to clear all metadata fields.
Currently `createStrippedScalarQuantity()` strips resource meta-data
and transforms dynamic reservations into a static reservation.
However, no current code depends on the reservations in resources
returned by this helper function. This leads to boilerplate code
around call sites and performance overhead.
This patch updates the function to clear all reservation information.
Review: https://reviews.apache.org/r/67615/
commit 762c78d5351e2cbef4dc5deb58389ee48e56ef4f
Author: Meng Zhu <mz...@mesosphere.io>
Date: Thu Jun 21 09:09:36 2018 -0700
Refactored `struct Slave` in the allocator for better performance.
This patch refactors the `struct Slave` in the allocator.
In particular, it addresses the slowness of computing
agents' available resources. Instead of calculating them
every time on the fly, this patch "denormalizes" the agent
available resources by updating and persisting the field
each time an agent's allocated or total resources change.
Review: https://reviews.apache.org/r/67561/
{code}
1.4.x:
{code}
commit 8814f143b095addc3dffcc29dab275ba0b8f9d5d
Author: Meng Zhu <mz...@mesosphere.io>
Date: Thu Jun 21 09:09:39 2018 -0700
Modified `createStrippedScalarQuantity()` to clear all metadata fields.
Currently `createStrippedScalarQuantity()` strips resource meta-data
and transforms dynamic reservations into a static reservation.
However, no current code depends on the reservations in resources
returned by this helper function. This leads to boilerplate code
around call sites and performance overhead.
This patch updates the function to clear all reservation information.
Review: https://reviews.apache.org/r/67615/
commit d5827547b3a83b79f4373199158bc40c0ae379d9
Author: Meng Zhu <mz...@mesosphere.io>
Date: Thu Jun 21 09:09:36 2018 -0700
Refactored `struct Slave` in the allocator for better performance.
This patch refactors the `struct Slave` in the allocator.
In particular, it addresses the slowness of computing
agents' available resources. Instead of calculating them
every time on the fly, this patch "denormalizes" the agent
available resources by updating and persisting the field
each time an agent's allocated or total resources change.
Review: https://reviews.apache.org/r/67561/
{code}
> `slave.available()` in the allocator is expensive and drags down allocation performance.
> ----------------------------------------------------------------------------------------
>
> Key: MESOS-8986
> URL: https://issues.apache.org/jira/browse/MESOS-8986
> Project: Mesos
> Issue Type: Improvement
> Components: allocation
> Reporter: Meng Zhu
> Assignee: Meng Zhu
> Priority: Major
> Fix For: 1.4.2, 1.5.2, 1.7.0, 1.6.1
>
>
> We noticed that the [`slave.available()` function|https://github.com/apache/mesos/blob/d733b1031350e03bce443aa287044eb4eee1053a/src/master/allocator/mesos/hierarchical.hpp#L380-L388] in the allocator is expensive and gets called many times in each allocation cycle. In one of our profiling results, this function accounts for more than 80% of the allocation time, drags down the allocator performance significantly.
> One simple way to reduce the overhead is to "denormalize" the available resource field--instead of calculating it each time on the fly, we could cache the value and only update it when the agent's allocations or total resources change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)