You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "Neil Conway (JIRA)" <ji...@apache.org> on 2017/05/17 23:26:04 UTC
[jira] [Commented] (MESOS-7521) Major performance regression in drf
sorter
[ https://issues.apache.org/jira/browse/MESOS-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16014932#comment-16014932 ]
Neil Conway commented on MESOS-7521:
------------------------------------
Quick proof of concept patch here: https://reviews.apache.org/r/59355/
We could do more work to improve perf for the many-suppressed-frameworks case (e.g., keep a separate vector of inactive children for each node, which would avoid needing to sort inactive clients entirely), but that might be more disruptive.
Waiting on perf results from [~drexin] to see if it addresses their use-case or if more perf work is needed.
> Major performance regression in drf sorter
> ------------------------------------------
>
> Key: MESOS-7521
> URL: https://issues.apache.org/jira/browse/MESOS-7521
> Project: Mesos
> Issue Type: Bug
> Components: allocation
> Affects Versions: 1.3.0
> Reporter: Dario Rexin
> Assignee: Neil Conway
> Priority: Blocker
> Labels: perfomance
>
> The addition of hierarchical roles to the framework sorter (https://github.com/apache/mesos/commit/e5ef1992b2b8e84b5d1487f1578f18f2291cd082) has introduced a major performance regression to 1.2. Suppressing offers for frameworks does not seem to reduce allocation time anymore, like it used to in 1.2. Here are some relevant benchmark results:
> Mesos 1.2:
> {noformat}
> [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7
> Using 1000 agents and 6000 frameworks
> Added 6000 frameworks in 105957us
> Added 1000 agents in 34.937438secs
> allocate() took 27.408828secs to make 1000 offers with 1200 out of 6000 frameworks suppressing offers
> allocate() took 20.121897secs to make 1000 offers with 2400 out of 6000 frameworks suppressing offers
> allocate() took 12.964302secs to make 1000 offers with 3600 out of 6000 frameworks suppressing offers
> allocate() took 6.534221secs to make 1000 offers with 4800 out of 6000 frameworks suppressing offers
> allocate() took 8953us to make 0 offers with 6000 out of 6000 frameworks suppressing offers
> [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7 (106198 ms)
> {noformat}
> Mesos 1.3:
> {noformat}
> [ RUN ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7
> Using 1000 agents and 6000 frameworks
> Added 6000 frameworks in 1.036217secs
> Added 1000 agents in 10.093938secs
> allocate() took 10.629448secs to make 1000 offers with 1200 out of 6000 frameworks suppressing offers
> allocate() took 11.607185secs to make 1000 offers with 2400 out of 6000 frameworks suppressing offers
> allocate() took 12.896578secs to make 1000 offers with 3600 out of 6000 frameworks suppressing offers
> allocate() took 14.162431secs to make 1000 offers with 4800 out of 6000 frameworks suppressing offers
> allocate() took 257060us to make 0 offers with 6000 out of 6000 frameworks suppressing offers
> [ OK ] SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/7 (64011 ms)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)