You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Benjamin Mahler <be...@gmail.com> on 2015/11/05 17:40:22 UTC
Fwd: [jira] [Created] (MESOS-2353) Improve performance of the
master's state.json endpoint for large clusters.
The benchmark can be written independently and should exist before the fix
so we can measure the improvement. Vinod sounds like you might be
interested in writing one while waiting for michael?
---------- Forwarded message ----------
From: *Vinod Kone (JIRA)* <ji...@apache.org>
Date: Thursday, November 5, 2015
Subject: [jira] [Created] (MESOS-2353) Improve performance of the master's
state.json endpoint for large clusters.
To: benjamin.mahler@gmail.com
[
https://issues.apache.org/jira/browse/MESOS-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991934#comment-14991934
]
Vinod Kone commented on MESOS-2353:
-----------------------------------
Sure. If reviews and design doc are out by monday, that should be fine. Can
you ping me on IRC when you are online? Want to get a sense of what's
coming.
> Improve performance of the master's state.json endpoint for large
clusters.
>
---------------------------------------------------------------------------
>
> Key: MESOS-2353
> URL: https://issues.apache.org/jira/browse/MESOS-2353
> Project: Mesos
> Issue Type: Improvement
> Components: master
> Reporter: Benjamin Mahler
> Labels: newbie, scalability, twitter
>
> The master's state.json endpoint consistently takes a long time to
compute the JSON result, for large clusters:
> {noformat}
> $ time curl -s -o /dev/null localhost:5050/master/state.json
> Mon Jan 26 22:38:50 UTC 2015
> real 0m13.174s
> user 0m0.003s
> sys 0m0.022s
> {noformat}
> This can cause the master to get backlogged if there are many state.json
requests in flight.
> Looking at {{perf}} data, it seems most of the time is spent doing memory
allocation / de-allocation. This ticket will try to capture any low hanging
fruit to speed this up. Possibly we can leverage moves if they are not
already being used by the compiler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [jira] [Created] (MESOS-2353) Improve performance of the master's
state.json endpoint for large clusters.
Posted by Vinod Kone <vi...@gmail.com>.
Definitely! I wasn't sure if Michael already added a benchmark as part of
this reviews/prototyping.
On Thu, Nov 5, 2015 at 8:40 AM, Benjamin Mahler <be...@gmail.com>
wrote:
> The benchmark can be written independently and should exist before the fix
> so we can measure the improvement. Vinod sounds like you might be
> interested in writing one while waiting for michael?
>
>
> ---------- Forwarded message ----------
> From: *Vinod Kone (JIRA)* <ji...@apache.org>
> Date: Thursday, November 5, 2015
> Subject: [jira] [Created] (MESOS-2353) Improve performance of the master's
> state.json endpoint for large clusters.
> To: benjamin.mahler@gmail.com
>
>
>
> [
> https://issues.apache.org/jira/browse/MESOS-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991934#comment-14991934
> ]
>
> Vinod Kone commented on MESOS-2353:
> -----------------------------------
>
> Sure. If reviews and design doc are out by monday, that should be fine.
> Can you ping me on IRC when you are online? Want to get a sense of what's
> coming.
>
> > Improve performance of the master's state.json endpoint for large
> clusters.
> >
> ---------------------------------------------------------------------------
> >
> > Key: MESOS-2353
> > URL: https://issues.apache.org/jira/browse/MESOS-2353
> > Project: Mesos
> > Issue Type: Improvement
> > Components: master
> > Reporter: Benjamin Mahler
> > Labels: newbie, scalability, twitter
> >
> > The master's state.json endpoint consistently takes a long time to
> compute the JSON result, for large clusters:
> > {noformat}
> > $ time curl -s -o /dev/null localhost:5050/master/state.json
> > Mon Jan 26 22:38:50 UTC 2015
> > real 0m13.174s
> > user 0m0.003s
> > sys 0m0.022s
> > {noformat}
> > This can cause the master to get backlogged if there are many state.json
> requests in flight.
> > Looking at {{perf}} data, it seems most of the time is spent doing
> memory allocation / de-allocation. This ticket will try to capture any low
> hanging fruit to speed this up. Possibly we can leverage moves if they are
> not already being used by the compiler.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>
>