[jira] [Commented] (MESOS-2353) Improve performance of the master's state.json endpoint for large clusters.

["Benjamin Mahler (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532056#comment-14532056 ] 

Benjamin Mahler commented on MESOS-2353:
----------------------------------------

How about "caching" the JSON of 'Framework', 'Slave', etc inside the respective structs in the Master, updating it whenever there are changes. We have a similar approach with resource totals, given they are expensive to compute.

> Improve performance of the master's state.json endpoint for large clusters.
> ---------------------------------------------------------------------------
>
>                 Key: MESOS-2353
>                 URL: https://issues.apache.org/jira/browse/MESOS-2353
>             Project: Mesos
>          Issue Type: Improvement
>          Components: master
>            Reporter: Benjamin Mahler
>              Labels: newbie, scalability, twitter
>
> The master's state.json endpoint consistently takes a long time to compute the JSON result, for large clusters:
> {noformat}
> $ time curl -s -o /dev/null localhost:5050/master/state.json
> Mon Jan 26 22:38:50 UTC 2015
> real	0m13.174s
> user	0m0.003s
> sys	0m0.022s
> {noformat}
> This can cause the master to get backlogged if there are many state.json requests in flight.
> Looking at {{perf}} data, it seems most of the time is spent doing memory allocation / de-allocation. This ticket will try to capture any low hanging fruit to speed this up. Possibly we can leverage moves if they are not already being used by the compiler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)