[jira] [Comment Edited] (MESOS-7944) Implement jemalloc support for Mesos

["Benno Evers (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-7944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161001#comment-16161001 ] 

Benno Evers edited comment on MESOS-7944 at 9/11/17 9:59 AM:
-------------------------------------------------------------

Since I've started to work on this, I have now a much better idea of what needs to be done.

First of all, since the added features are not mesos-specific, I think it's best to add them directly to libprocess. However, the choice of preferred malloc should be up the binary, not enforced by a shared library, so instead compiling against jemalloc we should detect at runtime whether we're running under jemalloc or not. (similar to what folly does here: https://github.com/facebook/folly/blob/master/folly/Malloc.h#L150)

At the endpoint, the minimum features I would like are the ability to get the (exact) heap allocation statistics as JSON, or download current (stochastic) heap profile dumps as files. Depending on the complexity of it, we should also think about providing a way to have the master dump profiles periodically and store them on disk, and a way to generate jeprof-graphs automatically.

Finally, the new `--enable-memory-profiling` configure option (tentative name) for mesos would build a bundled version of jemalloc with all the necessary configuration options enabled, and link the mesos-master and mesos-slave binaries against this library.


was (Author: bennoe):
Since I've started to work on this, I have now a much sharper idea of what needs to be done.

First of all, since the added features are not mesos-specific, I think it's best to add them directly to libprocess. However, the choice of preferred malloc should be up the binary, not enforced by a shared library, so instead compiling against jemalloc we should detect at runtime whether we're running under jemalloc or not. (similar to what folly does here: https://github.com/facebook/folly/blob/master/folly/Malloc.h#L150)

At the endpoint, the minimum features I would like are the ability to get the (exact) heap allocation statistics as JSON, or download current (stochastic) heap profile dumps as files. Depending on the complexity of it, we should also think about providing a way to have the master dump profiles periodically and store them on disk, and a way to generate jeprof-graphs automatically.

Finally, the new `--enable-memory-profiling` configure option (tentative name) for mesos would build a bundled version of jemalloc with all the necessary configuration options enabled, and link the mesos-master and mesos-slave binaries against this library.

> Implement jemalloc support for Mesos
> ------------------------------------
>
>                 Key: MESOS-7944
>                 URL: https://issues.apache.org/jira/browse/MESOS-7944
>             Project: Mesos
>          Issue Type: Bug
>            Reporter: Benno Evers
>            Assignee: Benno Evers
>
> After investigation in MESOS-7876 and discussion on the mailing list, this task is for tracking progress on adding out-of-the-box memory profiling support using jemalloc to Mesos.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)