Using jemalloc as default allocator

[Benno Evers <be...@mesosphere.com>]

Hi guys,

it's quite late in the release cycle, but I've been thinking about
proposing to enable the `--enable-jemalloc-allocator` configuration setting
by default for linux builds of Mesos.

The thinking is that
 - Benchmarks consistently show a small to medium performance improvement
 - The bundled jemalloc version (5.0.1) has been released as stable for
over a year and has not seen any severe bugs
 - Our own Mesos builds with jemalloc don't show any issues so far

What do you think?

Best regards,
-- 
Benno Evers
Software Engineer, Mesosphere

Re: Using jemalloc as default allocator

[Benjamin Mahler <bm...@apache.org>]

I would be interested in knowing what other projects have done around this
(e.g. Rust, Redis seem to use it by default on Linux, and I see ongoing
discussion in other projects e.g. Ruby).

To James' point, while anyone technically can use jemalloc, I have only
seen 1 user doing it. Maybe there are more, but it certainly isn't
widespread. Sometimes our users don't even know about whether they're
running an optimized build, they just install a package and run it.

We should aim to do something better by default for users, if possible, no?

Also, james, which profiling support are you referring to in the context of
jemalloc? Are you saying that memory profiling works without it? Right now
as far as I know, we can't use memory profiling when trying to investigate
user issues because nearly no one is using jemalloc.

On Mon, Aug 13, 2018 at 3:48 AM Benno Evers <be...@mesosphere.com> wrote:

> Ok then, let's not do it for now.
>
> Best regards,
>
> On Fri, Aug 10, 2018 at 6:10 PM, James Peach <jp...@apache.org> wrote:
>
>>
>>
>> > On Aug 10, 2018, at 8:56 AM, Benno Evers <be...@mesosphere.com> wrote:
>> >
>> > Hi guys,
>> >
>> > it's quite late in the release cycle, but I've been thinking about
>> > proposing to enable the `--enable-jemalloc-allocator` configuration
>> setting
>> > by default for linux builds of Mesos.
>> >
>> > The thinking is that
>> > - Benchmarks consistently show a small to medium performance improvement
>> > - The bundled jemalloc version (5.0.1) has been released as stable for
>> > over a year and has not seen any severe bugs
>> > - Our own Mesos builds with jemalloc don't show any issues so far
>> >
>> > What do you think?
>>
>> I don't think it's worth it. Anyone who wants to use jemalloc can already
>> do it, and the Mesos profiling support works nicely without also forcing a
>> build-time dependency. In general, I feel that bundling dependencies is a
>> burden on the build (e.g out bundled jemalloc is already out of date).
>>
>> J
>
>
>
>
> --
> Benno Evers
> Software Engineer, Mesosphere
>

Re: Using jemalloc as default allocator

[Benno Evers <be...@mesosphere.com>]

Ok then, let's not do it for now.

Best regards,

On Fri, Aug 10, 2018 at 6:10 PM, James Peach <jp...@apache.org> wrote:

>
>
> > On Aug 10, 2018, at 8:56 AM, Benno Evers <be...@mesosphere.com> wrote:
> >
> > Hi guys,
> >
> > it's quite late in the release cycle, but I've been thinking about
> > proposing to enable the `--enable-jemalloc-allocator` configuration
> setting
> > by default for linux builds of Mesos.
> >
> > The thinking is that
> > - Benchmarks consistently show a small to medium performance improvement
> > - The bundled jemalloc version (5.0.1) has been released as stable for
> > over a year and has not seen any severe bugs
> > - Our own Mesos builds with jemalloc don't show any issues so far
> >
> > What do you think?
>
> I don't think it's worth it. Anyone who wants to use jemalloc can already
> do it, and the Mesos profiling support works nicely without also forcing a
> build-time dependency. In general, I feel that bundling dependencies is a
> burden on the build (e.g out bundled jemalloc is already out of date).
>
> J




-- 
Benno Evers
Software Engineer, Mesosphere

Re: Using jemalloc as default allocator

[James Peach <jp...@apache.org>]

> On Aug 10, 2018, at 8:56 AM, Benno Evers <be...@mesosphere.com> wrote:
> 
> Hi guys,
> 
> it's quite late in the release cycle, but I've been thinking about
> proposing to enable the `--enable-jemalloc-allocator` configuration setting
> by default for linux builds of Mesos.
> 
> The thinking is that
> - Benchmarks consistently show a small to medium performance improvement
> - The bundled jemalloc version (5.0.1) has been released as stable for
> over a year and has not seen any severe bugs
> - Our own Mesos builds with jemalloc don't show any issues so far
> 
> What do you think?

I don't think it's worth it. Anyone who wants to use jemalloc can already do it, and the Mesos profiling support works nicely without also forcing a build-time dependency. In general, I feel that bundling dependencies is a burden on the build (e.g out bundled jemalloc is already out of date).

J