You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Antoine Pitrou <an...@python.org> on 2024/06/05 15:18:36 UTC
[Discuss][C++] Switch to mimalloc by default?
Hello,
Arrow C++ features a MemoryPool abstraction that allows using different
allocators interchangeably. Several MemoryPool implementations are
provided with Arrow C++ (though one can also build their own):
- a jemalloc-based implementation, currently the default on Linux
- a mimalloc-based implementation, currently the default on macOS and
Windows
- an implementation that defers to the system's standard allocator
(using the malloc() and free() calls), available as a fallback and for
experimentation
While jemalloc is the current default on Linux, our continuous
benchmarking infrastructure actually enables mimalloc instead.
Therefore, I've made a draft PR that switches our benchmarking to
jemalloc, so as to measure any concrete differences between the two:
https://github.com/apache/arrow/pull/41205
The results show that there is a large number of performance drops with
large effect sizes on the C++ microbenchmarks. There is also a smaller
number of C++ microbenchmarks with improved performance results. A
summary report with links to detailed results can be found here:
https://github.com/apache/arrow/runs/25745674261
With this in mind, I would like to propose that we switch the default to
mimalloc for all platforms. This would have several desirable effects:
- less variability between platforms
- mimalloc generally has a nicer, more consistent API and is easier to
work with (in particular, jemalloc's configuration scheme is slightly
abtruse)
- potentially better performance, or at least not significantly worse,
than the statu quo
We would have to keep at least one CI job with jemalloc enabled, to make
sure we're not regressing in that regard.
What do you think?
Regards
Antoine.
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Antoine Pitrou <an...@python.org>.
Hello,
Thanks all for this discussion. Given that there was no strong argument
against doing this, I decided to move forward and the change was made in
https://github.com/apache/arrow/pull/40875
Regards
Antoine.
On Wed, 5 Jun 2024 17:18:36 +0200
Antoine Pitrou <an...@python.org> wrote:
> Hello,
>
> Arrow C++ features a MemoryPool abstraction that allows using different
> allocators interchangeably. Several MemoryPool implementations are
> provided with Arrow C++ (though one can also build their own):
>
> - a jemalloc-based implementation, currently the default on Linux
> - a mimalloc-based implementation, currently the default on macOS and
> Windows
> - an implementation that defers to the system's standard allocator
> (using the malloc() and free() calls), available as a fallback and for
> experimentation
>
> While jemalloc is the current default on Linux, our continuous
> benchmarking infrastructure actually enables mimalloc instead.
> Therefore, I've made a draft PR that switches our benchmarking to
> jemalloc, so as to measure any concrete differences between the two:
> https://github.com/apache/arrow/pull/41205
>
> The results show that there is a large number of performance drops with
> large effect sizes on the C++ microbenchmarks. There is also a smaller
> number of C++ microbenchmarks with improved performance results. A
> summary report with links to detailed results can be found here:
> https://github.com/apache/arrow/runs/25745674261
>
>
> With this in mind, I would like to propose that we switch the default to
> mimalloc for all platforms. This would have several desirable effects:
>
> - less variability between platforms
> - mimalloc generally has a nicer, more consistent API and is easier to
> work with (in particular, jemalloc's configuration scheme is slightly
> abtruse)
> - potentially better performance, or at least not significantly worse,
> than the statu quo
>
> We would have to keep at least one CI job with jemalloc enabled, to make
> sure we're not regressing in that regard.
>
> What do you think?
>
> Regards
>
> Antoine.
>
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Sutou Kouhei <ko...@clear-code.com>.
Thanks for sharing jemalloc use-cases. We should keep it.
In <CA...@mail.gmail.com>
"Re: [Discuss][C++] Switch to mimalloc by default?" on Mon, 10 Jun 2024 11:37:09 -0700,
Anja <an...@gmail.com> wrote:
> I do know of use-cases where users explicitly set jemalloc as their
> preferred allocator. I would recommend keeping it.
>
> On Mon, 10 Jun 2024 at 08:09, Antoine Pitrou <an...@python.org> wrote:
>
>>
>> Hi Kou,
>>
>> Le 09/06/2024 à 09:16, Sutou Kouhei a écrit :
>> >
>> > Questions:
>> >
>> > 1. Do we need to keep jemalloc support? Compatibility? Can we
>> > drop support for jemalloc to decrease maintenance cost?
>>
>> I'm not sure there's much maintenance cost. I expect some people might
>> prefer jemalloc, and perhaps it performs better on some use cases. So
>> for now I would recommend keeping it.
>>
>> > 2. Is it OK that we add support for system mimalloc?
>>
>> Hmm... that sounds legitimate, but with the caveat that a system
>> mimalloc can override the standard malloc/free functions. Would that
>> affect an application using Arrow C++?
>>
>> > FYI: In general, I want to use system libraries as much as
>> > possible. But we can't use system jemalloc for bindings
>> > because most system jemalloc don't support dlopen():
>> > https://github.com/apache/arrow/issues/32530
>>
>> We use something similar for mimalloc:
>>
>> https://github.com/apache/arrow/blob/036fca0ae5c8956c83b69478d413c24f32398f8c/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2223
>>
>> Regards
>>
>> Antoine.
>>
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
In <fd...@python.org>
"Re: [Discuss][C++] Switch to mimalloc by default?" on Wed, 12 Jun 2024 11:03:42 +0200,
Antoine Pitrou <an...@python.org> wrote:
>> Can we use "-lc -lmimalloc" for system mimalloc?
>
> I don't know, it sounds rather fragile to me. Are we sure this would
> always prevent any problem from happening?
Hmm. We need to try to confirm it.
Anyway, this is not a blocker to switch to mimalloc by
default.
I've opened a new issue for this. We can discuss this
further in the issue:
https://github.com/apache/arrow/issues/42211
Thanks,
--
kou
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Antoine Pitrou <an...@python.org>.
Le 12/06/2024 à 04:45, Sutou Kouhei a écrit :
>
> It seems that we need to disable MI_OVERRIDE explicitly to
> not define malloc() in libmimalloc.so:
> https://github.com/microsoft/mimalloc/blob/03020fbf81541651e24289d2f7033a772a50f480/CMakeLists.txt#L10
Yes, that's what we do when building the bundled mimalloc :-)
> But we can't use LD_PRELOAD with the option. So system
> mimalloc will not disable MI_OVERRIDE.
A system mimalloc package should ideally provide two shared libraries:
* a `libmimalloc.so` with MI_OVERRIDE disabled
* a `libmimalloc-override.so` with MI_OVERRIDE enabled
But it's not our decision to make anyway.
> Can we use "-lc -lmimalloc" for system mimalloc?
I don't know, it sounds rather fragile to me. Are we sure this would
always prevent any problem from happening?
Regards
Antoine.
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
In <53...@python.org>
"Re: [Discuss][C++] Switch to mimalloc by default?" on Tue, 11 Jun 2024 10:45:12 +0200,
Antoine Pitrou <an...@python.org> wrote:
>> mimalloc doesn't override malloc()/free() as long as we
>> don't include mimalloc-override.h:
>> https://github.com/microsoft/mimalloc/blob/master/include/mimalloc-override.h
>
> I see malloc() defined by libmimalloc.so.2 here (Ubuntu 22.04):
I think that it's not used because malloc() in libc.so is
used. But I was wrong...
a.c:
----
#include <stdlib.h>
int
main(void)
{
free(malloc(2 * 1024L * 1024L * 1024L));
return 0;
}
----
malloc() of mimalloc is used:
----
$ gcc -o a a.c -lmimalloc && MIMALLOC_SHOW_STATS=1 ./a
heap stats: peak total freed current unit count
reserved: 2.0 GiB 4.0 GiB 4.0 GiB 0 ok
committed: 2.0 GiB 4.0 GiB 4.0 GiB 0 ok
reset: 0
purged: 0
touched: 64.2 KiB 64.2 KiB 2.0 GiB -2.0 GiB ok
segments: 1 1 1 0 ok
-abandoned: 0 0 0 0 ok
-cached: 0 0 0 0 ok
pages: 0 0 1 -1 ok
-abandoned: 0 0 0 0 ok
-extended: 0
-noretire: 0
arenas: 0
-crossover: 0
-rollback: 0
mmaps: 0
commits: 0
resets: 0
purges: 0
threads: 0 0 0 0 ok
searches: 0.0 avg
numa nodes: 1
elapsed: 0.001 s
process: user: 0.000 s, system: 0.001 s, faults: 0, rss: 18.2 MiB, commit: 2.0 GiB
----
If we link to libc.so explicitly, malloc() of mimalloc is
not used:
----
$ gcc -o a a.c -lc -lmimalloc && MIMALLOC_SHOW_STATS=1 ./a
----
It seems that we need to disable MI_OVERRIDE explicitly to
not define malloc() in libmimalloc.so:
https://github.com/microsoft/mimalloc/blob/03020fbf81541651e24289d2f7033a772a50f480/CMakeLists.txt#L10
But we can't use LD_PRELOAD with the option. So system
mimalloc will not disable MI_OVERRIDE.
Can we use "-lc -lmimalloc" for system mimalloc?
Or should we keep using bundled mimalloc?
Thanks,
--
kou
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Antoine Pitrou <an...@python.org>.
Le 11/06/2024 à 10:35, Sutou Kouhei a écrit :
> Hi,
>
> In <2a...@python.org>
> "Re: [Discuss][C++] Switch to mimalloc by default?" on Tue, 11 Jun 2024 10:21:12 +0200,
> Antoine Pitrou <an...@python.org> wrote:
>
>> I was thinking about find_package(). Good to know that it doesn't
>> affect the application, but would it affect Arrow C++ then?
>
> mimalloc doesn't override malloc()/free() as long as we
> don't include mimalloc-override.h:
> https://github.com/microsoft/mimalloc/blob/master/include/mimalloc-override.h
I see malloc() defined by libmimalloc.so.2 here (Ubuntu 22.04):
$ nm --dynamic --defined-only
/usr/lib/x86_64-linux-gnu/libmimalloc.so.2.0 | \grep "malloc"
0000000000012520 T _aligned_malloc
0000000000011380 T __libc_malloc
0000000000011380 T malloc
00000000000124e0 T malloc_good_size
000000000000a460 T malloc_size
000000000000a460 T malloc_usable_size
0000000000011260 T mi_heap_malloc
000000000001acd0 T mi_heap_malloc_aligned
000000000001a6c0 T mi_heap_malloc_aligned_at
0000000000011cb0 T mi_heap_mallocn
0000000000011100 T mi_heap_malloc_small
0000000000011380 T mi_malloc
000000000001ad10 T mi_malloc_aligned
000000000001a6d0 T mi_malloc_aligned_at
0000000000019410 T mi_malloc_good_size
0000000000011e00 T mi_mallocn
00000000000189e0 T mi_malloc_size
00000000000117b0 T mi_malloc_small
00000000000189f0 T mi_malloc_usable_size
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
In <2a...@python.org>
"Re: [Discuss][C++] Switch to mimalloc by default?" on Tue, 11 Jun 2024 10:21:12 +0200,
Antoine Pitrou <an...@python.org> wrote:
> I was thinking about find_package(). Good to know that it doesn't
> affect the application, but would it affect Arrow C++ then?
mimalloc doesn't override malloc()/free() as long as we
don't include mimalloc-override.h:
https://github.com/microsoft/mimalloc/blob/master/include/mimalloc-override.h
So arrow::MimallocMemoryPool only uses mimalloc and other
Arrow C++ codes don't use mimalloc.
> There
> might be a problem if a pointer is allocated by Arrow C++ (using the
> mimalloc malloc()) and deallocated by the application (using the libc
> free()).
If the application mixes
arrow::MemoryPool::Allocate()/Reallocate() and free(), it's
happen. But it's an application bug. (The application must
use arrow::MemoryPool::Free().) So we don't need to care
about the case.
Thanks,
--
kou
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Ruoxi Sun <za...@gmail.com>.
+1 on switching the default to mimalloc meanwhile keeping jemalloc as an
alternative.
*Regards,*
*Rossi SUN*
On Tue, Jun 11, 2024 at 2:38 AM Anja <an...@gmail.com> wrote:
> I do know of use-cases where users explicitly set jemalloc as their
> preferred allocator. I would recommend keeping it.
>
> On Mon, 10 Jun 2024 at 08:09, Antoine Pitrou <an...@python.org> wrote:
>
> >
> > Hi Kou,
> >
> > Le 09/06/2024 à 09:16, Sutou Kouhei a écrit :
> > >
> > > Questions:
> > >
> > > 1. Do we need to keep jemalloc support? Compatibility? Can we
> > > drop support for jemalloc to decrease maintenance cost?
> >
> > I'm not sure there's much maintenance cost. I expect some people might
> > prefer jemalloc, and perhaps it performs better on some use cases. So
> > for now I would recommend keeping it.
> >
> > > 2. Is it OK that we add support for system mimalloc?
> >
> > Hmm... that sounds legitimate, but with the caveat that a system
> > mimalloc can override the standard malloc/free functions. Would that
> > affect an application using Arrow C++?
> >
> > > FYI: In general, I want to use system libraries as much as
> > > possible. But we can't use system jemalloc for bindings
> > > because most system jemalloc don't support dlopen():
> > > https://github.com/apache/arrow/issues/32530
> >
> > We use something similar for mimalloc:
> >
> >
> https://github.com/apache/arrow/blob/036fca0ae5c8956c83b69478d413c24f32398f8c/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2223
> >
> > Regards
> >
> > Antoine.
> >
>
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Anja <an...@gmail.com>.
I do know of use-cases where users explicitly set jemalloc as their
preferred allocator. I would recommend keeping it.
On Mon, 10 Jun 2024 at 08:09, Antoine Pitrou <an...@python.org> wrote:
>
> Hi Kou,
>
> Le 09/06/2024 à 09:16, Sutou Kouhei a écrit :
> >
> > Questions:
> >
> > 1. Do we need to keep jemalloc support? Compatibility? Can we
> > drop support for jemalloc to decrease maintenance cost?
>
> I'm not sure there's much maintenance cost. I expect some people might
> prefer jemalloc, and perhaps it performs better on some use cases. So
> for now I would recommend keeping it.
>
> > 2. Is it OK that we add support for system mimalloc?
>
> Hmm... that sounds legitimate, but with the caveat that a system
> mimalloc can override the standard malloc/free functions. Would that
> affect an application using Arrow C++?
>
> > FYI: In general, I want to use system libraries as much as
> > possible. But we can't use system jemalloc for bindings
> > because most system jemalloc don't support dlopen():
> > https://github.com/apache/arrow/issues/32530
>
> We use something similar for mimalloc:
>
> https://github.com/apache/arrow/blob/036fca0ae5c8956c83b69478d413c24f32398f8c/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2223
>
> Regards
>
> Antoine.
>
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Antoine Pitrou <an...@python.org>.
Le 11/06/2024 à 10:01, Sutou Kouhei a écrit :
>
>>> 2. Is it OK that we add support for system mimalloc?
>>
>> Hmm... that sounds legitimate, but with the caveat that a system
>> mimalloc can override the standard malloc/free functions. Would that
>> affect an application using Arrow C++?
>
> Are you saying the LD_PRELOAD approach not the
> find_package(mimalloc) approach? If so, it affects an
> application using Arrow C++. If we use
> the find_package(mimalloc) approach, it doesn't affect an
> application using Arrow C++.
I was thinking about find_package(). Good to know that it doesn't affect
the application, but would it affect Arrow C++ then? There might be a
problem if a pointer is allocated by Arrow C++ (using the mimalloc
malloc()) and deallocated by the application (using the libc free()).
Regards
Antoine.
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
> I'm not sure there's much maintenance cost.
It's not high maintenance cost but it's sometimes broken.
For example: https://github.com/apache/arrow/issues/39517
> I expect some people might
> prefer jemalloc, and perhaps it performs better on some use cases. So
> for now I would recommend keeping it.
OK. If there are users who prefer jemalloc, let's keep
jemalloc support.
>> 2. Is it OK that we add support for system mimalloc?
>
> Hmm... that sounds legitimate, but with the caveat that a system
> mimalloc can override the standard malloc/free functions. Would that
> affect an application using Arrow C++?
Are you saying the LD_PRELOAD approach not the
find_package(mimalloc) approach? If so, it affects an
application using Arrow C++. If we use
the find_package(mimalloc) approach, it doesn't affect an
application using Arrow C++.
>> FYI: In general, I want to use system libraries as much as
>> possible. But we can't use system jemalloc for bindings
>> because most system jemalloc don't support dlopen():
>> https://github.com/apache/arrow/issues/32530
>
> We use something similar for mimalloc:
> https://github.com/apache/arrow/blob/036fca0ae5c8956c83b69478d413c24f32398f8c/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2223
Oh...
Anyway, I tried system mimalloc:
https://github.com/apache/arrow/pull/42090
Debian's mimalloc doesn't use -DMI_LOCAL_DYNAMIC_TLS=ON but
there is no dlopen() related error.
https://salsa.debian.org/debian/mimalloc/-/blob/master/debian/rules?ref_type=heads#L21
So I prefer mimalloc to jemalloc.
Thanks,
--
kou
In <eb...@python.org>
"Re: [Discuss][C++] Switch to mimalloc by default?" on Mon, 10 Jun 2024 17:08:58 +0200,
Antoine Pitrou <an...@python.org> wrote:
>
> Hi Kou,
>
> Le 09/06/2024 à 09:16, Sutou Kouhei a écrit :
>> Questions:
>> 1. Do we need to keep jemalloc support? Compatibility? Can we
>> drop support for jemalloc to decrease maintenance cost?
>
> I'm not sure there's much maintenance cost. I expect some people might
> prefer jemalloc, and perhaps it performs better on some use cases. So
> for now I would recommend keeping it.
>
>> 2. Is it OK that we add support for system mimalloc?
>
> Hmm... that sounds legitimate, but with the caveat that a system
> mimalloc can override the standard malloc/free functions. Would that
> affect an application using Arrow C++?
>
>> FYI: In general, I want to use system libraries as much as
>> possible. But we can't use system jemalloc for bindings
>> because most system jemalloc don't support dlopen():
>> https://github.com/apache/arrow/issues/32530
>
> We use something similar for mimalloc:
> https://github.com/apache/arrow/blob/036fca0ae5c8956c83b69478d413c24f32398f8c/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2223
>
> Regards
>
> Antoine.
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Antoine Pitrou <an...@python.org>.
Hi Kou,
Le 09/06/2024 à 09:16, Sutou Kouhei a écrit :
>
> Questions:
>
> 1. Do we need to keep jemalloc support? Compatibility? Can we
> drop support for jemalloc to decrease maintenance cost?
I'm not sure there's much maintenance cost. I expect some people might
prefer jemalloc, and perhaps it performs better on some use cases. So
for now I would recommend keeping it.
> 2. Is it OK that we add support for system mimalloc?
Hmm... that sounds legitimate, but with the caveat that a system
mimalloc can override the standard malloc/free functions. Would that
affect an application using Arrow C++?
> FYI: In general, I want to use system libraries as much as
> possible. But we can't use system jemalloc for bindings
> because most system jemalloc don't support dlopen():
> https://github.com/apache/arrow/issues/32530
We use something similar for mimalloc:
https://github.com/apache/arrow/blob/036fca0ae5c8956c83b69478d413c24f32398f8c/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2223
Regards
Antoine.
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
+1 based on the benchmark results.
Questions:
1. Do we need to keep jemalloc support? Compatibility? Can we
drop support for jemalloc to decrease maintenance cost?
2. Is it OK that we add support for system mimalloc?
We always use vendored mimalloc for now:
https://github.com/apache/arrow/blob/399408cb273c47f490f65cdad95bc184a652826c/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2197-L2251
FYI: In general, I want to use system libraries as much as
possible. But we can't use system jemalloc for bindings
because most system jemalloc don't support dlopen():
https://github.com/apache/arrow/issues/32530
If mimalloc doesn't have the restriction, mimalloc is better
for me.
Thanks,
--
kou
In <c6...@python.org>
"[Discuss][C++] Switch to mimalloc by default?" on Wed, 5 Jun 2024 17:18:36 +0200,
Antoine Pitrou <an...@python.org> wrote:
>
> Hello,
>
> Arrow C++ features a MemoryPool abstraction that allows using
> different allocators interchangeably. Several MemoryPool
> implementations are provided with Arrow C++ (though one can also build
> their own):
>
> - a jemalloc-based implementation, currently the default on Linux
> - a mimalloc-based implementation, currently the default on macOS and
> - Windows
> - an implementation that defers to the system's standard allocator
> - (using the malloc() and free() calls), available as a fallback and for
> - experimentation
>
> While jemalloc is the current default on Linux, our continuous
> benchmarking infrastructure actually enables mimalloc
> instead. Therefore, I've made a draft PR that switches our
> benchmarking to jemalloc, so as to measure any concrete differences
> between the two:
> https://github.com/apache/arrow/pull/41205
>
> The results show that there is a large number of performance drops
> with large effect sizes on the C++ microbenchmarks. There is also a
> smaller number of C++ microbenchmarks with improved performance
> results. A summary report with links to detailed results can be found
> here:
> https://github.com/apache/arrow/runs/25745674261
>
>
> With this in mind, I would like to propose that we switch the default
> to mimalloc for all platforms. This would have several desirable
> effects:
>
> - less variability between platforms
> - mimalloc generally has a nicer, more consistent API and is easier to
> - work with (in particular, jemalloc's configuration scheme is slightly
> - abtruse)
> - potentially better performance, or at least not significantly worse,
> - than the statu quo
>
> We would have to keep at least one CI job with jemalloc enabled, to
> make sure we're not regressing in that regard.
>
> What do you think?
>
> Regards
>
> Antoine.
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Felipe Oliveira Carvalho <fe...@gmail.com>.
+1. I think the benefits outweigh the risks.
On Wed, Jun 5, 2024 at 3:05 PM Anja <an...@gmail.com> wrote:
>
> I did want to start off by acknowledging that all of the pros you listed
> for mimalloc are accurate.
>
> I did want to contribute the times that people have been caught off-guard
> by the perceived increased memory allocation of mimalloc compared to the
> alternatives:
> E.g. https://github.com/microsoft/mimalloc/issues/393
> https://github.com/apache/arrow/issues/37361
>
> From your investigation Antoine, it does seem that the bulk of this is
> actually "pre-allocated virtual memory":
> https://github.com/apache/arrow/issues/40301 and thus people don't need to
> be generally concerned about this.
>
> I'm working on a blogpost that will hopefully decrease the number of
> "mimalloc is over-allocating memory" issues we will get. I think I should
> get this ready in anticipation of the switch. We don't want a bunch of open
> issues about memory leaks.
>
> On Wed, 5 Jun 2024 at 08:18, Antoine Pitrou <an...@python.org> wrote:
>
> >
> > Hello,
> >
> > Arrow C++ features a MemoryPool abstraction that allows using different
> > allocators interchangeably. Several MemoryPool implementations are
> > provided with Arrow C++ (though one can also build their own):
> >
> > - a jemalloc-based implementation, currently the default on Linux
> > - a mimalloc-based implementation, currently the default on macOS and
> > Windows
> > - an implementation that defers to the system's standard allocator
> > (using the malloc() and free() calls), available as a fallback and for
> > experimentation
> >
> > While jemalloc is the current default on Linux, our continuous
> > benchmarking infrastructure actually enables mimalloc instead.
> > Therefore, I've made a draft PR that switches our benchmarking to
> > jemalloc, so as to measure any concrete differences between the two:
> > https://github.com/apache/arrow/pull/41205
> >
> > The results show that there is a large number of performance drops with
> > large effect sizes on the C++ microbenchmarks. There is also a smaller
> > number of C++ microbenchmarks with improved performance results. A
> > summary report with links to detailed results can be found here:
> > https://github.com/apache/arrow/runs/25745674261
> >
> >
> > With this in mind, I would like to propose that we switch the default to
> > mimalloc for all platforms. This would have several desirable effects:
> >
> > - less variability between platforms
> > - mimalloc generally has a nicer, more consistent API and is easier to
> > work with (in particular, jemalloc's configuration scheme is slightly
> > abtruse)
> > - potentially better performance, or at least not significantly worse,
> > than the statu quo
> >
> > We would have to keep at least one CI job with jemalloc enabled, to make
> > sure we're not regressing in that regard.
> >
> > What do you think?
> >
> > Regards
> >
> > Antoine.
> >
Re: [Discuss][C++] Switch to mimalloc by default?
Posted by Anja <an...@gmail.com>.
I did want to start off by acknowledging that all of the pros you listed
for mimalloc are accurate.
I did want to contribute the times that people have been caught off-guard
by the perceived increased memory allocation of mimalloc compared to the
alternatives:
E.g. https://github.com/microsoft/mimalloc/issues/393
https://github.com/apache/arrow/issues/37361
From your investigation Antoine, it does seem that the bulk of this is
actually "pre-allocated virtual memory":
https://github.com/apache/arrow/issues/40301 and thus people don't need to
be generally concerned about this.
I'm working on a blogpost that will hopefully decrease the number of
"mimalloc is over-allocating memory" issues we will get. I think I should
get this ready in anticipation of the switch. We don't want a bunch of open
issues about memory leaks.
On Wed, 5 Jun 2024 at 08:18, Antoine Pitrou <an...@python.org> wrote:
>
> Hello,
>
> Arrow C++ features a MemoryPool abstraction that allows using different
> allocators interchangeably. Several MemoryPool implementations are
> provided with Arrow C++ (though one can also build their own):
>
> - a jemalloc-based implementation, currently the default on Linux
> - a mimalloc-based implementation, currently the default on macOS and
> Windows
> - an implementation that defers to the system's standard allocator
> (using the malloc() and free() calls), available as a fallback and for
> experimentation
>
> While jemalloc is the current default on Linux, our continuous
> benchmarking infrastructure actually enables mimalloc instead.
> Therefore, I've made a draft PR that switches our benchmarking to
> jemalloc, so as to measure any concrete differences between the two:
> https://github.com/apache/arrow/pull/41205
>
> The results show that there is a large number of performance drops with
> large effect sizes on the C++ microbenchmarks. There is also a smaller
> number of C++ microbenchmarks with improved performance results. A
> summary report with links to detailed results can be found here:
> https://github.com/apache/arrow/runs/25745674261
>
>
> With this in mind, I would like to propose that we switch the default to
> mimalloc for all platforms. This would have several desirable effects:
>
> - less variability between platforms
> - mimalloc generally has a nicer, more consistent API and is easier to
> work with (in particular, jemalloc's configuration scheme is slightly
> abtruse)
> - potentially better performance, or at least not significantly worse,
> than the statu quo
>
> We would have to keep at least one CI job with jemalloc enabled, to make
> sure we're not regressing in that regard.
>
> What do you think?
>
> Regards
>
> Antoine.
>