You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Stefan Fritsch <sf...@sfritsch.de> on 2011/02/06 23:34:44 UTC
reducing memory fragmentation
On Friday 04 February 2011, William A. Rowe Jr. wrote:
> >> My thought is that this might not be the only thing to 'refresh'
> >> when you have an app which needs current state. It would be a
> >> chance to flush out all caches and set the state basically to
> >> apr_initialize(), modulo any open resources or currently
> >> allocated pools. It could even go so far as to free() all the
> >> unused pool memory, giving us a really fresh start on heap
> >> pages that have been fragmented to heck.
> >
> > ++1 for some way of handling the fragmentation... of course,
> > we could just switch to pocore for the memory stuff in APR. :)
>
> Which helps for apr_allocator but not for the internal
> fragmentation within individual pools. An optional pocore feature
> for APR 2 sounds great, patches welcome :)
The pools give their memory back to the allocator when they are
cleared. Therefore at least the transaction pools and their children
should not be a major problem for fragmentation.
However, one major problem are the allocators not giving back memory,
or if configured with max_mem_free, giving it back in random order so
that it is very unlikely that the malloc implementation can give
memory back to the OS.
A second problem I have noticed with glibc: Its malloc allocates APR's
typical multiple-of-4k chunks in a normal linked list. This means that
there is some N bytes header allocated before each chunk. Now if APR
has allocated say 16k, and frees it again, the hole is 16k+N. However
this is not big enough to allocate two 8k chunks, which would need
(8k+N)*2 == 16k+2N bytes. I think this may contribute to the
fragmentation seen under Linux. In addition to that, glibc's malloc
allocates smaller chunks in the same linked list, so in practice we
have something like this:
8K pool chunk
8K pool chunk
some bytes from a third party lib
12K pool chunk
some more bytes from normal malloc
8K pool chunk
This doesn't help with fragmentation either.
Therefore I propose to use mmap/mumap in the apr_allocator instead of
malloc, either as a compile time option or whenever max_mem_free is
set for the allocator. This would then completely separate the
allocations from apr_allocator and normal malloc and reduce
fragmentation. And it would make apr_allocator_max_free_set actually
give memory back to the OS.
This would be a relatively simple change (compared to replacing the
whole allocator). Sounds ok?
Re: reducing memory fragmentation
Posted by Stefan Fritsch <sf...@sfritsch.de>.
On Sunday 06 February 2011, Stefan Fritsch wrote:
> Therefore I propose to use mmap/mumap in the apr_allocator instead
> of malloc, either as a compile time option or whenever
> max_mem_free is set for the allocator. This would then completely
> separate the allocations from apr_allocator and normal malloc and
> reduce fragmentation. And it would make apr_allocator_max_free_set
> actually give memory back to the OS.
>
> This would be a relatively simple change (compared to replacing the
> whole allocator). Sounds ok?
I have made some measurements for this, doing some relatively complex
requests (a dir index of a dir with 10000 files) with apr 1.4.2/httpd
trunk/mpm_event under Linux/glibc:
reqs/s mem usage
after tests
malloc MaxMemFree unlimited 42 92M
malloc MaxMemFree 128 41 93M
malloc MaxMemFree 4 43 70M
mmap MaxMemFree unlimited 45 85M
mmap MaxMemFree 512 43 66M
mmap MaxMemFree 128 35 31M
mmap MaxMemFree 4 32 21M
The accuracy is not too high, I guess all the numbers are taken to be
+/- 1.
As you see, the the current allocator_alloc version which uses malloc
does not really allow to give back significant ammounts of RAM to the
OS, regardless of MaxMemFree. But with mmap, MaxMemFree has a
significant effect. And with the same ammount of RAM used, the mmap
version is not slower than the malloc version. Therefore with Linux
glibc, the mmap version seems superior in all respects.
I will commit this alternative allocator selectable with a configure
option. For now, it will need to be manually selected. Later, we can
think about changing the default for some platforms.
Cheers,
Stefan
Re: reducing memory fragmentation
Posted by Stefan Fritsch <sf...@sfritsch.de>.
On Sunday 20 February 2011, Jim Jagielski wrote:
> On Feb 19, 2011, at 2:57 PM, Stefan Fritsch wrote:
> > I am sure that apr_allocator with mmap is not an advantage on all
> > platforms. Actually, apr allocating only multiples of 4k should
> > make it easier for malloc to avoid fragmentation. But glibc's
> > malloc fails miserably in that regard.
>
> Is there a way during configuration time that we can do
> some rough testing on "best" multiple to use...?
I don't know about the "best" multiple. One would probably have to do
benchmarks for that. But it is possible to check for mallocs that
behave similar to glibc's malloc. If several allocated chunks are not
allocated with distances that are a multiples of the page size, the
malloc is inefficient.
But before activating such a configure check, I would wait for some
feedback from some people using the mmap option in real-world.
For glibc/i386, the attached test program gives:
$ ./malloc_test ; echo $?
pagesize: 4096
minalloc: 8192
malloc offset: 8
1
If you force glibc to use mmap, the result is different. But i don't
know how efficient glibc is in this mode:
$ MALLOC_MMAP_THRESHOLD_=8192 ./malloc_test ; echo $?
pagesize: 4096
minalloc: 8192
malloc offset: 0
0
On FreeBSD amd64:
$ ./malloc_test ; echo $?
pagesize: 4096
minalloc: 8192
malloc offset: 0
0
On non-mainstream hardware (Linux glibc/ia64)
$ ./malloc_test ; echo $?
pagesize: 16384
minalloc: 16384
malloc offset: 16
1
On Linux i386 with tcmalloc:
$ LD_PRELOAD=/usr/lib/libtcmalloc_minimal.so.0.0.0 ./malloc_test ;
echo $?
pagesize: 4096
minalloc: 8192
malloc offset: 0
0
But AFAIK tcmalloc is not a good general choice either, because it
will never give memory back to the OS.
Re: reducing memory fragmentation
Posted by Jim Jagielski <ji...@apache.org>.
On Feb 19, 2011, at 2:57 PM, Stefan Fritsch wrote:
>
> I am sure that apr_allocator with mmap is not an advantage on all
> platforms. Actually, apr allocating only multiples of 4k should make
> it easier for malloc to avoid fragmentation. But glibc's malloc fails
> miserably in that regard.
>
Is there a way during configuration time that we can do
some rough testing on "best" multiple to use...?
Re: reducing memory fragmentation
Posted by Stefan Fritsch <sf...@sfritsch.de>.
On Saturday 19 February 2011, Greg Stein wrote:
> On Fri, Feb 18, 2011 at 16:55, Stefan Fritsch <sf...@sfritsch.de>
wrote:
> > On Thursday 17 February 2011, Jim Jagielski wrote:
> >> Please also look at Greg's pocore memory allocation stuff...
> >> his hash stuff is also quite nice. Would be useful to be
> >> able to use that as well....
> >
> > That looks like a much bigger change than what I have just
> > commited. But I agree that before trying to optimize apr's
> > allocator, one should try pocore's.
> >
> > Have you thought about how to do this? Probably every apr pool
> > would be a wrapper for a pocore pool. But what about the other
> > users of apr_allocator, like bucket allocators?
>
> There is a branch in apr for wrapping pocore's pools and hash
> tables[1].
Nice, I didn't know about that.
> Obviously, the indirection slows it down, but it does
> demonstrate how it would work. (and it does: I've run the entire
> svn test suite using this wrapping)
Have you made any measurements how much the slow down is?
> My experiments show that mmap/munmap are NOT speed-advantageous on
> MacOS. But if you're looking at long-term memory usage and avoiding
> fragmentation... I don't have a good way to test that. That said,
> pocore should not be subject to fragmentation like apr. Its
> coalescing feature (designed w/ the APIs present, but not yet
> coded) will avoid much fragmentation.
I am sure that apr_allocator with mmap is not an advantage on all
platforms. Actually, apr allocating only multiples of 4k should make
it easier for malloc to avoid fragmentation. But glibc's malloc fails
miserably in that regard.
For the purpose of httpd giving unused memory back to the OS, your
current apr/pocore branch won't be an improvement because the bucket
allocators still use apr_allocator, which will hold on to some free
memory.
Re: reducing memory fragmentation
Posted by Greg Stein <gs...@gmail.com>.
On Fri, Feb 18, 2011 at 16:55, Stefan Fritsch <sf...@sfritsch.de> wrote:
> On Thursday 17 February 2011, Jim Jagielski wrote:
>> Please also look at Greg's pocore memory allocation stuff...
>> his hash stuff is also quite nice. Would be useful to be
>> able to use that as well....
>
> That looks like a much bigger change than what I have just commited.
> But I agree that before trying to optimize apr's allocator, one should
> try pocore's.
>
> Have you thought about how to do this? Probably every apr pool would
> be a wrapper for a pocore pool. But what about the other users of
> apr_allocator, like bucket allocators?
There is a branch in apr for wrapping pocore's pools and hash
tables[1]. Obviously, the indirection slows it down, but it does
demonstrate how it would work. (and it does: I've run the entire svn
test suite using this wrapping)
Just one comment: use pocore's child pools rather than its "post"
notion. I need to get rid of the posts and simply speed up child
pools.
The basic problem with the apr allocator is its granularity. The reuse
of memory is also a problem. pocore will store arbitrary-sized
segments and unused fragments and whatnot into a red-black tree to
quickly grab it and reuse it.
My experiments show that mmap/munmap are NOT speed-advantageous on
MacOS. But if you're looking at long-term memory usage and avoiding
fragmentation... I don't have a good way to test that. That said,
pocore should not be subject to fragmentation like apr. Its coalescing
feature (designed w/ the APIs present, but not yet coded) will avoid
much fragmentation.
Cheers,
-g
[1] http://svn.apache.org/repos/asf/apr/apr/branches/gstein-pocore/
... see memory/unix/apr_pools.c and tables/apr_hash.c
Re: reducing memory fragmentation
Posted by Stefan Fritsch <sf...@sfritsch.de>.
On Thursday 17 February 2011, Jim Jagielski wrote:
> Please also look at Greg's pocore memory allocation stuff...
> his hash stuff is also quite nice. Would be useful to be
> able to use that as well....
That looks like a much bigger change than what I have just commited.
But I agree that before trying to optimize apr's allocator, one should
try pocore's.
Have you thought about how to do this? Probably every apr pool would
be a wrapper for a pocore pool. But what about the other users of
apr_allocator, like bucket allocators?
Re: reducing memory fragmentation
Posted by Jim Jagielski <ji...@jaguNET.com>.
Please also look at Greg's pocore memory allocation stuff...
his hash stuff is also quite nice. Would be useful to be
able to use that as well....
On Feb 6, 2011, at 5:34 PM, Stefan Fritsch wrote:
> On Friday 04 February 2011, William A. Rowe Jr. wrote:
>>>> My thought is that this might not be the only thing to 'refresh'
>>>> when you have an app which needs current state. It would be a
>>>> chance to flush out all caches and set the state basically to
>>>> apr_initialize(), modulo any open resources or currently
>>>> allocated pools. It could even go so far as to free() all the
>>>> unused pool memory, giving us a really fresh start on heap
>>>> pages that have been fragmented to heck.
>>>
>>> ++1 for some way of handling the fragmentation... of course,
>>> we could just switch to pocore for the memory stuff in APR. :)
>>
>> Which helps for apr_allocator but not for the internal
>> fragmentation within individual pools. An optional pocore feature
>> for APR 2 sounds great, patches welcome :)
>
> The pools give their memory back to the allocator when they are
> cleared. Therefore at least the transaction pools and their children
> should not be a major problem for fragmentation.
>
> However, one major problem are the allocators not giving back memory,
> or if configured with max_mem_free, giving it back in random order so
> that it is very unlikely that the malloc implementation can give
> memory back to the OS.
>
> A second problem I have noticed with glibc: Its malloc allocates APR's
> typical multiple-of-4k chunks in a normal linked list. This means that
> there is some N bytes header allocated before each chunk. Now if APR
> has allocated say 16k, and frees it again, the hole is 16k+N. However
> this is not big enough to allocate two 8k chunks, which would need
> (8k+N)*2 == 16k+2N bytes. I think this may contribute to the
> fragmentation seen under Linux. In addition to that, glibc's malloc
> allocates smaller chunks in the same linked list, so in practice we
> have something like this:
>
> 8K pool chunk
> 8K pool chunk
> some bytes from a third party lib
> 12K pool chunk
> some more bytes from normal malloc
> 8K pool chunk
>
> This doesn't help with fragmentation either.
>
> Therefore I propose to use mmap/mumap in the apr_allocator instead of
> malloc, either as a compile time option or whenever max_mem_free is
> set for the allocator. This would then completely separate the
> allocations from apr_allocator and normal malloc and reduce
> fragmentation. And it would make apr_allocator_max_free_set actually
> give memory back to the OS.
>
> This would be a relatively simple change (compared to replacing the
> whole allocator). Sounds ok?
>