apr-2 pool status [Was: Poor performance with new apr_pool]

[Mladen Turk <mt...@apache.org>]

So, it's been proven that the new apr pool
doesn't perform well except on some obscure platforms
with third-party libraries.

Further more the new pool broke the API since Paul
decided he doesn't need the unmanaged pools ;)

Since there were absolutely nothing wrong with the old
code can we have it back, or at least compile time
selectable (like it isn't complex enough) ?

... and yes, can we get rid of the POOL_DEBUG.
It totally breaks the apr concept of a clean API,
and doubles the pool code without (at least to me)
any good reason.

Regards
-- 
^(TM)

Re: apr-2 pool status [Was: Poor performance with new apr_pool]

[Mladen Turk <mt...@apache.org>]

Sander Striker wrote:
>>
>> ... and yes, can we get rid of the POOL_DEBUG.
>> It totally breaks the apr concept of a clean API,
>> and doubles the pool code without (at least to me)
>> any good reason.
> 
> Darn, you almost had me there (April Fools!).  Just in case you were
> being serious, the pool debug code has certainly helped track down
> memory allocation related issues in several applications.  The [old]
> pools logic masks a lot, and is therefor not so useful in combination
> with tools like valgrind, duma, etc.
> 

If after all those years you still make bugs, what's the point man?

But seriously, the debug pool api is something
that simply doesn't fit in the apr. If it does, I certainly
know few places in the apr where debug api would help.
I volunteer to write the extra debug api for critical apr objects
like files, mutexes and sockets. Beside, I have nothing
else to do with my life. I mean I always wondered how many
times the EINTR is happening inside socket_read. That would be
kind a cool to know for any nerd out there.

Cheers
-- 
^(TM)

Re: apr-2 pool status [Was: Poor performance with new apr_pool]

[Sander Striker <s....@striker.nl>]

On Wed, Apr 1, 2009 at 10:03 AM, Mladen Turk <mt...@apache.org> wrote:
> So, it's been proven that the new apr pool
> doesn't perform well except on some obscure platforms
> with third-party libraries.
>
> Further more the new pool broke the API since Paul
> decided he doesn't need the unmanaged pools ;)
>
> Since there were absolutely nothing wrong with the old
> code can we have it back, or at least compile time
> selectable (like it isn't complex enough) ?
>
> ... and yes, can we get rid of the POOL_DEBUG.
> It totally breaks the apr concept of a clean API,
> and doubles the pool code without (at least to me)
> any good reason.

Darn, you almost had me there (April Fools!).  Just in case you were
being serious, the pool debug code has certainly helped track down
memory allocation related issues in several applications.  The [old]
pools logic masks a lot, and is therefor not so useful in combination
with tools like valgrind, duma, etc.

Cheers,

Sander

Re: apr-2 pool status [Was: Poor performance with new apr_pool]

[Mladen Turk <mt...@apache.org>]

Mladen Turk wrote:
> So, it's been proven that the new apr pool
> doesn't perform well except on some obscure platforms
> with third-party libraries.
> 

Just did some more profiling on the new apr-pool code
inside httpd. I've simply
#undef malloc
#undef calloc
and add size and number of call counters
(some memory profiler would be better, but
  this one is good enough for the subject)

The plain httpd -k start of httpd trunk
gives the following results in child_init hook

New apr pool:
APR_POOL STATS -- child init
     malloc calls: 14819
     free calls  : 7451
     malloc size : 1327972
     free size   : 784919
     pool create : 148
     pool clear  : 3
     pool destroy: 83
     pool size   : 4172
     palloc calls: 14642
     palloc size : 656700

1.4 apr pool:
APR_POOL STATS -- child init
     malloc calls: 129
     free calls  : 0
     malloc size : 1101928
     pool create : 148
     pool clear  : 3
     pool destroy: 83
     sizeof(pool): 64
     palloc calls: 14640
     palloc size : 698440 - aligned

1.4 apr pool with MIN_ALLOC == 2048
APR_POOL STATS -- child init
     malloc calls: 211
     free calls  : 0
     malloc size : 540776
     free size   : 0
     pool create : 148
     pool clear  : 3
     pool destroy: 83
     pool size   : 64
     palloc calls: 14640
     palloc size : 698440

This means that average apr_palloc/calloc is 45 bytes
(48 bytes in old apr because of alignment) just to
startup the server. If you involve the request processing
think the average size will be even lower.

So I doubt in any numbers showing the simple
pointer math is slower then a function call
even with tcmalloc, but ... ;)

Finally, what does those numbers mean?
Both implementations suck :)
For an total requested size of 640K we ended
with allocated 1M from the system using 1.4
apr pool.
Interestingly the new implementation is much better
regarding that. After settling it consumes less
memory, but it suffers from peek usage, where
the usage goes much higher until pool get cleared.

However if I change the MIN_ALLOC to 2048 and
adjust the BOUNDARY_INDEX to 10, the total
memory usage lowers to 512K.

Regards
-- 
^(TM)