You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Sander Striker <st...@apache.org> on 2004/08/01 19:46:14 UTC
RE: [PROOF-OF-CONCEPT?] logging memory used by an allocator
> From: Jeff Trawick [mailto:trawick@attglobal.net]
> Sent: Wednesday, July 28, 2004 1:33 PM
> To: dev@apr.apache.org; dev@httpd.apache.org
> Subject: [PROOF-OF-CONCEPT?] logging memory used by an allocator
>
> A couple of questions come up from an application perspective:
>
> am I leaking memory? if so, on what operation?
> how much memory does it take to perform a certain operation?
>
> If the application can find out how much heap memory is
> presently owned by a certain allocator, it can be easier to
> address such questions.
Wouldn't you want to know the memory currently being held by
the allocator as well as all the memory the allocator dished
out to it's pools?
> The attached apr.patch adds apr_allocator_memsize_get() to
> find the amount of heap memory presently owned by the
> allocator. (debug pool flavor not implemented; what is
> implemented isn't tested much ;) )
Given that memory management is on the critical path we need
to be careful what we add. But this patch seems pretty harmless
in that respect.
> The attached httpd.patch adds %Z log format to mod_log_config
> to log the memory size of the allocator used by the request
> pool. (I would lean towards implementing this feature in a
> debug module instead of in mod_log_config.)
I assume you implemented this because of an itch? :)
Sander
Fwd: [PROOF-OF-CONCEPT?] logging memory used by an allocator
Posted by Jeff Trawick <tr...@gmail.com>.
---------- Forwarded message ----------
From: Jeff Trawick <tr...@gmail.com>
Date: Tue, 3 Aug 2004 14:45:16 -0400
Subject: Re: [PROOF-OF-CONCEPT?] logging memory used by an allocator
To: Sander Striker <st...@apache.org>
On Sun, 1 Aug 2004 19:46:14 +0200, Sander Striker <st...@apache.org> wrote:
> > From: Jeff Trawick [mailto:trawick@attglobal.net]
> > Sent: Wednesday, July 28, 2004 1:33 PM
> > To: dev@apr.apache.org; dev@httpd.apache.org
> > Subject: [PROOF-OF-CONCEPT?] logging memory used by an allocator
> >
> > A couple of questions come up from an application perspective:
> >
> > am I leaking memory? if so, on what operation?
> > how much memory does it take to perform a certain operation?
> >
> > If the application can find out how much heap memory is
> > presently owned by a certain allocator, it can be easier to
> > address such questions.
>
> Wouldn't you want to know the memory currently being held by
> the allocator as well as all the memory the allocator dished
> out to it's pools?
I'm sure that could be useful to some folks; I can't think of a use at
the present with Apache unless some logging of the allocator shows
that some request uses a bunch of memory, and we know that there are
various pools in use on that request so we add some temporary debug
code to add pool granularity to the measurement
> > The attached apr.patch adds apr_allocator_memsize_get() to
> > find the amount of heap memory presently owned by the
> > allocator. (debug pool flavor not implemented; what is
> > implemented isn't tested much ;) )
>
> Given that memory management is on the critical path we need
> to be careful what we add. But this patch seems pretty harmless
> in that respect.
but doesn't "memsize_get" suck as a name? any better ideas?
> > The attached httpd.patch adds %Z log format to mod_log_config
> > to log the memory size of the allocator used by the request
> > pool. (I would lean towards implementing this feature in a
> > debug module instead of in mod_log_config.)
>
> I assume you implemented this because of an itch? :)
biggest itch was, in the context of a suspected storage leak, wanting
to draw a line between Apache memory use and some potential
third-party module or library issue; hard without some such
measurements to even understand the normal Apache storage growth in a
threaded server as more and more threads in the process eventually
handle expensive requests
more specifically:
good for a *rough* idea of memory requirements; I haven't a clue how
much heap memory it takes to process some request; 10K? 100K?
good for determining when it is useful to use MaxMemFree by
identifying situations where an infrequent request takes a lot more
pool memory than normal to process; in such case, it is practical to
set MaxMemFree to approximately the "normal" amount of memory
required, since malloc()/free() overhead won't be a killer
Fwd: [PROOF-OF-CONCEPT?] logging memory used by an allocator
Posted by Jeff Trawick <tr...@gmail.com>.
---------- Forwarded message ----------
From: Jeff Trawick <tr...@gmail.com>
Date: Tue, 3 Aug 2004 14:45:16 -0400
Subject: Re: [PROOF-OF-CONCEPT?] logging memory used by an allocator
To: Sander Striker <st...@apache.org>
On Sun, 1 Aug 2004 19:46:14 +0200, Sander Striker <st...@apache.org> wrote:
> > From: Jeff Trawick [mailto:trawick@attglobal.net]
> > Sent: Wednesday, July 28, 2004 1:33 PM
> > To: dev@apr.apache.org; dev@httpd.apache.org
> > Subject: [PROOF-OF-CONCEPT?] logging memory used by an allocator
> >
> > A couple of questions come up from an application perspective:
> >
> > am I leaking memory? if so, on what operation?
> > how much memory does it take to perform a certain operation?
> >
> > If the application can find out how much heap memory is
> > presently owned by a certain allocator, it can be easier to
> > address such questions.
>
> Wouldn't you want to know the memory currently being held by
> the allocator as well as all the memory the allocator dished
> out to it's pools?
I'm sure that could be useful to some folks; I can't think of a use at
the present with Apache unless some logging of the allocator shows
that some request uses a bunch of memory, and we know that there are
various pools in use on that request so we add some temporary debug
code to add pool granularity to the measurement
> > The attached apr.patch adds apr_allocator_memsize_get() to
> > find the amount of heap memory presently owned by the
> > allocator. (debug pool flavor not implemented; what is
> > implemented isn't tested much ;) )
>
> Given that memory management is on the critical path we need
> to be careful what we add. But this patch seems pretty harmless
> in that respect.
but doesn't "memsize_get" suck as a name? any better ideas?
> > The attached httpd.patch adds %Z log format to mod_log_config
> > to log the memory size of the allocator used by the request
> > pool. (I would lean towards implementing this feature in a
> > debug module instead of in mod_log_config.)
>
> I assume you implemented this because of an itch? :)
biggest itch was, in the context of a suspected storage leak, wanting
to draw a line between Apache memory use and some potential
third-party module or library issue; hard without some such
measurements to even understand the normal Apache storage growth in a
threaded server as more and more threads in the process eventually
handle expensive requests
more specifically:
good for a *rough* idea of memory requirements; I haven't a clue how
much heap memory it takes to process some request; 10K? 100K?
good for determining when it is useful to use MaxMemFree by
identifying situations where an infrequent request takes a lot more
pool memory than normal to process; in such case, it is practical to
set MaxMemFree to approximately the "normal" amount of memory
required, since malloc()/free() overhead won't be a killer