Question about malloc / realloc in module

[Christoph Gröver <gr...@sitepark.com>]

Hello list,

In a module I have to allocate and reallocate a chunk of memory.
Because (AFAIK) Apache or its libraries do not support reallocation
I use the standard functions malloc/realloc (and free), of course.

But what if there's a problem in another module?

Is it possible that due to some errors somewhere else the
malloc/realloc of my modules are run (and therefor memory is allocated)
but the corresponding free is not called, because the thread/process
had a strange ending.

So would this result in a memory leak? (if this happens often!)

Hope you can enlight me on this topic.

Bye,

-- 
Christoph Gröver, grover@sitepark.com

Re: Question about malloc / realloc in module

[Eric Covener <co...@gmail.com>]

> But what if a process exists in a faulty way. Perhaps some PHP code
> that exists abnormally?

If the process exits, you've got no worries about memory.  If a thread
hangs [indefinitely], before you can free memory, the memory is
probably the least of your problems.

Re: Question about malloc / realloc in module

[Christoph Gröver <gr...@sitepark.com>]

Hi Joshua,

Thank you for your answer.

> If you malloc/calloc/realloc without a free you will leak memory.
> 
Well, I use free(). I have implemented counters which prove that for
every malloc there's a free() called with normal operations.

But what if a process exists in a faulty way. Perhaps some PHP code
that exists abnormally?

> Do you have some reason to believe that another module might prevent
> your module's calls to free() from being run?  What do you have in
> mind specifically?

I have a system in which I encounter an Apache with a rapidly
growing memory consumption. Memory usage grows in a linear way.

Though I can see that in normal operation (on a test system) I have a
free() called for every malloc(), I can't debug this on  a production
system with much more load going on.

> 
> You can also mimic realloc by just allocating more memory from the
> pool and calling memcpy.  The old memory will leak into the pool but
> be resolved on pool cleanup.
> 

Perhaps I will try this, if I have the feeling that memory leaks
are the cause for the growing memory usage.

Thank you for your suggestions.

Greetings
-- 
Christoph Gröver, grover@sitepark.com
Sitepark Gesellschaft für Informationsmanagement mbH, AG Münster, HRB
5017 Rothenburg 14-16, D-48143 Münster, Telefon (0251) 48265-50
Geschäftsführer: Dipl.-Phys. Martin Kurze, Dipl.-Des. Thorsten Liebold

Re: Question about malloc / realloc in module

[Joshua Marantz <jm...@google.com>]

If you malloc/calloc/realloc without a free you will leak memory.

Do you have some reason to believe that another module might prevent your
module's calls to free() from being run?  What do you have in mind
specifically?

You can also mimic realloc by just allocating more memory from the pool and
calling memcpy.  The old memory will leak into the pool but be resolved on
pool cleanup.


I can tell you that malloc/free works fine inside an Apache module:
mod_pagespeed is written in C++ and uses new/delete extensively with the
default underlying malloc/free implementation.  We load-test extensively and
watch for leaks carefully via top, in addition to running valgrind on small
system tests.  This methodology is sound.  I'm not sure why you think your
calls to free would not happen.

-Josh

On Wed, Sep 14, 2011 at 8:01 AM, Christoph Gröver <gr...@sitepark.com>wrote:

>
> Hello list,
>
> In a module I have to allocate and reallocate a chunk of memory.
> Because (AFAIK) Apache or its libraries do not support reallocation
> I use the standard functions malloc/realloc (and free), of course.
>
> But what if there's a problem in another module?
>
> Is it possible that due to some errors somewhere else the
> malloc/realloc of my modules are run (and therefor memory is allocated)
> but the corresponding free is not called, because the thread/process
> had a strange ending.
>
> So would this result in a memory leak? (if this happens often!)
>
> Hope you can enlight me on this topic.
>
> Bye,
>
> --
> Christoph Gröver, grover@sitepark.com
>
>

Re: Question about malloc / realloc in module

[Ben Noordhuis <in...@bnoordhuis.nl>]

On Wed, Sep 14, 2011 at 14:01, Christoph Gröver <gr...@sitepark.com> wrote:
> In a module I have to allocate and reallocate a chunk of memory.
> Because (AFAIK) Apache or its libraries do not support reallocation
> I use the standard functions malloc/realloc (and free), of course.

Don't do that. Use apr_palloc() and copy over the data.

If you're worried about memory wastage, create a child pool and
apr_palloc() from there. apr_pool_destroy() the pool to reclaim the
memory.

Re: Question about malloc / realloc in module

[Christoph Gröver <gr...@sitepark.com>]

Thank you, Nick, for your answer,

> > So would this result in a memory leak? (if this happens often!)
> 
> Potentially yes, unless you can do an exhaustive analysis of all
> possible processing paths.
> 
> The fix for that is to register a free as a pool cleanup
> immediately after the malloc/realloc.  See mod_proxy_html
> for an example.

Thanks for this suggestion. If I do not find out what is causing the
apache childs to grow in memory usage, I will try this to be sure
it's not my memory allocations.

Greetings.

-- 
Christoph Gröver, grover@sitepark.com

Re: Question about malloc / realloc in module

[Nick Kew <ni...@apache.org>]

On Wed, 14 Sep 2011 14:01:19 +0200
Christoph Gröver <gr...@sitepark.com> wrote:

> So would this result in a memory leak? (if this happens often!)

Potentially yes, unless you can do an exhaustive analysis of all
possible processing paths.

The fix for that is to register a free as a pool cleanup
immediately after the malloc/realloc.  See mod_proxy_html
for an example.

-- 
Nick Kew