Problem avoiding memory leak with thread creation

[Chris Fallin <ch...@cfallin.org>]

Hello everyone,

I'm developing a small server application using APR. As part of the 
design each new session spawns a thread. The application will be 
long-running and so must not leak memory.

In order to prevent leaks each thread gets its own subpool, which it 
frees on exit. However, I see in apr_thread_create (at 
threadproc/unix/thread.c:160) that a subpool is allocated from the pool 
passed to the function. This pool is the one that is destroyed in 
apr_thread_exit at line 192. Thus, the pool passed in to 
apr_thread_create is never freed unless done manually after 
apr_thread_exit destroys the child pool, which is hard because it exits 
the thread.

The fact that leads me to believe it may be a bug is that on line 151 
the 'pool' member is assigned to the pool that's passed in. Or maybe 
it's a design decision - I don't know. Nevertheless, what is the 
recommended way to avoid a memory leak in this situation?

Thanks in advance
-- 
Chris Fallin
Email: chris@cfallin.org
AIM  : ProgrammerNerd1
URL  : http://www.cfallin.org/

Re: Problem avoiding memory leak with thread creation

[Cliff Woolley <jw...@virginia.edu>]

On Thu, 4 Sep 2003, Chris Fallin wrote:

> In order to prevent leaks each thread gets its own subpool, which it
> frees on exit. However, I see in apr_thread_create (at
> threadproc/unix/thread.c:160) that a subpool is allocated from the pool
> passed to the function. This pool is the one that is destroyed in
> apr_thread_exit at line 192. Thus, the pool passed in to
> apr_thread_create is never freed unless done manually after
> apr_thread_exit destroys the child pool, which is hard because it exits
> the thread.

As for whether what you described is a bug or not, I'm not sure, but you
might be able to get around it one way or the other by calling
apr_allocator_create() yourself, then apr_pool_create_ex() -- that way
your subpool has its own block of memory that you can destroy after the
thread is done, rather than having it go back into the parent pool's
allocator.  (apr_allocator_max_free_set() might also be useful.)

Hope this helps.  No doubt one of the pool wizards will jump in with other
suggestions...

--Cliff

Re: Problem avoiding memory leak with thread creation

[Chris Fallin <ch...@cfallin.org>]

Hi Matt,

Thanks - it appears that's the best solution. I have a separate thread 
now that receives requests from the connection threads to join them and 
then free their pool - problem solved.

Thanks for your help,
Chris

Matt Carter wrote:
> Hi Chris,
> 
> Hope I can be of some help. It appears to me that your leak is caused by
> where you are trying to assign responsibility for freeing the thread
> (i.e. the subpool in which you allocated the thread).
> 
> The thread's pool must be allocated and freed outside of the lifetime
> of the thread object (This means it cannot be freed from in the thread
> itself).
> 
> It is your responsibility to free the subpool which you pass into
> apr_thread_create() _after_ the thread has exited. The only way to be
> sure a thread has actually exited is to call apr_thread_join() from the
> main thread, which causes it to wait until the thread has exited. The
> main thread can then free the subpool used for that thread.
> 
> (People usually call apr_thread_join() on threads before program
> shutdown to wait for all the thread to terminate cleanly, before shared
> resources are cleaned up and the program exits.)
> 
> You need a mechanism for signalling from a connection thread to the main
> thread to tell the main thread a connection thread has finished so that
> it can clean up what it allocated (your subpool).
> 
> This should be quite easy, since you should also have a list of all
> active threads and their corresponding pools held in your main thread so
> that you can join() them all on shutdown.
> 
> The thread scheme in apr is very much like pthreads, for which
> documentation and examples are abundant, you may wish to search the web
> where you should find some good examples of multi-threaded servers,
> and where join comes into play..
> 
> Regards,
> 
> Matt
> 
> 
> On Fri, 2003-09-05 at 05:27, Chris Fallin wrote:
> 
>>Hello everyone,
>>
>>I'm developing a small server application using APR. As part of the 
>>design each new session spawns a thread. The application will be 
>>long-running and so must not leak memory.
>>
>>In order to prevent leaks each thread gets its own subpool, which it 
>>frees on exit. However, I see in apr_thread_create (at 
>>threadproc/unix/thread.c:160) that a subpool is allocated from the pool 
>>passed to the function. This pool is the one that is destroyed in 
>>apr_thread_exit at line 192. Thus, the pool passed in to 
>>apr_thread_create is never freed unless done manually after 
>>apr_thread_exit destroys the child pool, which is hard because it exits 
>>the thread.
>>
>>The fact that leads me to believe it may be a bug is that on line 151 
>>the 'pool' member is assigned to the pool that's passed in. Or maybe 
>>it's a design decision - I don't know. Nevertheless, what is the 
>>recommended way to avoid a memory leak in this situation?
>>
>>Thanks in advance
> 
> 

-- 
Chris Fallin
Email: chris@cfallin.org
AIM  : ProgrammerNerd1
URL  : http://www.cfallin.org/

Re: Problem avoiding memory leak with thread creation

[Matt Carter <ma...@pro.ukshells.co.uk>]

Hi Chris,

Hope I can be of some help. It appears to me that your leak is caused by
where you are trying to assign responsibility for freeing the thread
(i.e. the subpool in which you allocated the thread).

The thread's pool must be allocated and freed outside of the lifetime
of the thread object (This means it cannot be freed from in the thread
itself).

It is your responsibility to free the subpool which you pass into
apr_thread_create() _after_ the thread has exited. The only way to be
sure a thread has actually exited is to call apr_thread_join() from the
main thread, which causes it to wait until the thread has exited. The
main thread can then free the subpool used for that thread.

(People usually call apr_thread_join() on threads before program
shutdown to wait for all the thread to terminate cleanly, before shared
resources are cleaned up and the program exits.)

You need a mechanism for signalling from a connection thread to the main
thread to tell the main thread a connection thread has finished so that
it can clean up what it allocated (your subpool).

This should be quite easy, since you should also have a list of all
active threads and their corresponding pools held in your main thread so
that you can join() them all on shutdown.

The thread scheme in apr is very much like pthreads, for which
documentation and examples are abundant, you may wish to search the web
where you should find some good examples of multi-threaded servers,
and where join comes into play..

Regards,

Matt


On Fri, 2003-09-05 at 05:27, Chris Fallin wrote:
> Hello everyone,
> 
> I'm developing a small server application using APR. As part of the 
> design each new session spawns a thread. The application will be 
> long-running and so must not leak memory.
> 
> In order to prevent leaks each thread gets its own subpool, which it 
> frees on exit. However, I see in apr_thread_create (at 
> threadproc/unix/thread.c:160) that a subpool is allocated from the pool 
> passed to the function. This pool is the one that is destroyed in 
> apr_thread_exit at line 192. Thus, the pool passed in to 
> apr_thread_create is never freed unless done manually after 
> apr_thread_exit destroys the child pool, which is hard because it exits 
> the thread.
> 
> The fact that leads me to believe it may be a bug is that on line 151 
> the 'pool' member is assigned to the pool that's passed in. Or maybe 
> it's a design decision - I don't know. Nevertheless, what is the 
> recommended way to avoid a memory leak in this situation?
> 
> Thanks in advance