Re: Terminiting threads in a process RE: [PATCH] Problems with MPM threaded

[rb...@covalent.net]

On Sat, 14 Jul 2001, Sander Striker wrote:

> >> By having the possibility of having other children processes, you now
> >> need a mechanism to kill all threads in the same process efficiently.
> >> You'd need to kick them out of the accept mutex, but I'm not seeing
> >> how that's going to happen quickly.  -- justin
> >
> > You need to be able to kill all threads in a process efficiently
> > regardless.  We will always need to be able to re-spawn processes that
> > die, and we will always have the concept of MaxRequestsPerChild.  If a
> > process is going to die and be replaced, then we need to kill all the
> > threads in that process as quickly as possible.  Only allowing one
> > process at a time doesn't remove that requirement.
>
> The way I see it, each process has a single pool instance as the parent
> for all the threads. Resetting or destroying that pool should effectively
> kill all threads. What am I missing?

Nothing.

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

Thread pools are isolated WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Justin Erenkrantz <je...@ebuilt.com>]

> Ah, ok, I'm getting the picture now. Well, for my purposes, the child
> pool creation doesn't have to be in apr_thread_create. There doesn't
> have to be child pool creation at all. But, I do want the worker fn
> to be able to create a child pool from the pool passed into
> apr_thread_create. So, we need a way to pass in the data to the
> worker fn.

No.  I thought we were going to per-thread pools.  And, this is why we 
were going this way.  This removes all of the notions of cleanups from 
a parent trying to forcibly cleanup the thread.  IMHO, there should be
no relationship between the parent SMS and its threads' SMS.  -- justin

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

> > I have some use cases that just want a thread. Since no child-pool is
> > required in this case, it becomes unnecessary overhead (that is currently
> > broken, as the child-pool is only cleaned in apr_thread_exit() but that
> > whole thing is screwey).
> 
> Again, that is incorrect.  The THREAD-pool is cleaned whenever the parent
> pool is cleaned.  Yes, there is some extra overhead, but remember that the
> problem with pools (as the originally existed, and are still implemented
> in the server), is that you can't have two threads working with the same
> pool.

You are correct, I failed to mention that it is not *only* cleaned in
apr_thread_exit(), but also when the parent pool is cleaned. It is,
however, inappropriate to assume that the parent pool will ever get
cleaned.

Right now, if my application were to have a global/root pool that
repeatedly created short-lived threads (as it does, BTW), it would
eventually run out of memory. So, like I said, if apr_thread_exit()
is not called (and in all of apr, apr-util, and httpd is is never
called, not even in the test* programs), then that application is wholly
dependent on the ability of the thread's parent pool to clean up the
memory.


> IF SMS's replace all pools, then that might change.  As things stand
> today though, that hasn't happened, and may not happen quickly.
> 
> > So if apr threads didn't know anything about child-pools, then they
> > would just simply create a thread that called my worker_function().
> > I could pass in my opaque data to that worker_function(), and do
> > the child-pool creations and destruction right there. I just don't see
> > how child-pool creation MUST be in apr_thread_create()
> 
> Yes, you could do that, but then EVERYBODY who wanted to create a thread
> would have to do that, and why are we forcing people to duplicate that
> code everytime a thread is created?

I agree that this is a useful function *at some level*, and so I propose
that this particular utility be built on top of a lower-level set of APR
thread routines. I will even write these routines myself, just because
I believe they are useful.

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

> > I have some use cases that just want a thread. Since no child-pool is
> > required in this case, it becomes unnecessary overhead (that is currently
> > broken, as the child-pool is only cleaned in apr_thread_exit() but that
> > whole thing is screwey).
> 
> Again, that is incorrect.  The THREAD-pool is cleaned whenever the parent
> pool is cleaned.  Yes, there is some extra overhead, but remember that the
> problem with pools (as the originally existed, and are still implemented
> in the server), is that you can't have two threads working with the same
> pool.

You are correct, I failed to mention that it is not *only* cleaned in
apr_thread_exit(), but also when the parent pool is cleaned. It is,
however, inappropriate to assume that the parent pool will ever get
cleaned.

Right now, if my application were to have a global/root pool that
repeatedly created short-lived threads (as it does, BTW), it would
eventually run out of memory. So, like I said, if apr_thread_exit()
is not called (and in all of apr, apr-util, and httpd is is never
called, not even in the test* programs), then that application is wholly
dependent on the ability of the thread's parent pool to clean up the
memory.


> IF SMS's replace all pools, then that might change.  As things stand
> today though, that hasn't happened, and may not happen quickly.
> 
> > So if apr threads didn't know anything about child-pools, then they
> > would just simply create a thread that called my worker_function().
> > I could pass in my opaque data to that worker_function(), and do
> > the child-pool creations and destruction right there. I just don't see
> > how child-pool creation MUST be in apr_thread_create()
> 
> Yes, you could do that, but then EVERYBODY who wanted to create a thread
> would have to do that, and why are we forcing people to duplicate that
> code everytime a thread is created?

I agree that this is a useful function *at some level*, and so I propose
that this particular utility be built on top of a lower-level set of APR
thread routines. I will even write these routines myself, just because
I believe they are useful.

-aaron

RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Sander Striker <st...@apache.org>]

> > Why are we so desperate in opting out the child-pool creation?
> > I don't really have problems with a child pool for each thread. 
> Actually,
> > it will make the dynamic locking a lot easier to implement if it stays.
> 
> I have some use cases that just want a thread. Since no child-pool is
> required in this case, it becomes unnecessary overhead (that is currently
> broken, as the child-pool is only cleaned in apr_thread_exit() but that
> whole thing is screwey).
> 
> So if apr threads didn't know anything about child-pools, then they
> would just simply create a thread that called my worker_function().
> I could pass in my opaque data to that worker_function(), and do
> the child-pool creations and destruction right there. I just don't see
> how child-pool creation MUST be in apr_thread_create()

Ah, ok, I'm getting the picture now. Well, for my purposes, the child
pool creation doesn't have to be in apr_thread_create. There doesn't
have to be child pool creation at all. But, I do want the worker fn
to be able to create a child pool from the pool passed into
apr_thread_create. So, we need a way to pass in the data to the
worker fn.

The only thing I need is something like this for the sms locking scheme:

struct thread_data
{
    ...
    apr_sms_t *sms;
    void     (*worker_fn)(void *data);
    void      *data;
};

void thread_stub(void *data)
{
    apr_sms_t *pms;
    apr_os_thread_t thread;
    struct thread_data *td = (struct thread_data *)data;

    thread = apr_os_thread_current();

    pms = td->sms;
    apr_sms_thread_register(pms, thread);
    
    td->worker_fn(td->data);

    apr_sms_thread_unregister(pms, thread);
}
 
In apr_thread_create an instance of thread_data is created
and initialized, then instead of running the worker function
in the thread, we launch the stub which in turn calls the
worker.

Possibly we can use the stub for more than only this?

Sander

RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Sander Striker <st...@apache.org>]

> > Why are we so desperate in opting out the child-pool creation?
> > I don't really have problems with a child pool for each thread. 
> Actually,
> > it will make the dynamic locking a lot easier to implement if it stays.
> 
> I have some use cases that just want a thread. Since no child-pool is
> required in this case, it becomes unnecessary overhead (that is currently
> broken, as the child-pool is only cleaned in apr_thread_exit() but that
> whole thing is screwey).
> 
> So if apr threads didn't know anything about child-pools, then they
> would just simply create a thread that called my worker_function().
> I could pass in my opaque data to that worker_function(), and do
> the child-pool creations and destruction right there. I just don't see
> how child-pool creation MUST be in apr_thread_create()

Ah, ok, I'm getting the picture now. Well, for my purposes, the child
pool creation doesn't have to be in apr_thread_create. There doesn't
have to be child pool creation at all. But, I do want the worker fn
to be able to create a child pool from the pool passed into
apr_thread_create. So, we need a way to pass in the data to the
worker fn.

The only thing I need is something like this for the sms locking scheme:

struct thread_data
{
    ...
    apr_sms_t *sms;
    void     (*worker_fn)(void *data);
    void      *data;
};

void thread_stub(void *data)
{
    apr_sms_t *pms;
    apr_os_thread_t thread;
    struct thread_data *td = (struct thread_data *)data;

    thread = apr_os_thread_current();

    pms = td->sms;
    apr_sms_thread_register(pms, thread);
    
    td->worker_fn(td->data);

    apr_sms_thread_unregister(pms, thread);
}
 
In apr_thread_create an instance of thread_data is created
and initialized, then instead of running the worker function
in the thread, we launch the stub which in turn calls the
worker.

Possibly we can use the stub for more than only this?

Sander

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[rb...@covalent.net]

On Sun, 15 Jul 2001, Aaron Bannert wrote:

> On Sun, Jul 15, 2001 at 07:16:35PM +0200, Sander Striker wrote:
> > > Fair enough. It's just that in order to opt-out of the child-pool creating
> > > process in apr_thread_create, we're going to have to add a parameter
> >
> > Why are we so desperate in opting out the child-pool creation?
> > I don't really have problems with a child pool for each thread. Actually,
> > it will make the dynamic locking a lot easier to implement if it stays.
>
> I have some use cases that just want a thread. Since no child-pool is
> required in this case, it becomes unnecessary overhead (that is currently
> broken, as the child-pool is only cleaned in apr_thread_exit() but that
> whole thing is screwey).

Again, that is incorrect.  The THREAD-pool is cleaned whenever the parent
pool is cleaned.  Yes, there is some extra overhead, but remember that the
problem with pools (as the originally existed, and are still implemented
in the server), is that you can't have two threads working with the same
pool.

IF SMS's replace all pools, then that might change.  As things stand
today though, that hasn't happened, and may not happen quickly.

> So if apr threads didn't know anything about child-pools, then they
> would just simply create a thread that called my worker_function().
> I could pass in my opaque data to that worker_function(), and do
> the child-pool creations and destruction right there. I just don't see
> how child-pool creation MUST be in apr_thread_create()

Yes, you could do that, but then EVERYBODY who wanted to create a thread
would have to do that, and why are we forcing people to duplicate that
code everytime a thread is created?

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[rb...@covalent.net]

On Sun, 15 Jul 2001, Aaron Bannert wrote:

> On Sun, Jul 15, 2001 at 07:16:35PM +0200, Sander Striker wrote:
> > > Fair enough. It's just that in order to opt-out of the child-pool creating
> > > process in apr_thread_create, we're going to have to add a parameter
> >
> > Why are we so desperate in opting out the child-pool creation?
> > I don't really have problems with a child pool for each thread. Actually,
> > it will make the dynamic locking a lot easier to implement if it stays.
>
> I have some use cases that just want a thread. Since no child-pool is
> required in this case, it becomes unnecessary overhead (that is currently
> broken, as the child-pool is only cleaned in apr_thread_exit() but that
> whole thing is screwey).

Again, that is incorrect.  The THREAD-pool is cleaned whenever the parent
pool is cleaned.  Yes, there is some extra overhead, but remember that the
problem with pools (as the originally existed, and are still implemented
in the server), is that you can't have two threads working with the same
pool.

IF SMS's replace all pools, then that might change.  As things stand
today though, that hasn't happened, and may not happen quickly.

> So if apr threads didn't know anything about child-pools, then they
> would just simply create a thread that called my worker_function().
> I could pass in my opaque data to that worker_function(), and do
> the child-pool creations and destruction right there. I just don't see
> how child-pool creation MUST be in apr_thread_create()

Yes, you could do that, but then EVERYBODY who wanted to create a thread
would have to do that, and why are we forcing people to duplicate that
code everytime a thread is created?

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

On Sun, Jul 15, 2001 at 07:16:35PM +0200, Sander Striker wrote:
> > Fair enough. It's just that in order to opt-out of the child-pool creating
> > process in apr_thread_create, we're going to have to add a parameter
> 
> Why are we so desperate in opting out the child-pool creation?
> I don't really have problems with a child pool for each thread. Actually,
> it will make the dynamic locking a lot easier to implement if it stays.

I have some use cases that just want a thread. Since no child-pool is
required in this case, it becomes unnecessary overhead (that is currently
broken, as the child-pool is only cleaned in apr_thread_exit() but that
whole thing is screwey).

So if apr threads didn't know anything about child-pools, then they
would just simply create a thread that called my worker_function().
I could pass in my opaque data to that worker_function(), and do
the child-pool creations and destruction right there. I just don't see
how child-pool creation MUST be in apr_thread_create()

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

On Sun, Jul 15, 2001 at 07:16:35PM +0200, Sander Striker wrote:
> > Fair enough. It's just that in order to opt-out of the child-pool creating
> > process in apr_thread_create, we're going to have to add a parameter
> 
> Why are we so desperate in opting out the child-pool creation?
> I don't really have problems with a child pool for each thread. Actually,
> it will make the dynamic locking a lot easier to implement if it stays.

I have some use cases that just want a thread. Since no child-pool is
required in this case, it becomes unnecessary overhead (that is currently
broken, as the child-pool is only cleaned in apr_thread_exit() but that
whole thing is screwey).

So if apr threads didn't know anything about child-pools, then they
would just simply create a thread that called my worker_function().
I could pass in my opaque data to that worker_function(), and do
the child-pool creations and destruction right there. I just don't see
how child-pool creation MUST be in apr_thread_create()

-aaron

RE: DCEthreads (was Re: Pools in threads)

[Sander Striker <st...@apache.org>]

>>> And, it would add 3.5 million lines of code to APR.  
>> 
>> Nah. It would add the lines to add thread cancellation.  The
>> 3.5 million LOC is the _entire_ dce rpc codebase. Only part
>> of that is the dce threads library. And, I believe there are
>> some old dce/rpc team members in our midst who could know how
>> this all fit together. I could be wrong though.
> 
> [warning: tangent -- i changed the subject appropriately]
> 
> I must admit that it sounds tantalizing, but I have a comment and a
> question:
> 
> - It appears that the Free DCE package quoted above is under the GPL
>   and is therefore incompatible with the Apache License.

Yes.  However, I wasn't suggesting using or copying code from the
free dce package.  You can merely examine it and decide whether
implementing thread cancellation in such a way is something we want
in APR.  Then, you write your own implementation.

Furthermore, I would like to compare the Free DCE threads implementation
with the DCE/RPC threads implementation.  Then pick the best design
decisions from both.

> - What is meant by "*emulated* on top of POSIX threads"? What are the
>   tradeoffs, if any, compared to a implementation that is native to
>   the OS?

Sander

DCEthreads (was Re: Pools in threads)

[Aaron Bannert <aa...@ebuilt.com>]

On Mon, Jul 16, 2001 at 09:05:40AM +0200, Sander Striker wrote:
> > On Mon, Jul 16, 2001 at 01:54:14AM +0200, Luke Kenneth Casson 
> > Leighton wrote:
> > > ah, yes, but it _is_ supported by DCEthreads - see
> > > http://sourceforge.net/projects/freedce which provides,
> > > horror-of-horrors, thread cancellation *emulated* on
> > > top of POSIX threads.
> > [snip, snip]
> > > ... but what i am basically saying is, on NT and Unix,
> > > thread cancellation _is_ possible, it's just that i
> > > think you might not be too happy about the coding-route
> > > you might have to take to _do_ it :) :)
> > 
> > Possible, but not robust.  
> 
> DCE/RPC is _very_ robust. That being the case its underlying
> mechanisms are very likely to be robust too.
> 
> > And, it would add 3.5 million lines of code to APR.  
> 
> Nah. It would add the lines to add thread cancellation.  The
> 3.5 million LOC is the _entire_ dce rpc codebase. Only part
> of that is the dce threads library. And, I believe there are
> some old dce/rpc team members in our midst who could know how
> this all fit together. I could be wrong though.

[warning: tangent -- i changed the subject appropriately]

I must admit that it sounds tantalizing, but I have a comment and a
question:

- It appears that the Free DCE package quoted above is under the GPL
  and is therefore incompatible with the Apache License.

- What is meant by "*emulated* on top of POSIX threads"? What are the
  tradeoffs, if any, compared to a implementation that is native to
  the OS?

-aaron

Re: Pools in threads

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

> Nah. It would add the lines to add thread cancellation.  The
> 3.5 million LOC is the _entire_ dce rpc codebase. 

which includes about 15 separate services such as DCE/DFS,
which is actually AFS [without the bugfixes and with a
new RPC mechanism]

which you don't need, as sander says.

> Only part
> of that is the dce threads library. And, I believe there are
> some old dce/rpc team members in our midst who could know how
> this all fit together. I could be wrong though.

i'll ask.  might get a response tomorrow.

luke

RE: Pools in threads

[Sander Striker <st...@apache.org>]

> On Mon, Jul 16, 2001 at 01:54:14AM +0200, Luke Kenneth Casson 
> Leighton wrote:
> > ah, yes, but it _is_ supported by DCEthreads - see
> > http://sourceforge.net/projects/freedce which provides,
> > horror-of-horrors, thread cancellation *emulated* on
> > top of POSIX threads.
> [snip, snip]
> > ... but what i am basically saying is, on NT and Unix,
> > thread cancellation _is_ possible, it's just that i
> > think you might not be too happy about the coding-route
> > you might have to take to _do_ it :) :)
> 
> Possible, but not robust.  

DCE/RPC is _very_ robust. That being the case its underlying
mechanisms are very likely to be robust too.

> And, it would add 3.5 million lines of code to APR.  

Nah. It would add the lines to add thread cancellation.  The
3.5 million LOC is the _entire_ dce rpc codebase. Only part
of that is the dce threads library. And, I believe there are
some old dce/rpc team members in our midst who could know how
this all fit together. I could be wrong though.

Sander

Re: Pools in threads

[Justin Erenkrantz <je...@ebuilt.com>]

On Mon, Jul 16, 2001 at 01:54:14AM +0200, Luke Kenneth Casson Leighton wrote:
> ah, yes, but it _is_ supported by DCEthreads - see
> http://sourceforge.net/projects/freedce which provides,
> horror-of-horrors, thread cancellation *emulated* on
> top of POSIX threads.
[snip, snip]
> ... but what i am basically saying is, on NT and Unix,
> thread cancellation _is_ possible, it's just that i
> think you might not be too happy about the coding-route
> you might have to take to _do_ it :) :)

Possible, but not robust.  And, it would add 3.5 million lines 
of code to APR.  Well, we could the APROS in that many LOC.  =)
Kind of like Quake - have an OS inside of your web-server.

Thread cancellation just opens up a can of worms not because the OS
doesn't support them, but because their semantics are ill-defined.
We'd have to add cancellation points to our code.  Not going to
happen.  Well, I'm not going to do it.  I might even veto it out 
of pure spite.  =)  -- justin

Re: Pools in threads

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

On Sun, Jul 15, 2001 at 04:43:37PM -0700, Justin Erenkrantz wrote:
> On Mon, Jul 16, 2001 at 12:46:31AM +0200, Luke Kenneth Casson Leighton wrote:
> > > In order to provide a win against the current pool code in a threaded
> > > MPM, we *need* to have thread-specific SMS that have no locks or
> > > association to anything other than a simple unlocked (from APR's
> > > perspective) malloc/free (aka std) SMS.  -- justin
> > 
> > okay.
> > 
> > well... uhmmm... this is going to sound odd.  i'm not even sure
> > if it will help, because i am a bit out of my depth in understanding
> > the problem.
> > 
> > how about a 'pass-through' sms for threads?
> 
> I'm not sure I'm following you.  The thread SMS never needs locks.
 
i am not necessarily referring to locks.

if there is something _ever_ so slightly different about the
circumstances under which sms instances need to be used, then
what i am suggesting is that instead of doing this:

apr_sms_general_t
apr_sms_thread_capable_general_t
[which was what you were recommending ;)]
apr_sms_trivial_t
apr_sms_thread_capable_trivial_t

instead having a:
apr_sms_thread_wrapping_redirector_t

and use *that* to do the thread-capable-very-specific-things
that i suspect you will need to do for all-and-any SMSes,
pretty much, and that might not necessarily be locking,
which simplifies the coding needed.

> Sander was under the assumption that the parent of a thread can clean up
> a thread.  That isn't supported with POSIX pthreads (cleanly).
> Apparently, Win32 has some support for this.  This lack basically kills
> off the parent/child relationship between SMSes across threads.
 
ah, yes, but it _is_ supported by DCEthreads - see
http://sourceforge.net/projects/freedce which provides,
horror-of-horrors, thread cancellation *emulated* on
top of POSIX threads.

DCEthreads is an implementation of POSIX (draft 4) threads
which of course drastically changed from the final
release.

now, if you examine Windows NT threads, i _bet_ they
support thread cancellation properly, because they
do support thread cancellation via dce/rpc (and dcom,
which is basically an object-orientated interface on
top of dce/rpc).

why?  because, well, that's a long story.

also, take a lok at www.opengroup.org/dce, download the
88mb of the dce 1.22 source code, unpack it to its 430mb
uncompressed size [no, i'm not kidding: yes, that's right,
four hundred and thirty megabytes - 3.5 million lines of code,
i had to use xargs to do wc on them], and you will find that there is
DCEthreads supported for AIX, HP/UX, Solaris, DG/UX, SGI's Unix
(forgotten what it is!  IRIX *duh* :)

... but what i am basically saying is, on NT and Unix,
thread cancellation _is_ possible, it's just that i
think you might not be too happy about the coding-route
you might have to take to _do_ it :) :)


> pthread_cancel() gets us into all sorts of problems we don't want to 
> deal with.  It has been discussed years ago between Ryan, Dean and 
> Manoj - see around 08-1999 - "First in a long line of APR related 
> patches".  And, it isn't guaranteed to work, either (mutex acquires 
> are *not* interruptable).  In that same thread, Ryan also brings up a
> precursor to SMS.  Rather interesting thread, really.  (See
> www.apachelabs.org if you don't have a favorite new-httpd archive.)
> 
> Unless you have a clean way of killing off a thread from a parent,
> thread-based SMS must be directly parented from a basic non-locking 
> SMS (like std SMS).  
> 
> I'm open to any creative suggestions that work.  I'm just not aware 
> of any that work.  -- justin

:)

Re: Pools in threads

[Justin Erenkrantz <je...@ebuilt.com>]

On Mon, Jul 16, 2001 at 12:46:31AM +0200, Luke Kenneth Casson Leighton wrote:
> > In order to provide a win against the current pool code in a threaded
> > MPM, we *need* to have thread-specific SMS that have no locks or
> > association to anything other than a simple unlocked (from APR's
> > perspective) malloc/free (aka std) SMS.  -- justin
> 
> okay.
> 
> well... uhmmm... this is going to sound odd.  i'm not even sure
> if it will help, because i am a bit out of my depth in understanding
> the problem.
> 
> how about a 'pass-through' sms for threads?

I'm not sure I'm following you.  The thread SMS never needs locks.

Sander was under the assumption that the parent of a thread can clean up
a thread.  That isn't supported with POSIX pthreads (cleanly).
Apparently, Win32 has some support for this.  This lack basically kills
off the parent/child relationship between SMSes across threads.

pthread_cancel() gets us into all sorts of problems we don't want to 
deal with.  It has been discussed years ago between Ryan, Dean and 
Manoj - see around 08-1999 - "First in a long line of APR related 
patches".  And, it isn't guaranteed to work, either (mutex acquires 
are *not* interruptable).  In that same thread, Ryan also brings up a
precursor to SMS.  Rather interesting thread, really.  (See
www.apachelabs.org if you don't have a favorite new-httpd archive.)

Unless you have a clean way of killing off a thread from a parent,
thread-based SMS must be directly parented from a basic non-locking 
SMS (like std SMS).  

I'm open to any creative suggestions that work.  I'm just not aware 
of any that work.  -- justin

Re: Pools in threads

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

> In order to provide a win against the current pool code in a threaded
> MPM, we *need* to have thread-specific SMS that have no locks or
> association to anything other than a simple unlocked (from APR's
> perspective) malloc/free (aka std) SMS.  -- justin

okay.

well... uhmmm... this is going to sound odd.  i'm not even sure
if it will help, because i am a bit out of my depth in understanding
the problem.

how about a 'pass-through' sms for threads?

all it does is to create an apr_sms_thread_passthrough() is:

memcpy(&thr_pthru, &someothersms_api);
then OVERRIDE the create function, calling the *someothersms*->create.

you then do your 'thread-specific' stuff that you need to do,
e.g. locking _whatever_, i really don't know, as a wrapper around
the someothersms.

in other words, via this technique, you can turn a non-thread-capable
SMS into a thread-safe one.

... does that sound a bit mad, but useful, or am i way off the mark.

luke

Re: Pools in threads

[Justin Erenkrantz <je...@ebuilt.com>]

On Sun, Jul 15, 2001 at 07:52:46PM +0200, Sander Striker wrote:
> Can't we pass in a data structure to the thread function which has
> all the required information? The data the user wants to pass in will
> need to be in the data structure aswell ofcourse.

-0.9.  I don't like the idea of marshaling the data to the thread's
worker function.

I'm seeing the proposed marshaling like so:

struct {
 void *data;
 apr_pool_t *pool;
} apr_thread_starter_t;

void * worker_function(void *v)
{
 apr_thread_starter *ts = v;
 whatever_the_thread_really_wants_t *d = ts->data;
 ...
}

apr_thread_create(blah, blah, blah)
{
  apr_thread_starter *ts = apr_palloc(...);
  ts->data = the-data-that-user-really-wants-to-pass-in;  
  ts->sms = apr_pool_create(blah, blah, blah);

  thread_create(go call worker_function with ts as the sole argument)
}

Yuck.  

Since I'm seeing the thread SMS be completely separate from the parent
SMS, I'm just not sure why we even want to pass the sms along.  

Unless you can make a case that the thread's SMS needs to know about the
parent SMS.  If so, then we've got major problems in our SMS design.

In order to provide a win against the current pool code in a threaded
MPM, we *need* to have thread-specific SMS that have no locks or
association to anything other than a simple unlocked (from APR's
perspective) malloc/free (aka std) SMS.  -- justin

Re: Design of pools and threads

[Justin Erenkrantz <je...@ebuilt.com>]

On Mon, Jul 16, 2001 at 10:58:17AM -0700, Brian Pane wrote:
>   * The parent pool should control the cleanup of its children
>     to ensure timely resource release (important for memory, but
>     even more important for file descriptors).  In the case of
>     an httpd with a high request volume, this cleanup needs to
>     happen for a request and all its subrequests right after
>     the response is sent; otherwise we

Correct except for the case of a parent/child relationship across 
threads as the parent can't cleanup the children in this case.

FWIW, I'm not talking about the SMS used for a request.  It could be 
cleaned up (i.e. reset) after the request is completed (exactly like 
it is now - no code changes).  It'd free the resources - just like now.
We're simply divorcing its ancestor (the per-thread SMS) from the 
per-process SMS.  The children of this per-thread SMS still have the 
same manner of operation as before.  -- justin

Re: Design of pools and threads

[Brian Pane <bp...@pacbell.net>]

Justin Erenkrantz wrote:
[...]

>- Why do we want an independent per-thread SMS?  Because it now removes 
>  the requirement for locking.  A thread may not be reentrant upon
>  itself - executing in two places at once.  It's a thread with one flow
>  of control.  Memory allocation without the need for locking should 
>  be a good thing.  In order to scale well with threads, I believe httpd 
>  requires independent (thread-local free list) per-thread SMS.
>- Does merely having a per-thread SMS (and its associated thread-local 
>  free list) necessitate a break from its parent SMS (i.e. independent)?
>  Yes, I believe so.  If we were to assume that the per-thread SMS had 
>  a parent/child relationship to the per-process SMS, then we must now 
>  have locking.  By design, a SMS must allocate memory from its parent.
>  Since the parent SMS would now be reentrant (as the parent, it has 
>  spawned multiple threads that are executing in parallel all asking it 
>  for memory), the parent SMS must acquire a lock before allocating 
>  memory.  This lock is the very thing we are trying to avoid in the 
>  first place.  Also, when a parent/child relationship exists between 
>  two SMS, the parent may clean up the child's SMS when it is destroyed.
>- However, for reasons described above, it is impossible for the parent
>  to clean up its children threads forcibly.  The only thing the parent 
>  can do is wait for the thread to exit on its own behalf (via
>  apr_thread_join).  The parent may only make hints (such as 
>  workers_may_exit in threaded MPM) to the thread that it should exit.
>- So, would having an independent per-thread SMS break any assumptions
>  in APR or httpd?  I don't think so.  Why?  I believe that the thread
>  is a logical breaking-off point for memory allocation.  If we were to
>  assume a thread were to create a SMS when it begins, all subsequent
>  processing in that thread will use this SMS.  When this independent
>  per-thread SMS is destroyed (as the thread is exiting - remember we
>  have assumed that the thread must exit voluntarily), the memory is
>  reclaimed.  All descriptors or sockets opened during that threads'
>  life is now returned to the OS.
>
I really think we need a hybrid model that splits the two roles
of a parent in the SMS design.  One of my messages in the archives
talks about this in more detail, but the basic idea is:
  * The parent pool should control the cleanup of its children
    to ensure timely resource release (important for memory, but
    even more important for file descriptors).  In the case of
    an httpd with a high request volume, this cleanup needs to
    happen for a request and all its subrequests right after
    the response is sent; otherwise we
  * But the parent doesn't need to be where the children go
    to get blocks when they need more memory; instead, they
    should call directly to the most efficient source of memory
    that's suitable for the application (e.g., a per-thread SMS).

--Brian

Design of pools and threads

[Justin Erenkrantz <je...@ebuilt.com>]

On Sun, Jul 15, 2001 at 07:20:28PM -0700, rbb@covalent.net wrote:
> Pools are by their very nature hierarchical.  That is why the relationship
> is there, and why it needs to remain there.  You can't just rely on the
> thread to cleanup itself.  Pools (and SMS's) are used for a LOT more than
> just memory allocation in the server.  They make sure that descriptors are
> closed correctly.  In some places, they are used to make sure that
> anything the child opened that isn't returned to the OS automatically is
> returned when the child goes away.
> 
> If you divorce the thread pool from the pool for the child, you WILL break
> many assumptions in the server.

Here is what I understand the situation to be.  Please correct me if 
any of my statements are incorrect.

- A parent may not terminate any child threads forcibly (via a system
  call or other mechanism).  The best mechanism we can hope for (at this
  time - again no real progress has been made on alternatives) is to
  indicate via some way that the thread can honor the request to
  terminate gracefully.
- For various safety/robustness issues, we have decided to not implement 
  a pthread_cancel()-like mechanism.  There are three reasons for this - 
  we may arbitrarily cancel the thread even if it is in the middle of an 
  important transaction and certain operations (acquiring mutexes) are
  non-cancellable.  (It is true that we could set cancellation points
  in the APR-using programs to prevent important transactions from being
  cancelled.)  Furthermore, when issuing a pthread_cancel-like call, 
  certain OSes may leak the resources of a thread.  This is the third
  consideration to implementing this option.  With all of these 
  factors in consideration, Dean, Manoj, and you (and others) decided 
  not to implement a pthread_cancel-like mechanism in APR.  I do think 
  that this was the correct decision.
- Since APR has no way of forcing a thread to exit, this agreement lives 
  entirely outside of the bounds of APR.  What I mean by this is that 
  the APR library can not provide this thread-exit mechanism 
  transparently to APR-using programs via cleanups.  Only the thread 
  itself may choose to shut down.  If the thread so wishes, it may 
  consult any mechanisms it desires (such as shared memory, sockets, 
  pipes, etc.).  But, we have no clean way for a parent to kill a child 
  thread that does not wish to exit.
- As an example of what I mean, please look at the threaded MPM.  Each 
  worker thread is in control of when they may exit, and the mechanism 
  for determining when to exit lies outside of APR (via the 
  process-local workers_may_exit variable and the POD).
- You have suggested that a thread pool has a relationship to its parent
  (from the process before thread was created).  I believe this
  assumption is invalid IF we are to attempt an independent per-thread 
  SMS.  And, I believe that a per-process memory model does not work 
  with a threaded httpd architecture.
- Why do we want an independent per-thread SMS?  Because it now removes 
  the requirement for locking.  A thread may not be reentrant upon
  itself - executing in two places at once.  It's a thread with one flow
  of control.  Memory allocation without the need for locking should 
  be a good thing.  In order to scale well with threads, I believe httpd 
  requires independent (thread-local free list) per-thread SMS.
- Does merely having a per-thread SMS (and its associated thread-local 
  free list) necessitate a break from its parent SMS (i.e. independent)?
  Yes, I believe so.  If we were to assume that the per-thread SMS had 
  a parent/child relationship to the per-process SMS, then we must now 
  have locking.  By design, a SMS must allocate memory from its parent.
  Since the parent SMS would now be reentrant (as the parent, it has 
  spawned multiple threads that are executing in parallel all asking it 
  for memory), the parent SMS must acquire a lock before allocating 
  memory.  This lock is the very thing we are trying to avoid in the 
  first place.  Also, when a parent/child relationship exists between 
  two SMS, the parent may clean up the child's SMS when it is destroyed.
- However, for reasons described above, it is impossible for the parent
  to clean up its children threads forcibly.  The only thing the parent 
  can do is wait for the thread to exit on its own behalf (via
  apr_thread_join).  The parent may only make hints (such as 
  workers_may_exit in threaded MPM) to the thread that it should exit.
- So, would having an independent per-thread SMS break any assumptions
  in APR or httpd?  I don't think so.  Why?  I believe that the thread
  is a logical breaking-off point for memory allocation.  If we were to
  assume a thread were to create a SMS when it begins, all subsequent
  processing in that thread will use this SMS.  When this independent
  per-thread SMS is destroyed (as the thread is exiting - remember we
  have assumed that the thread must exit voluntarily), the memory is
  reclaimed.  All descriptors or sockets opened during that threads'
  life is now returned to the OS.
- Furthermore, we know that the process may not exit before the children
  threads may exit.  For the entire lifetime of the thread, any 
  process-local variables in use by the threads must remain valid.  
  This is inherent in the threading model of all the operating systems 
  we encounter.  A malicious parent indeed, could reset its SMS 
  *before* allowing all of its children to exit.  But, let's not worry
  about that (or any error conditions - we're screwed anyway).
- Again, let's consider the case of the threaded MPM.  The parent of 
  the threads (i.e. child process) will only destroy the process-level 
  SMS when it calls clean_child_exit().  Any per-process open 
  descriptors are now closed as the process terminates.  This call 
  occurs after the call to apr_thread_join() on all started threads.  
  No more child threads are present when this code is called.  Again, 
  this code is correct.

I hope I now have made my position and rationale clearer.  Again, I may
have misinterpreted code or design or made blatantly wrong statements.
I heartily welcome any feedback or corrections.  I am human - 
therefore, I am usually wrong.

Respectfully yours,
Justin Erenkrantz
jerenkrantz@ebuilt.com

Re: Pools in threads WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

> There is *zero* benefit to having any relationship between the 
> thread's memory pool and the parent's memory pool.  You can't cleanup 
> the thread from the parent anyway, so trust the thread to cleanup 
> itself (as well as its pool and any memory allocations).  I fail to 
> see the problem here.  Sever the imagined relationship.  The code 
> becomes simpler, too.  -- justin

coooool.  problem-solving by conventions.  follow these rules,
everything's okay.

Re: Pools in threads WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[rb...@covalent.net]

On Sun, 15 Jul 2001, Justin Erenkrantz wrote:

> On Sun, Jul 15, 2001 at 02:33:54PM -0700, rbb@covalent.net wrote:
> > Guys, before you make comments like this, you should really read the code.
> > First of all, the thread_exit call needs to take in the thread variable.
> > Without it, platforms like OS/2 can't return values from the thread exit
> > routine.
>
> We could stipulate in APR that on OS/2 we won't return the result
> code.  Big deal.  It's not the end of the world.  And, it's only a hack
> that works in conjunction with OS/2's apr_thread_join.  OS/2 threads
> obviously don't support return codes.  APR is only as good as the
> underlying thread library.

No, because that is utterly bogus.  The whole point of APR, is to allow a
program to be written that will work on ANY platform.  There are very few
features that can't be implemented cross platform.

> If you are willing to lose the return code on OS/2, you now have OS/2
> not needing the thread structure either.  APR is a game of tradeoffs -
> this is one we should make in order to get a better API at the cost of
> something that this native OS doesn't provide anyway.

No.  APR is not a game of tradeoffs.  It has never been a game of
tradeoffs.  We only make decisions like this when it is absolutely
impossible to implement a feature on a given system.  As we have already
shown, this can be implemented.

> We should be seeing red flags everywhere because our own code isn't
> calling the apr_thread_exit() function.  Why?  The API is, uh,
> lacking.

No, that is not why we don't call apr_thread_exit.  We don't call
apr_thread_exit, because we don't need it.  There are two ways for threads
to terminate.  1)  Call apr_thread_exit.  2)  return from the function
that was started when the thread was created.  We happen to use option 2,
because it fits our model better.  Do not misinterpret a choice for a
flaw.  Even before the APR threading code was written, there was a hybrid
thread/process version of Apache.  Versions of this can be found in the
apache-apr repository.  That original hybrid Apache did not EVER call
pthread_exit().  This was a conscious choice made by myself, Bill
Stoddard, and Manoj Kasichainula.

> > Secondly, as long as the thread has access to the apr_thread_t, it also
> > has access to the pool within that value.
>
> Well, I disagree that the thread should have access to its underlying
> apr_thread_t.  It's a very awkward thing to do.  Doable?  Certainly.
> Clean?  No.  You now need a pointer into the thread array (or
> linked-list if that is how you do it - almost all places do it now with
> arrays) passed to the worker function.  Gag.

So it has been implemented poorly.  That doesn't mean anything.  A poor
implementation is very often where things begin.  It is relatively simple
to fix the implementation.

> And, once a thread is spawned, it is now on its own for memory.  So,
> it should create a SMS independent of any other SMS.  That's the goal
> and driving force behind SMS - we need a per-thread SMS.

SMS has nothing to do with threads.  I made a mistake when I wrote APR.  I
tied it to pools.  But pools don't work for all apps.  They happen to work
really well for the web server, but most apps don't have memory profiles
like a web server.  The idea of an SMS-like memory allocator was discussed
years ago, and the idea was to make APR useful outside of the web server.

> There is *zero* benefit to having any relationship between the
> thread's memory pool and the parent's memory pool.  You can't cleanup
> the thread from the parent anyway, so trust the thread to cleanup
> itself (as well as its pool and any memory allocations).  I fail to
> see the problem here.  Sever the imagined relationship.  The code
> becomes simpler, too.  -- justin

Pools are by their very nature hierarchical.  That is why the relationship
is there, and why it needs to remain there.  You can't just rely on the
thread to cleanup itself.  Pools (and SMS's) are used for a LOT more than
just memory allocation in the server.  They make sure that descriptors are
closed correctly.  In some places, they are used to make sure that
anything the child opened that isn't returned to the OS automatically is
returned when the child goes away.

If you divorce the thread pool from the pool for the child, you WILL break
many assumptions in the server.

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

Re: Pools in threads WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Justin Erenkrantz <je...@ebuilt.com>]

On Sun, Jul 15, 2001 at 02:33:54PM -0700, rbb@covalent.net wrote:
> Guys, before you make comments like this, you should really read the code.
> First of all, the thread_exit call needs to take in the thread variable.
> Without it, platforms like OS/2 can't return values from the thread exit
> routine.

We could stipulate in APR that on OS/2 we won't return the result 
code.  Big deal.  It's not the end of the world.  And, it's only a hack
that works in conjunction with OS/2's apr_thread_join.  OS/2 threads
obviously don't support return codes.  APR is only as good as the
underlying thread library.

BeOS, Unix (pthread), and Win32 don't need the thread structure to 
destroy the thread or return the result code.  That seems to be the 
consensus on how threads should operate.  

If you are willing to lose the return code on OS/2, you now have OS/2 
not needing the thread structure either.  APR is a game of tradeoffs - 
this is one we should make in order to get a better API at the cost of 
something that this native OS doesn't provide anyway.

We should be seeing red flags everywhere because our own code isn't
calling the apr_thread_exit() function.  Why?  The API is, uh, 
lacking.

> Secondly, as long as the thread has access to the apr_thread_t, it also
> has access to the pool within that value.

Well, I disagree that the thread should have access to its underlying
apr_thread_t.  It's a very awkward thing to do.  Doable?  Certainly.
Clean?  No.  You now need a pointer into the thread array (or
linked-list if that is how you do it - almost all places do it now with
arrays) passed to the worker function.  Gag.

And, once a thread is spawned, it is now on its own for memory.  So, 
it should create a SMS independent of any other SMS.  That's the goal 
and driving force behind SMS - we need a per-thread SMS.

There is *zero* benefit to having any relationship between the 
thread's memory pool and the parent's memory pool.  You can't cleanup 
the thread from the parent anyway, so trust the thread to cleanup 
itself (as well as its pool and any memory allocations).  I fail to 
see the problem here.  Sever the imagined relationship.  The code 
becomes simpler, too.  -- justin

RE: Pools in threads

[Sander Striker <st...@apache.org>]

>>> Fair enough. It's just that in order to opt-out of the 
>>> child-pool creating
>>> process in apr_thread_create, we're going to have to add a parameter
>> 
>> Why are we so desperate in opting out the child-pool creation?
>> I don't really have problems with a child pool for each thread. 
>> Actually, it will make the dynamic locking a lot easier to implement
>> if it stays.
> 
> The problem is that apr_thread_create is completely bogus right now.
>
> The child thread has no access to that pool in apr_thread_t->pool.
> If you want a pool in the thread, you should be creating inside
> of the thread's function not in apr_thread_create.

I can live with that. It's the registration of the thread with
the parent sms [when/if pools==sms] that has my interest :)
 
> And, apr_thread_exit call should *not* be taking in an apr_thread_t
> because the child doesn't have access to that either.  

Can't we pass in a data structure to the thread function which has
all the required information? The data the user wants to pass in will
need to be in the data structure aswell ofcourse.

> We're enforcing the use of global variables to work around this and 
> that's plain wrong.  -- justin

Yes, I agree.

Sander

Re: Pools in threads WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[rb...@covalent.net]

On Sun, 15 Jul 2001, Justin Erenkrantz wrote:

> On Sun, Jul 15, 2001 at 07:16:35PM +0200, Sander Striker wrote:
> > > Fair enough. It's just that in order to opt-out of the child-pool creating
> > > process in apr_thread_create, we're going to have to add a parameter
> >
> > Why are we so desperate in opting out the child-pool creation?
> > I don't really have problems with a child pool for each thread. Actually,
> > it will make the dynamic locking a lot easier to implement if it stays.
>
> The problem is that apr_thread_create is completely bogus right now.
>
> The child thread has no access to that pool in apr_thread_t->pool.
> If you want a pool in the thread, you should be creating inside
> of the thread's function not in apr_thread_create.
>
> And, apr_thread_exit call should *not* be taking in an apr_thread_t
> because the child doesn't have access to that either.
>
> We're enforcing the use of global variables to work around this and
> that's plain wrong.  -- justin

Guys, before you make comments like this, you should really read the code.
First of all, the thread_exit call needs to take in the thread variable.
Without it, platforms like OS/2 can't return values from the thread exit
routine.

Secondly, as long as the thread has access to the apr_thread_t, it also
has access to the pool within that value.

What you all need to realize, is that some of the annoying pieces of APR
exist because we tried to do it the way that makes sense to all of you,
and we had to change it to make it work on all platforms.  Please, look
through what ALL platforms do before you decide to go changing API's.  You
are going to break things that work today.

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

Pools in threads WAS Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Justin Erenkrantz <je...@ebuilt.com>]

On Sun, Jul 15, 2001 at 07:16:35PM +0200, Sander Striker wrote:
> > Fair enough. It's just that in order to opt-out of the child-pool creating
> > process in apr_thread_create, we're going to have to add a parameter
> 
> Why are we so desperate in opting out the child-pool creation?
> I don't really have problems with a child pool for each thread. Actually,
> it will make the dynamic locking a lot easier to implement if it stays.

The problem is that apr_thread_create is completely bogus right now.

The child thread has no access to that pool in apr_thread_t->pool.
If you want a pool in the thread, you should be creating inside
of the thread's function not in apr_thread_create.

And, apr_thread_exit call should *not* be taking in an apr_thread_t
because the child doesn't have access to that either.  

We're enforcing the use of global variables to work around this and 
that's plain wrong.  -- justin

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

On Tue, Jul 17, 2001 at 10:12:35AM -0700, Aaron Bannert wrote:

> - Optionally creating a child-pool for each thread. This provides us two
> things in my mind:
>   1) Allowing the application to choose an SMS for this thread-pool (which
>      could very well be a special SMS created just for this purpose).

for example, you may wish to create a special 'security' thread.
it uses an apr_sms_mlock_t instance.  this special thread handles
all of your security, keeping private keys and plain-text user
passwords in-memory.  

luke

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

On Tue, Jul 17, 2001 at 10:12:35AM -0700, Aaron Bannert wrote:

> - Optionally creating a child-pool for each thread. This provides us two
> things in my mind:
>   1) Allowing the application to choose an SMS for this thread-pool (which
>      could very well be a special SMS created just for this purpose).

for example, you may wish to create a special 'security' thread.
it uses an apr_sms_mlock_t instance.  this special thread handles
all of your security, keeping private keys and plain-text user
passwords in-memory.  

luke

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

> We have hit an impass in my mind.  Dean and I are saying that having each
> thread have it's own pool is a requirement.  Not just for httpd, but for
> anything using pools.  Dean, if I am mis-interpreting you, then I am
> sorry, and please correct me.
> 
> Aaron,  you disagree.  you want each app to make that decision on their
> own.  I see no way to implement that in a thread-safe manner.

(Just as a side note, I don't remember seeing anything refuting the technical
aspects of such an implementation. I would be interested in one, of course,
if you think it is infeasible.)


> So, I would suggest that we approach this problem in another way.  My ONLY
> goal for APR is to make a library that is useful for cross-platform
> development and eases the pain of cross-platform development for
> developers.  That was the goal when I created APR, and my original test
> case was the Apache web server.
> 
> I believe that the problem is that the threaded code is creating the
> pool, and not advertising it to the thread itself.  This is an easy thing
> to fix.  I do not agree that every APR app that uses threads should have
> to create their own thread pools.  That is wasted effort by every APR app.
> 
> I would like to see a conclusion to this thread quickly.  So, could people
> please post their desires quickly, and what they want to see from this.

In my mind, there are really only two major points that need addressing:

- Optionally creating a child-pool for each thread. This provides us two
things in my mind:
  1) Allowing the application to choose an SMS for this thread-pool (which
     could very well be a special SMS created just for this purpose).
  2) Allowing an application to refrain from creating a child-pool.

- Like you [Ryan] said, it is a problem that the thread creation routines
right now are not advertising the pool to the thread itself. This to
me can be corrected with a better API. The two things I've brought up
here are:
  1) void * worker_fn(void *opaque_data) should have an additional
     pool param.
  2) apr_thread_exit() is unacceptable. It currently requires a
     apr_thread_t structure. I would be happy if either the worker_fn was
     extended to pass in the apr_thread_t OR if we could architect a way
     for apr_thread_exit() to not need the apr_thread_t struct.


I have a feeling the second point will be more palatable, and as such
I'll volunteer to cook up a patch illustrating these changes.

As for the first point, which is the reason this thread has been going on
so long, I'm willing to compromise on the idea of forcefully creating
a childpool always, if it necessary, merely because I see it only as
overhead for some applications, but not a functional defect.

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

> We have hit an impass in my mind.  Dean and I are saying that having each
> thread have it's own pool is a requirement.  Not just for httpd, but for
> anything using pools.  Dean, if I am mis-interpreting you, then I am
> sorry, and please correct me.
> 
> Aaron,  you disagree.  you want each app to make that decision on their
> own.  I see no way to implement that in a thread-safe manner.

(Just as a side note, I don't remember seeing anything refuting the technical
aspects of such an implementation. I would be interested in one, of course,
if you think it is infeasible.)


> So, I would suggest that we approach this problem in another way.  My ONLY
> goal for APR is to make a library that is useful for cross-platform
> development and eases the pain of cross-platform development for
> developers.  That was the goal when I created APR, and my original test
> case was the Apache web server.
> 
> I believe that the problem is that the threaded code is creating the
> pool, and not advertising it to the thread itself.  This is an easy thing
> to fix.  I do not agree that every APR app that uses threads should have
> to create their own thread pools.  That is wasted effort by every APR app.
> 
> I would like to see a conclusion to this thread quickly.  So, could people
> please post their desires quickly, and what they want to see from this.

In my mind, there are really only two major points that need addressing:

- Optionally creating a child-pool for each thread. This provides us two
things in my mind:
  1) Allowing the application to choose an SMS for this thread-pool (which
     could very well be a special SMS created just for this purpose).
  2) Allowing an application to refrain from creating a child-pool.

- Like you [Ryan] said, it is a problem that the thread creation routines
right now are not advertising the pool to the thread itself. This to
me can be corrected with a better API. The two things I've brought up
here are:
  1) void * worker_fn(void *opaque_data) should have an additional
     pool param.
  2) apr_thread_exit() is unacceptable. It currently requires a
     apr_thread_t structure. I would be happy if either the worker_fn was
     extended to pass in the apr_thread_t OR if we could architect a way
     for apr_thread_exit() to not need the apr_thread_t struct.


I have a feeling the second point will be more palatable, and as such
I'll volunteer to cook up a patch illustrating these changes.

As for the first point, which is the reason this thread has been going on
so long, I'm willing to compromise on the idea of forcefully creating
a childpool always, if it necessary, merely because I see it only as
overhead for some applications, but not a functional defect.

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

> > I totally agree, but only as a solution in httpd.
> 
> no, everywhere.
> 
> > I also believe that we should provide this [application-specific requirement]
> > outside of the basic thread support in APR.
> >
> > Please allow me to use pseudocode:
> >
> > void * worker_function(void * opaque_application_data) {
> >
> >    apr_pool_t *thread_pool;
> >
> >    create_child_pool(&thread_pool, global_root_pool);
> 
> now you've got mutexes in global_root_pool.  see my performance comment
> above.

I'm not advocating this scenario as a real solution, but merely trying
to illustrate how the child-pool creation routines don't really have to
be in apr_thread_create(). The application using APR threads can do this
before calling apr_thread_create() or it can do it in it's worker_function.

Either way, it is better to leave it totally up to the application, no?

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

> > I totally agree, but only as a solution in httpd.
> 
> no, everywhere.
> 
> > I also believe that we should provide this [application-specific requirement]
> > outside of the basic thread support in APR.
> >
> > Please allow me to use pseudocode:
> >
> > void * worker_function(void * opaque_application_data) {
> >
> >    apr_pool_t *thread_pool;
> >
> >    create_child_pool(&thread_pool, global_root_pool);
> 
> now you've got mutexes in global_root_pool.  see my performance comment
> above.

I'm not advocating this scenario as a real solution, but merely trying
to illustrate how the child-pool creation routines don't really have to
be in apr_thread_create(). The application using APR threads can do this
before calling apr_thread_create() or it can do it in it's worker_function.

Either way, it is better to leave it totally up to the application, no?

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Justin Erenkrantz <je...@ebuilt.com>]

On Tue, Jul 17, 2001 at 04:49:04PM -0700, dean gaudet wrote:
> > Please allow me to use pseudocode:
> >
> > void * worker_function(void * opaque_application_data) {
> >
> >    apr_pool_t *thread_pool;
> >
> >    create_child_pool(&thread_pool, global_root_pool);
> 
> now you've got mutexes in global_root_pool.  see my performance comment
> above.

IMHO, the global_root_pool should be the "standard" SMS - which is 
just a SMS that calls malloc/free - no locking there.  It actually 
has no free-list/allocation-list.  This strategy may or may not 
work - depending if you'd ever want to cleanup the global_root_pool 
(in Aaron's example).  Which I don't think we'd want to do anyway.
-- justin

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Justin Erenkrantz <je...@ebuilt.com>]

On Tue, Jul 17, 2001 at 10:59:03AM -0700, Brian Pane wrote:
> the unsynchronized approach can crash due to a race condition.)  Thus my
> argument is that this scenario (forking from a threaded MPM and then trying
> to clean up the other threads' resources in the child process) isn't a
> valid use case for the design of the pools.

+1.  -- justin

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Brian Pane <bp...@pacbell.net>]

Aaron Bannert wrote:

>On Tue, Jul 17, 2001 at 10:17:01AM -0700, Brian Pane wrote:
>
>>Aaron Bannert wrote:
>>
>>>>I'm not sure that the alternative is workable, either.
>>>>
>>>>At the time of the fork, when the child process gets a snapshot of
>>>>the parent's memory, it's possible that some thread other than the one
>>>>invoking fork could be halfway through registering a new resource (e.g.,
>>>>file descriptor) in its pool.  There's no guarantee that it's safe to
>>>>attempt a cleanup of that other thread's pool in the child process;
>>>>if the fork has caught the data structures in an intermediate state,
>>>>attempting to destroy that pool might yield a segv.
>>>>
>>>Correct me if I'm wrong, but that would be a property of a buggy MPM.
>>>If the MPM chooses to mix non-synchronized fork()s and thread invocation,
>>>than that's what it gets.
>>>
>>Synchronizing the forks with pool resource registration isn't a scalable
>>design; it requires locking a process-wide lock every time you want to
>>register a resource in a per-thread pool.  It would be a huge mistake
>>to slow down the routine operation of the httpd in order to optimize for
>>fork.
>>
>
>I wasn't suggesting that, I was suggesting that it's a bad idea to start
>fork()ing off processes from an MPM that's also doing threads and you
>want that fork()ed of process to later do cleanups.
>
I agree.

>My point is it's solely an MPM architecture decision. Are you saying that
>this is a typical scenario?
>
I think it's an atypical scenario, and one that can't be implemented very
well anyway.  (The synchronized approach has a performance problem, and
the unsynchronized approach can crash due to a race condition.)  Thus my
argument is that this scenario (forking from a threaded MPM and then trying
to clean up the other threads' resources in the child process) isn't a
valid use case for the design of the pools.

--Brian

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Brian Pane <bp...@pacbell.net>]

Aaron Bannert wrote:

>On Tue, Jul 17, 2001 at 10:17:01AM -0700, Brian Pane wrote:
>
>>Aaron Bannert wrote:
>>
>>>>I'm not sure that the alternative is workable, either.
>>>>
>>>>At the time of the fork, when the child process gets a snapshot of
>>>>the parent's memory, it's possible that some thread other than the one
>>>>invoking fork could be halfway through registering a new resource (e.g.,
>>>>file descriptor) in its pool.  There's no guarantee that it's safe to
>>>>attempt a cleanup of that other thread's pool in the child process;
>>>>if the fork has caught the data structures in an intermediate state,
>>>>attempting to destroy that pool might yield a segv.
>>>>
>>>Correct me if I'm wrong, but that would be a property of a buggy MPM.
>>>If the MPM chooses to mix non-synchronized fork()s and thread invocation,
>>>than that's what it gets.
>>>
>>Synchronizing the forks with pool resource registration isn't a scalable
>>design; it requires locking a process-wide lock every time you want to
>>register a resource in a per-thread pool.  It would be a huge mistake
>>to slow down the routine operation of the httpd in order to optimize for
>>fork.
>>
>
>I wasn't suggesting that, I was suggesting that it's a bad idea to start
>fork()ing off processes from an MPM that's also doing threads and you
>want that fork()ed of process to later do cleanups.
>
I agree.

>My point is it's solely an MPM architecture decision. Are you saying that
>this is a typical scenario?
>
I think it's an atypical scenario, and one that can't be implemented very
well anyway.  (The synchronized approach has a performance problem, and
the unsynchronized approach can crash due to a race condition.)  Thus my
argument is that this scenario (forking from a threaded MPM and then trying
to clean up the other threads' resources in the child process) isn't a
valid use case for the design of the pools.

--Brian

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

On Tue, Jul 17, 2001 at 10:17:01AM -0700, Brian Pane wrote:
> Aaron Bannert wrote:
> 
> >>I'm not sure that the alternative is workable, either.
> >>
> >>At the time of the fork, when the child process gets a snapshot of
> >>the parent's memory, it's possible that some thread other than the one
> >>invoking fork could be halfway through registering a new resource (e.g.,
> >>file descriptor) in its pool.  There's no guarantee that it's safe to
> >>attempt a cleanup of that other thread's pool in the child process;
> >>if the fork has caught the data structures in an intermediate state,
> >>attempting to destroy that pool might yield a segv.
> >>
> >
> >Correct me if I'm wrong, but that would be a property of a buggy MPM.
> >If the MPM chooses to mix non-synchronized fork()s and thread invocation,
> >than that's what it gets.
> >
> Synchronizing the forks with pool resource registration isn't a scalable
> design; it requires locking a process-wide lock every time you want to
> register a resource in a per-thread pool.  It would be a huge mistake
> to slow down the routine operation of the httpd in order to optimize for
> fork.

I wasn't suggesting that, I was suggesting that it's a bad idea to start
fork()ing off processes from an MPM that's also doing threads and you
want that fork()ed of process to later do cleanups.

My point is it's solely an MPM architecture decision. Are you saying that
this is a typical scenario?

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

On Tue, Jul 17, 2001 at 10:17:01AM -0700, Brian Pane wrote:
> Aaron Bannert wrote:
> 
> >>I'm not sure that the alternative is workable, either.
> >>
> >>At the time of the fork, when the child process gets a snapshot of
> >>the parent's memory, it's possible that some thread other than the one
> >>invoking fork could be halfway through registering a new resource (e.g.,
> >>file descriptor) in its pool.  There's no guarantee that it's safe to
> >>attempt a cleanup of that other thread's pool in the child process;
> >>if the fork has caught the data structures in an intermediate state,
> >>attempting to destroy that pool might yield a segv.
> >>
> >
> >Correct me if I'm wrong, but that would be a property of a buggy MPM.
> >If the MPM chooses to mix non-synchronized fork()s and thread invocation,
> >than that's what it gets.
> >
> Synchronizing the forks with pool resource registration isn't a scalable
> design; it requires locking a process-wide lock every time you want to
> register a resource in a per-thread pool.  It would be a huge mistake
> to slow down the routine operation of the httpd in order to optimize for
> fork.

I wasn't suggesting that, I was suggesting that it's a bad idea to start
fork()ing off processes from an MPM that's also doing threads and you
want that fork()ed of process to later do cleanups.

My point is it's solely an MPM architecture decision. Are you saying that
this is a typical scenario?

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Brian Pane <bp...@pacbell.net>]

Aaron Bannert wrote:

>>I'm not sure that the alternative is workable, either.
>>
>>At the time of the fork, when the child process gets a snapshot of
>>the parent's memory, it's possible that some thread other than the one
>>invoking fork could be halfway through registering a new resource (e.g.,
>>file descriptor) in its pool.  There's no guarantee that it's safe to
>>attempt a cleanup of that other thread's pool in the child process;
>>if the fork has caught the data structures in an intermediate state,
>>attempting to destroy that pool might yield a segv.
>>
>
>Correct me if I'm wrong, but that would be a property of a buggy MPM.
>If the MPM chooses to mix non-synchronized fork()s and thread invocation,
>than that's what it gets.
>
Synchronizing the forks with pool resource registration isn't a scalable
design; it requires locking a process-wide lock every time you want to
register a resource in a per-thread pool.  It would be a huge mistake
to slow down the routine operation of the httpd in order to optimize for
fork.

--Brian

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Brian Pane <bp...@pacbell.net>]

Aaron Bannert wrote:

>>I'm not sure that the alternative is workable, either.
>>
>>At the time of the fork, when the child process gets a snapshot of
>>the parent's memory, it's possible that some thread other than the one
>>invoking fork could be halfway through registering a new resource (e.g.,
>>file descriptor) in its pool.  There's no guarantee that it's safe to
>>attempt a cleanup of that other thread's pool in the child process;
>>if the fork has caught the data structures in an intermediate state,
>>attempting to destroy that pool might yield a segv.
>>
>
>Correct me if I'm wrong, but that would be a property of a buggy MPM.
>If the MPM chooses to mix non-synchronized fork()s and thread invocation,
>than that's what it gets.
>
Synchronizing the forks with pool resource registration isn't a scalable
design; it requires locking a process-wide lock every time you want to
register a resource in a per-thread pool.  It would be a huge mistake
to slow down the routine operation of the httpd in order to optimize for
fork.

--Brian

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

> I'm not sure that the alternative is workable, either.
> 
> At the time of the fork, when the child process gets a snapshot of
> the parent's memory, it's possible that some thread other than the one
> invoking fork could be halfway through registering a new resource (e.g.,
> file descriptor) in its pool.  There's no guarantee that it's safe to
> attempt a cleanup of that other thread's pool in the child process;
> if the fork has caught the data structures in an intermediate state,
> attempting to destroy that pool might yield a segv.

Correct me if I'm wrong, but that would be a property of a buggy MPM.
If the MPM chooses to mix non-synchronized fork()s and thread invocation,
than that's what it gets.

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

> I'm not sure that the alternative is workable, either.
> 
> At the time of the fork, when the child process gets a snapshot of
> the parent's memory, it's possible that some thread other than the one
> invoking fork could be halfway through registering a new resource (e.g.,
> file descriptor) in its pool.  There's no guarantee that it's safe to
> attempt a cleanup of that other thread's pool in the child process;
> if the fork has caught the data structures in an intermediate state,
> attempting to destroy that pool might yield a segv.

Correct me if I'm wrong, but that would be a property of a buggy MPM.
If the MPM chooses to mix non-synchronized fork()s and thread invocation,
than that's what it gets.

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Brian Pane <bp...@pacbell.net>]

William A. Rowe, Jr. wrote:

>From: <rb...@covalent.net>
>Sent: Tuesday, July 17, 2001 11:13 AM
>
>>I believe that the problem is that the threaded code is creating the
>>pool, and not advertising it to the thread itself.  This is an easy thing
>>to fix.  I do not agree that every APR app that uses threads should have
>>to create their own thread pools.  That is wasted effort by every APR app.
>>
>>I would like to see a conclusion to this thread quickly.  So, could people
>>please post their desires quickly, and what they want to see from this.
>>
>
>thread-local pools would be a nice improvement, perhaps as an argument to
>apr_thread_create.  They can be flagged such that they avoid locking their own
>own allocations (of course their 'parent' allocator, malloc, or the parent
>free-list management, may still need to lock based on platform and so forth). 
>
>But _unless_ they remain rooted to the top level pool, in apr_process_create 
>they become orphaned, and their cleanups are never executed.  That isn't workable.
>
I'm not sure that the alternative is workable, either.

At the time of the fork, when the child process gets a snapshot of
the parent's memory, it's possible that some thread other than the one
invoking fork could be halfway through registering a new resource (e.g.,
file descriptor) in its pool.  There's no guarantee that it's safe to
attempt a cleanup of that other thread's pool in the child process;
if the fork has caught the data structures in an intermediate state,
attempting to destroy that pool might yield a segv.

--Brian

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Brian Pane <bp...@pacbell.net>]

William A. Rowe, Jr. wrote:

>From: <rb...@covalent.net>
>Sent: Tuesday, July 17, 2001 11:13 AM
>
>>I believe that the problem is that the threaded code is creating the
>>pool, and not advertising it to the thread itself.  This is an easy thing
>>to fix.  I do not agree that every APR app that uses threads should have
>>to create their own thread pools.  That is wasted effort by every APR app.
>>
>>I would like to see a conclusion to this thread quickly.  So, could people
>>please post their desires quickly, and what they want to see from this.
>>
>
>thread-local pools would be a nice improvement, perhaps as an argument to
>apr_thread_create.  They can be flagged such that they avoid locking their own
>own allocations (of course their 'parent' allocator, malloc, or the parent
>free-list management, may still need to lock based on platform and so forth). 
>
>But _unless_ they remain rooted to the top level pool, in apr_process_create 
>they become orphaned, and their cleanups are never executed.  That isn't workable.
>
I'm not sure that the alternative is workable, either.

At the time of the fork, when the child process gets a snapshot of
the parent's memory, it's possible that some thread other than the one
invoking fork could be halfway through registering a new resource (e.g.,
file descriptor) in its pool.  There's no guarantee that it's safe to
attempt a cleanup of that other thread's pool in the child process;
if the fork has caught the data structures in an intermediate state,
attempting to destroy that pool might yield a segv.

--Brian

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Justin Erenkrantz <je...@ebuilt.com>]

On Tue, Jul 17, 2001 at 12:13:08PM -0500, William A. Rowe, Jr. wrote:
> Because, otherwise, all resources (files, sockets, locks, etc) are not closed when
> another thread forks.
> 
> There is a difference between a clean thread exit and cleaning up after a thread.
> We can't do a clean thread exit.  We can clean up after one, if the resources are
> apr_entities with properly registered cleanups.

But, if the thread exits cleanly by calling apr_thread_exit() or has its
own private SMS (I don't care which, but I think storing the SMS in the
apr_thread_t object is a bit unclean), it'll return all of its
resources with properly registered cleanups when the per-thread SMS is 
destroyed as the thread exits cleanly.  Again, I'm just not seeing what 
the problem is here.  -- justin

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

From: "Justin Erenkrantz" <je...@ebuilt.com>
Sent: Tuesday, July 17, 2001 12:00 PM


> On Tue, Jul 17, 2001 at 11:26:55AM -0500, William A. Rowe, Jr. wrote:
> > But _unless_ they remain rooted to the top level pool, in apr_process_create 
> > they become orphaned, and their cleanups are never executed.  That isn't workable.
> 
> I disagree with this.  I gave my big long monologue on this on Sunday
> night.  In summary, since you can't cleanup the thread itself, why are 
> we cleaning up its memory?  -- justin

Because, otherwise, all resources (files, sockets, locks, etc) are not closed when
another thread forks.

There is a difference between a clean thread exit and cleaning up after a thread.
We can't do a clean thread exit.  We can clean up after one, if the resources are
apr_entities with properly registered cleanups.

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Justin Erenkrantz <je...@ebuilt.com>]

On Tue, Jul 17, 2001 at 11:26:55AM -0500, William A. Rowe, Jr. wrote:
> But _unless_ they remain rooted to the top level pool, in apr_process_create 
> they become orphaned, and their cleanups are never executed.  That isn't workable.

I disagree with this.  I gave my big long monologue on this on Sunday
night.  In summary, since you can't cleanup the thread itself, why are 
we cleaning up its memory?  -- justin

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

From: <rb...@covalent.net>
Sent: Tuesday, July 17, 2001 11:13 AM
> 
> I believe that the problem is that the threaded code is creating the
> pool, and not advertising it to the thread itself.  This is an easy thing
> to fix.  I do not agree that every APR app that uses threads should have
> to create their own thread pools.  That is wasted effort by every APR app.
> 
> I would like to see a conclusion to this thread quickly.  So, could people
> please post their desires quickly, and what they want to see from this.

thread-local pools would be a nice improvement, perhaps as an argument to
apr_thread_create.  They can be flagged such that they avoid locking their own
own allocations (of course their 'parent' allocator, malloc, or the parent
free-list management, may still need to lock based on platform and so forth). 

But _unless_ they remain rooted to the top level pool, in apr_process_create 
they become orphaned, and their cleanups are never executed.  That isn't workable.

It sounds like (forgive me if I'm stating the obvious) that a pool needs two
kinds of parent sms pools.  One is the 'heirarchy' for purposes of setup and
teardown.  The other is the 'allocator', where the blocks will be allocated from.

Want to use the conventional 1.3 model?  The heirarchial parent is the parent pool,
but the allocatation parent is the single top level pool.

Want to flip to a malloc, that trusts the clib to 'do the right thing'(sm) with
cpu/thread specific free lists?  Then we build a very simple malloc/free sms that
can be passed as the allocation parent when the pool is created.

If I've abused the sms concept here, then correct me, but I don't see any issues
remaining beyond semantics and syntax.

Bill

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

From: <rb...@covalent.net>
Sent: Tuesday, July 17, 2001 11:13 AM
> 
> I believe that the problem is that the threaded code is creating the
> pool, and not advertising it to the thread itself.  This is an easy thing
> to fix.  I do not agree that every APR app that uses threads should have
> to create their own thread pools.  That is wasted effort by every APR app.
> 
> I would like to see a conclusion to this thread quickly.  So, could people
> please post their desires quickly, and what they want to see from this.

thread-local pools would be a nice improvement, perhaps as an argument to
apr_thread_create.  They can be flagged such that they avoid locking their own
own allocations (of course their 'parent' allocator, malloc, or the parent
free-list management, may still need to lock based on platform and so forth). 

But _unless_ they remain rooted to the top level pool, in apr_process_create 
they become orphaned, and their cleanups are never executed.  That isn't workable.

It sounds like (forgive me if I'm stating the obvious) that a pool needs two
kinds of parent sms pools.  One is the 'heirarchy' for purposes of setup and
teardown.  The other is the 'allocator', where the blocks will be allocated from.

Want to use the conventional 1.3 model?  The heirarchial parent is the parent pool,
but the allocatation parent is the single top level pool.

Want to flip to a malloc, that trusts the clib to 'do the right thing'(sm) with
cpu/thread specific free lists?  Then we build a very simple malloc/free sms that
can be passed as the allocation parent when the pool is created.

If I've abused the sms concept here, then correct me, but I don't see any issues
remaining beyond semantics and syntax.

Bill

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[rb...@covalent.net]

On Tue, 17 Jul 2001, Aaron Bannert wrote:

> On Tue, Jul 17, 2001 at 01:29:47AM -0700, dean gaudet wrote:
> > On Sun, 15 Jul 2001, Sander Striker wrote:
> >
> > > Why are we so desperate in opting out the child-pool creation?
> > > I don't really have problems with a child pool for each thread. Actually,
> > > it will make the dynamic locking a lot easier to implement if it stays.
> >
> > all threads MUST have their own private pool root.
> >
> > otherwise you're just throwing scalability away in locks.  (which is
> > proved by the claim i saw that ian got better performance by defining
> > ALLOC_USE_MALLOC on an 8-way.)
>
> I totally agree, but only as a solution in httpd.
>
>
> I also believe that we should provide this [application-specific requirement]
> outside of the basic thread support in APR.
>
> Please allow me to use pseudocode:
>
> void * worker_function(void * opaque_application_data) {
>
>    apr_pool_t *thread_pool;
>
>    create_child_pool(&thread_pool, global_root_pool);
>
>    do_thread_stuff();
>
>    cleanup_child_pool(thread_pool);
>
>    if (apr_thread_exit_makes_sense_to_call_from_inside_worker_function) {
>       int rv = APR_SUCCESS;
>       apr_thread_exit(&rv);
>    }
>
>    return NULL;
> }
>
> What I mean by this gibberish is that the pool-handling code can exist
> completely outside of our thread_creation-type functions in APR. Is there
> any reason for APR threads to do more than this (that is, just simple thread
> creation)?

We have hit an impass in my mind.  Dean and I are saying that having each
thread have it's own pool is a requirement.  Not just for httpd, but for
anything using pools.  Dean, if I am mis-interpreting you, then I am
sorry, and please correct me.

Aaron,  you disagree.  you want each app to make that decision on their
own.  I see no way to implement that in a thread-safe manner.

So, I would suggest that we approach this problem in another way.  My ONLY
goal for APR is to make a library that is useful for cross-platform
development and eases the pain of cross-platform development for
developers.  That was the goal when I created APR, and my original test
case was the Apache web server.

I believe that the problem is that the threaded code is creating the
pool, and not advertising it to the thread itself.  This is an easy thing
to fix.  I do not agree that every APR app that uses threads should have
to create their own thread pools.  That is wasted effort by every APR app.

I would like to see a conclusion to this thread quickly.  So, could people
please post their desires quickly, and what they want to see from this.

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[dean gaudet <dg...@arctic.org>]

On Tue, 17 Jul 2001, Aaron Bannert wrote:

> On Tue, Jul 17, 2001 at 01:29:47AM -0700, dean gaudet wrote:
> > On Sun, 15 Jul 2001, Sander Striker wrote:
> >
> > > Why are we so desperate in opting out the child-pool creation?
> > > I don't really have problems with a child pool for each thread. Actually,
> > > it will make the dynamic locking a lot easier to implement if it stays.
> >
> > all threads MUST have their own private pool root.
> >
> > otherwise you're just throwing scalability away in locks.  (which is
> > proved by the claim i saw that ian got better performance by defining
> > ALLOC_USE_MALLOC on an 8-way.)
>
> I totally agree, but only as a solution in httpd.

no, everywhere.

> I also believe that we should provide this [application-specific requirement]
> outside of the basic thread support in APR.
>
> Please allow me to use pseudocode:
>
> void * worker_function(void * opaque_application_data) {
>
>    apr_pool_t *thread_pool;
>
>    create_child_pool(&thread_pool, global_root_pool);

now you've got mutexes in global_root_pool.  see my performance comment
above.

-dean

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[dean gaudet <dg...@arctic.org>]

On Tue, 17 Jul 2001, Aaron Bannert wrote:

> On Tue, Jul 17, 2001 at 01:29:47AM -0700, dean gaudet wrote:
> > On Sun, 15 Jul 2001, Sander Striker wrote:
> >
> > > Why are we so desperate in opting out the child-pool creation?
> > > I don't really have problems with a child pool for each thread. Actually,
> > > it will make the dynamic locking a lot easier to implement if it stays.
> >
> > all threads MUST have their own private pool root.
> >
> > otherwise you're just throwing scalability away in locks.  (which is
> > proved by the claim i saw that ian got better performance by defining
> > ALLOC_USE_MALLOC on an 8-way.)
>
> I totally agree, but only as a solution in httpd.

no, everywhere.

> I also believe that we should provide this [application-specific requirement]
> outside of the basic thread support in APR.
>
> Please allow me to use pseudocode:
>
> void * worker_function(void * opaque_application_data) {
>
>    apr_pool_t *thread_pool;
>
>    create_child_pool(&thread_pool, global_root_pool);

now you've got mutexes in global_root_pool.  see my performance comment
above.

-dean

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[rb...@covalent.net]

On Tue, 17 Jul 2001, Aaron Bannert wrote:

> On Tue, Jul 17, 2001 at 01:29:47AM -0700, dean gaudet wrote:
> > On Sun, 15 Jul 2001, Sander Striker wrote:
> >
> > > Why are we so desperate in opting out the child-pool creation?
> > > I don't really have problems with a child pool for each thread. Actually,
> > > it will make the dynamic locking a lot easier to implement if it stays.
> >
> > all threads MUST have their own private pool root.
> >
> > otherwise you're just throwing scalability away in locks.  (which is
> > proved by the claim i saw that ian got better performance by defining
> > ALLOC_USE_MALLOC on an 8-way.)
>
> I totally agree, but only as a solution in httpd.
>
>
> I also believe that we should provide this [application-specific requirement]
> outside of the basic thread support in APR.
>
> Please allow me to use pseudocode:
>
> void * worker_function(void * opaque_application_data) {
>
>    apr_pool_t *thread_pool;
>
>    create_child_pool(&thread_pool, global_root_pool);
>
>    do_thread_stuff();
>
>    cleanup_child_pool(thread_pool);
>
>    if (apr_thread_exit_makes_sense_to_call_from_inside_worker_function) {
>       int rv = APR_SUCCESS;
>       apr_thread_exit(&rv);
>    }
>
>    return NULL;
> }
>
> What I mean by this gibberish is that the pool-handling code can exist
> completely outside of our thread_creation-type functions in APR. Is there
> any reason for APR threads to do more than this (that is, just simple thread
> creation)?

We have hit an impass in my mind.  Dean and I are saying that having each
thread have it's own pool is a requirement.  Not just for httpd, but for
anything using pools.  Dean, if I am mis-interpreting you, then I am
sorry, and please correct me.

Aaron,  you disagree.  you want each app to make that decision on their
own.  I see no way to implement that in a thread-safe manner.

So, I would suggest that we approach this problem in another way.  My ONLY
goal for APR is to make a library that is useful for cross-platform
development and eases the pain of cross-platform development for
developers.  That was the goal when I created APR, and my original test
case was the Apache web server.

I believe that the problem is that the threaded code is creating the
pool, and not advertising it to the thread itself.  This is an easy thing
to fix.  I do not agree that every APR app that uses threads should have
to create their own thread pools.  That is wasted effort by every APR app.

I would like to see a conclusion to this thread quickly.  So, could people
please post their desires quickly, and what they want to see from this.

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

On Tue, Jul 17, 2001 at 01:29:47AM -0700, dean gaudet wrote:
> On Sun, 15 Jul 2001, Sander Striker wrote:
> 
> > Why are we so desperate in opting out the child-pool creation?
> > I don't really have problems with a child pool for each thread. Actually,
> > it will make the dynamic locking a lot easier to implement if it stays.
> 
> all threads MUST have their own private pool root.
> 
> otherwise you're just throwing scalability away in locks.  (which is
> proved by the claim i saw that ian got better performance by defining
> ALLOC_USE_MALLOC on an 8-way.)

I totally agree, but only as a solution in httpd.


I also believe that we should provide this [application-specific requirement]
outside of the basic thread support in APR.

Please allow me to use pseudocode:

void * worker_function(void * opaque_application_data) {

   apr_pool_t *thread_pool;

   create_child_pool(&thread_pool, global_root_pool);

   do_thread_stuff();

   cleanup_child_pool(thread_pool);

   if (apr_thread_exit_makes_sense_to_call_from_inside_worker_function) {
      int rv = APR_SUCCESS;
      apr_thread_exit(&rv);
   }

   return NULL;
}

What I mean by this gibberish is that the pool-handling code can exist
completely outside of our thread_creation-type functions in APR. Is there
any reason for APR threads to do more than this (that is, just simple thread
creation)?

-aaron

Re: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

On Tue, Jul 17, 2001 at 01:29:47AM -0700, dean gaudet wrote:
> On Sun, 15 Jul 2001, Sander Striker wrote:
> 
> > Why are we so desperate in opting out the child-pool creation?
> > I don't really have problems with a child pool for each thread. Actually,
> > it will make the dynamic locking a lot easier to implement if it stays.
> 
> all threads MUST have their own private pool root.
> 
> otherwise you're just throwing scalability away in locks.  (which is
> proved by the claim i saw that ian got better performance by defining
> ALLOC_USE_MALLOC on an 8-way.)

I totally agree, but only as a solution in httpd.


I also believe that we should provide this [application-specific requirement]
outside of the basic thread support in APR.

Please allow me to use pseudocode:

void * worker_function(void * opaque_application_data) {

   apr_pool_t *thread_pool;

   create_child_pool(&thread_pool, global_root_pool);

   do_thread_stuff();

   cleanup_child_pool(thread_pool);

   if (apr_thread_exit_makes_sense_to_call_from_inside_worker_function) {
      int rv = APR_SUCCESS;
      apr_thread_exit(&rv);
   }

   return NULL;
}

What I mean by this gibberish is that the pool-handling code can exist
completely outside of our thread_creation-type functions in APR. Is there
any reason for APR threads to do more than this (that is, just simple thread
creation)?

-aaron

RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[rb...@covalent.net]

On Sun, 15 Jul 2001, Sander Striker wrote:

> > Fair enough. It's just that in order to opt-out of the child-pool creating
> > process in apr_thread_create, we're going to have to add a parameter
>
> Why are we so desperate in opting out the child-pool creation?
> I don't really have problems with a child pool for each thread. Actually,
> it will make the dynamic locking a lot easier to implement if it stays.

The reality is that it should stay.  The problem we have right now, is
that httpd creates a thread-pool, and then APR creates one.  So, if httpd
didn't create one, and just used the one APR created automatically, this
problem would go away completely.

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

Re: request_rec question

[rb...@covalent.net]

On Tue, 17 Jul 2001, Ian Holsman wrote:

> rbb@covalent.net wrote:
>
> >On Tue, 17 Jul 2001, Brian Pane wrote:
> >
> >>Is there anything that actually uses the
> >>headers_out or err_headers_out fields in
> >>the request_rec that's created for a subrequest?
> >>Anything written to these fields appears to
> >>be discarded upon completion of the subrequest.
> >>We could save a couple of apr_table_make calls
> >>per subrequest if the httpd didn't have to
> >>create these two tables.
> >>
> >
> >You can't save those calls, because that would require every module in the
> >world to check to see if the table had been made before it tried to add
> >something to it.  Remember that a sub-request goes through the same basic
> >logic as a real request.
> >
>
> so should we fix the code so that the headers_out/err_headers_out of a
> subrequest gets merged into
> the parents request?

Nope.  While a sub-request goes through the same logic, the headers from a
sub-request do not always apply equally to the main-line request.  I can
easily an instance where an output header on a sub-request, say Location,
doesn't apply to the main request.

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

Re: request_rec question

[Ian Holsman <ia...@cnet.com>]

rbb@covalent.net wrote:

>On Tue, 17 Jul 2001, Brian Pane wrote:
>
>>Is there anything that actually uses the
>>headers_out or err_headers_out fields in
>>the request_rec that's created for a subrequest?
>>Anything written to these fields appears to
>>be discarded upon completion of the subrequest.
>>We could save a couple of apr_table_make calls
>>per subrequest if the httpd didn't have to
>>create these two tables.
>>
>
>You can't save those calls, because that would require every module in the
>world to check to see if the table had been made before it tried to add
>something to it.  Remember that a sub-request goes through the same basic
>logic as a real request.
>

so should we fix the code so that the headers_out/err_headers_out of a 
subrequest gets merged into
the parents request?

>
>
>Ryan
>
>_____________________________________________________________________________
>Ryan Bloom                        	rbb@apache.org
>Covalent Technologies			rbb@covalent.net
>-----------------------------------------------------------------------------
>

Re: request_rec question

[rb...@covalent.net]

On Tue, 17 Jul 2001, Brian Pane wrote:

> Is there anything that actually uses the
> headers_out or err_headers_out fields in
> the request_rec that's created for a subrequest?
> Anything written to these fields appears to
> be discarded upon completion of the subrequest.
> We could save a couple of apr_table_make calls
> per subrequest if the httpd didn't have to
> create these two tables.

You can't save those calls, because that would require every module in the
world to check to see if the table had been made before it tried to add
something to it.  Remember that a sub-request goes through the same basic
logic as a real request.

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

Re: request_rec question

[Brian Pane <bp...@pacbell.net>]

dean gaudet wrote:

>see modules/mappers/mod_negotiation.c search for "fast redirect".
>
>no comment on how clean this is :)
>
>it's something that should be moved to a core routine.
>
As a side-effect of the table overlays that this code does, it
code can 'promote' anything that another module has stored in
the subrequest's headers_out into the parent request's headers_out.

Is that a bug or a feature?

--Brian

Re: request_rec question

[dean gaudet <dg...@arctic.org>]

see modules/mappers/mod_negotiation.c search for "fast redirect".

no comment on how clean this is :)

it's something that should be moved to a core routine.

-dean

On Tue, 17 Jul 2001, Brian Pane wrote:

> Is there anything that actually uses the
> headers_out or err_headers_out fields in
> the request_rec that's created for a subrequest?
> Anything written to these fields appears to
> be discarded upon completion of the subrequest.
> We could save a couple of apr_table_make calls
> per subrequest if the httpd didn't have to
> create these two tables.
>
> --Brian
>
>
>
>

request_rec question

[Brian Pane <bp...@pacbell.net>]

Is there anything that actually uses the
headers_out or err_headers_out fields in
the request_rec that's created for a subrequest?
Anything written to these fields appears to
be discarded upon completion of the subrequest.
We could save a couple of apr_table_make calls
per subrequest if the httpd didn't have to
create these two tables.

--Brian

RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[dean gaudet <dg...@arctic.org>]

On Sun, 15 Jul 2001, Sander Striker wrote:

> Why are we so desperate in opting out the child-pool creation?
> I don't really have problems with a child pool for each thread. Actually,
> it will make the dynamic locking a lot easier to implement if it stays.

all threads MUST have their own private pool root.

otherwise you're just throwing scalability away in locks.  (which is
proved by the claim i saw that ian got better performance by defining
ALLOC_USE_MALLOC on an 8-way.)

-dean

RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[rb...@covalent.net]

On Sun, 15 Jul 2001, Sander Striker wrote:

> > Fair enough. It's just that in order to opt-out of the child-pool creating
> > process in apr_thread_create, we're going to have to add a parameter
>
> Why are we so desperate in opting out the child-pool creation?
> I don't really have problems with a child pool for each thread. Actually,
> it will make the dynamic locking a lot easier to implement if it stays.

The reality is that it should stay.  The problem we have right now, is
that httpd creates a thread-pool, and then APR creates one.  So, if httpd
didn't create one, and just used the one APR created automatically, this
problem would go away completely.

Ryan

_____________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
Covalent Technologies			rbb@covalent.net
-----------------------------------------------------------------------------

RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[dean gaudet <dg...@arctic.org>]

On Sun, 15 Jul 2001, Sander Striker wrote:

> Why are we so desperate in opting out the child-pool creation?
> I don't really have problems with a child pool for each thread. Actually,
> it will make the dynamic locking a lot easier to implement if it stays.

all threads MUST have their own private pool root.

otherwise you're just throwing scalability away in locks.  (which is
proved by the claim i saw that ian got better performance by defining
ALLOC_USE_MALLOC on an 8-way.)

-dean

RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Sander Striker <st...@apache.org>]

> Fair enough. It's just that in order to opt-out of the child-pool creating
> process in apr_thread_create, we're going to have to add a parameter

Why are we so desperate in opting out the child-pool creation?
I don't really have problems with a child pool for each thread. Actually,
it will make the dynamic locking a lot easier to implement if it stays.

> to the call and that means changing programs that currently using it,
> like httpd.
>
> Would it be nicer to just add new parallel apr_smsthread_create
> [name is not
> important] functions that do this and allow for the opting-out also? I can
> provide a patch for this.

Hmmm, the thought of a double interface doesn't bring out happy feelings
with me I'm afraid.

Sander

RE: Terminating threads in a process, WAS: RE: [PATCH] Problems with MPM threaded

[Sander Striker <st...@apache.org>]

> Fair enough. It's just that in order to opt-out of the child-pool creating
> process in apr_thread_create, we're going to have to add a parameter

Why are we so desperate in opting out the child-pool creation?
I don't really have problems with a child pool for each thread. Actually,
it will make the dynamic locking a lot easier to implement if it stays.

> to the call and that means changing programs that currently using it,
> like httpd.
>
> Would it be nicer to just add new parallel apr_smsthread_create
> [name is not
> important] functions that do this and allow for the opting-out also? I can
> provide a patch for this.

Hmmm, the thought of a double interface doesn't bring out happy feelings
with me I'm afraid.

Sander

Re: Terminiting threads in a process RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

On Sun, Jul 15, 2001 at 06:49:51PM +0200, Luke Kenneth Casson Leighton wrote:
> On Sat, Jul 14, 2001 at 12:40:06PM -0700, Aaron Bannert wrote:
> 
> > APR threads, when created, would now take an additional parameter that
> > is the mechanism (an sms implementation) by which it should create child
> > pools. As it is now, the "pool" that is passed in to apr_thread_create()
> > serves as the "parent" pool for the new thread-specific sms.  If this
> > parameter were null, the apr_thread_create() function would not create
> > a sub-pool or register cleanup routines (which satisfies my requirements).
> 
> if at all possible, the behaviour should match as closely as possible
> the existing situation, when this parameter is NULL, even if the
> current situation has bugs / is not very nice.
> 
> this is mainly so that people do not object to having the behaviour
> of existing code disrupted.

Fair enough. It's just that in order to opt-out of the child-pool creating
process in apr_thread_create, we're going to have to add a parameter
to the call and that means changing programs that currently using it,
like httpd.

Would it be nicer to just add new parallel apr_smsthread_create [name is not
important] functions that do this and allow for the opting-out also? I can
provide a patch for this.

-aaron

Re: Terminiting threads in a process RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

On Sun, Jul 15, 2001 at 06:49:51PM +0200, Luke Kenneth Casson Leighton wrote:
> On Sat, Jul 14, 2001 at 12:40:06PM -0700, Aaron Bannert wrote:
> 
> > APR threads, when created, would now take an additional parameter that
> > is the mechanism (an sms implementation) by which it should create child
> > pools. As it is now, the "pool" that is passed in to apr_thread_create()
> > serves as the "parent" pool for the new thread-specific sms.  If this
> > parameter were null, the apr_thread_create() function would not create
> > a sub-pool or register cleanup routines (which satisfies my requirements).
> 
> if at all possible, the behaviour should match as closely as possible
> the existing situation, when this parameter is NULL, even if the
> current situation has bugs / is not very nice.
> 
> this is mainly so that people do not object to having the behaviour
> of existing code disrupted.

Fair enough. It's just that in order to opt-out of the child-pool creating
process in apr_thread_create, we're going to have to add a parameter
to the call and that means changing programs that currently using it,
like httpd.

Would it be nicer to just add new parallel apr_smsthread_create [name is not
important] functions that do this and allow for the opting-out also? I can
provide a patch for this.

-aaron

Re: Terminiting threads in a process RE: [PATCH] Problems with MPM threaded

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

On Sat, Jul 14, 2001 at 12:40:06PM -0700, Aaron Bannert wrote:

> APR threads, when created, would now take an additional parameter that
> is the mechanism (an sms implementation) by which it should create child
> pools. As it is now, the "pool" that is passed in to apr_thread_create()
> serves as the "parent" pool for the new thread-specific sms.  If this
> parameter were null, the apr_thread_create() function would not create
> a sub-pool or register cleanup routines (which satisfies my requirements).

if at all possible, the behaviour should match as closely as possible
the existing situation, when this parameter is NULL, even if the
current situation has bugs / is not very nice.

this is mainly so that people do not object to having the behaviour
of existing code disrupted.

luke

Re: Terminiting threads in a process RE: [PATCH] Problems with MPM threaded

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

On Sat, Jul 14, 2001 at 12:40:06PM -0700, Aaron Bannert wrote:

> APR threads, when created, would now take an additional parameter that
> is the mechanism (an sms implementation) by which it should create child
> pools. As it is now, the "pool" that is passed in to apr_thread_create()
> serves as the "parent" pool for the new thread-specific sms.  If this
> parameter were null, the apr_thread_create() function would not create
> a sub-pool or register cleanup routines (which satisfies my requirements).

if at all possible, the behaviour should match as closely as possible
the existing situation, when this parameter is NULL, even if the
current situation has bugs / is not very nice.

this is mainly so that people do not object to having the behaviour
of existing code disrupted.

luke

Re: Terminiting threads in a process RE: [PATCH] Problems with MPM threaded

[dean gaudet <dg...@arctic.org>]

On Sat, 14 Jul 2001, Aaron Bannert wrote:

> On Sat, Jul 14, 2001 at 12:10:30PM -0700, rbb@covalent.net wrote:
> > On Sat, 14 Jul 2001, Sander Striker wrote:
> > > The way I see it, each process has a single pool instance as the parent
> > > for all the threads. Resetting or destroying that pool should effectively
> > > kill all threads. What am I missing?
> >
> > Nothing.
>
> Good, I think we're all in agreement here.

no, this thread is long and i haven't read it, but i'm not at all in
agreement.  in fact i'll veto outright any change requiring async
notification.

i furthermore veto any change requiring more mutexes in pools:  if you
guys weren't paying attention the last two weeks when this performance
problem was pointed out then i suggest going back and re-reading many of
the threads i contributed to.

maybe i'm just missing the point, i'll go try to read this thread now.

-dean

Re: Terminiting threads in a process RE: [PATCH] Problems with MPM threaded

[dean gaudet <dg...@arctic.org>]

On Sat, 14 Jul 2001, Aaron Bannert wrote:

> On Sat, Jul 14, 2001 at 12:10:30PM -0700, rbb@covalent.net wrote:
> > On Sat, 14 Jul 2001, Sander Striker wrote:
> > > The way I see it, each process has a single pool instance as the parent
> > > for all the threads. Resetting or destroying that pool should effectively
> > > kill all threads. What am I missing?
> >
> > Nothing.
>
> Good, I think we're all in agreement here.

no, this thread is long and i haven't read it, but i'm not at all in
agreement.  in fact i'll veto outright any change requiring async
notification.

i furthermore veto any change requiring more mutexes in pools:  if you
guys weren't paying attention the last two weeks when this performance
problem was pointed out then i suggest going back and re-reading many of
the threads i contributed to.

maybe i'm just missing the point, i'll go try to read this thread now.

-dean

Re: Terminiting threads in a process RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

On Sat, Jul 14, 2001 at 12:10:30PM -0700, rbb@covalent.net wrote:
> On Sat, 14 Jul 2001, Sander Striker wrote:
> > The way I see it, each process has a single pool instance as the parent
> > for all the threads. Resetting or destroying that pool should effectively
> > kill all threads. What am I missing?
> 
> Nothing.

Good, I think we're all in agreement here.

Now let's move on to a discussion of how it is currently implemented in
httpd and APR. As an application developer, I'd like to see APR's threads
more generally applicable (ie. allowing various sms implementations
for each thread's pool, but not necessarily forcing all threads to have
a child pool).

As a [wannabe] contributor to Apache, I'd like to see httpd work. So I
have a proposal:


APR threads, when created, would now take an additional parameter that
is the mechanism (an sms implementation) by which it should create child
pools. As it is now, the "pool" that is passed in to apr_thread_create()
serves as the "parent" pool for the new thread-specific sms.  If this
parameter were null, the apr_thread_create() function would not create
a sub-pool or register cleanup routines (which satisfies my requirements).

Alternatively, the subpool can be already passed into apr_thread_create(),
so it doesn't need to know anything about sms, and it leaves it up
to the parent to take care of cleaning up the thread when the pool
falls out of scope (which is what I was originally proposing).

-aaron

Re: Terminiting threads in a process RE: [PATCH] Problems with MPM threaded

[Aaron Bannert <aa...@ebuilt.com>]

On Sat, Jul 14, 2001 at 12:10:30PM -0700, rbb@covalent.net wrote:
> On Sat, 14 Jul 2001, Sander Striker wrote:
> > The way I see it, each process has a single pool instance as the parent
> > for all the threads. Resetting or destroying that pool should effectively
> > kill all threads. What am I missing?
> 
> Nothing.

Good, I think we're all in agreement here.

Now let's move on to a discussion of how it is currently implemented in
httpd and APR. As an application developer, I'd like to see APR's threads
more generally applicable (ie. allowing various sms implementations
for each thread's pool, but not necessarily forcing all threads to have
a child pool).

As a [wannabe] contributor to Apache, I'd like to see httpd work. So I
have a proposal:


APR threads, when created, would now take an additional parameter that
is the mechanism (an sms implementation) by which it should create child
pools. As it is now, the "pool" that is passed in to apr_thread_create()
serves as the "parent" pool for the new thread-specific sms.  If this
parameter were null, the apr_thread_create() function would not create
a sub-pool or register cleanup routines (which satisfies my requirements).

Alternatively, the subpool can be already passed into apr_thread_create(),
so it doesn't need to know anything about sms, and it leaves it up
to the parent to take care of cleaning up the thread when the pool
falls out of scope (which is what I was originally proposing).

-aaron