RE: Pools in threads

[Sander Striker <st...@apache.org>]

>>> Fair enough. It's just that in order to opt-out of the 
>>> child-pool creating
>>> process in apr_thread_create, we're going to have to add a parameter
>> 
>> Why are we so desperate in opting out the child-pool creation?
>> I don't really have problems with a child pool for each thread. 
>> Actually, it will make the dynamic locking a lot easier to implement
>> if it stays.
> 
> The problem is that apr_thread_create is completely bogus right now.
>
> The child thread has no access to that pool in apr_thread_t->pool.
> If you want a pool in the thread, you should be creating inside
> of the thread's function not in apr_thread_create.

I can live with that. It's the registration of the thread with
the parent sms [when/if pools==sms] that has my interest :)
 
> And, apr_thread_exit call should *not* be taking in an apr_thread_t
> because the child doesn't have access to that either.  

Can't we pass in a data structure to the thread function which has
all the required information? The data the user wants to pass in will
need to be in the data structure aswell ofcourse.

> We're enforcing the use of global variables to work around this and 
> that's plain wrong.  -- justin

Yes, I agree.

Sander

RE: DCEthreads (was Re: Pools in threads)

[Sander Striker <st...@apache.org>]

>>> And, it would add 3.5 million lines of code to APR.  
>> 
>> Nah. It would add the lines to add thread cancellation.  The
>> 3.5 million LOC is the _entire_ dce rpc codebase. Only part
>> of that is the dce threads library. And, I believe there are
>> some old dce/rpc team members in our midst who could know how
>> this all fit together. I could be wrong though.
> 
> [warning: tangent -- i changed the subject appropriately]
> 
> I must admit that it sounds tantalizing, but I have a comment and a
> question:
> 
> - It appears that the Free DCE package quoted above is under the GPL
>   and is therefore incompatible with the Apache License.

Yes.  However, I wasn't suggesting using or copying code from the
free dce package.  You can merely examine it and decide whether
implementing thread cancellation in such a way is something we want
in APR.  Then, you write your own implementation.

Furthermore, I would like to compare the Free DCE threads implementation
with the DCE/RPC threads implementation.  Then pick the best design
decisions from both.

> - What is meant by "*emulated* on top of POSIX threads"? What are the
>   tradeoffs, if any, compared to a implementation that is native to
>   the OS?

Sander

DCEthreads (was Re: Pools in threads)

[Aaron Bannert <aa...@ebuilt.com>]

On Mon, Jul 16, 2001 at 09:05:40AM +0200, Sander Striker wrote:
> > On Mon, Jul 16, 2001 at 01:54:14AM +0200, Luke Kenneth Casson 
> > Leighton wrote:
> > > ah, yes, but it _is_ supported by DCEthreads - see
> > > http://sourceforge.net/projects/freedce which provides,
> > > horror-of-horrors, thread cancellation *emulated* on
> > > top of POSIX threads.
> > [snip, snip]
> > > ... but what i am basically saying is, on NT and Unix,
> > > thread cancellation _is_ possible, it's just that i
> > > think you might not be too happy about the coding-route
> > > you might have to take to _do_ it :) :)
> > 
> > Possible, but not robust.  
> 
> DCE/RPC is _very_ robust. That being the case its underlying
> mechanisms are very likely to be robust too.
> 
> > And, it would add 3.5 million lines of code to APR.  
> 
> Nah. It would add the lines to add thread cancellation.  The
> 3.5 million LOC is the _entire_ dce rpc codebase. Only part
> of that is the dce threads library. And, I believe there are
> some old dce/rpc team members in our midst who could know how
> this all fit together. I could be wrong though.

[warning: tangent -- i changed the subject appropriately]

I must admit that it sounds tantalizing, but I have a comment and a
question:

- It appears that the Free DCE package quoted above is under the GPL
  and is therefore incompatible with the Apache License.

- What is meant by "*emulated* on top of POSIX threads"? What are the
  tradeoffs, if any, compared to a implementation that is native to
  the OS?

-aaron

Re: Pools in threads

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

> Nah. It would add the lines to add thread cancellation.  The
> 3.5 million LOC is the _entire_ dce rpc codebase. 

which includes about 15 separate services such as DCE/DFS,
which is actually AFS [without the bugfixes and with a
new RPC mechanism]

which you don't need, as sander says.

> Only part
> of that is the dce threads library. And, I believe there are
> some old dce/rpc team members in our midst who could know how
> this all fit together. I could be wrong though.

i'll ask.  might get a response tomorrow.

luke

RE: Pools in threads

[Sander Striker <st...@apache.org>]

> On Mon, Jul 16, 2001 at 01:54:14AM +0200, Luke Kenneth Casson 
> Leighton wrote:
> > ah, yes, but it _is_ supported by DCEthreads - see
> > http://sourceforge.net/projects/freedce which provides,
> > horror-of-horrors, thread cancellation *emulated* on
> > top of POSIX threads.
> [snip, snip]
> > ... but what i am basically saying is, on NT and Unix,
> > thread cancellation _is_ possible, it's just that i
> > think you might not be too happy about the coding-route
> > you might have to take to _do_ it :) :)
> 
> Possible, but not robust.  

DCE/RPC is _very_ robust. That being the case its underlying
mechanisms are very likely to be robust too.

> And, it would add 3.5 million lines of code to APR.  

Nah. It would add the lines to add thread cancellation.  The
3.5 million LOC is the _entire_ dce rpc codebase. Only part
of that is the dce threads library. And, I believe there are
some old dce/rpc team members in our midst who could know how
this all fit together. I could be wrong though.

Sander

Re: Pools in threads

[Justin Erenkrantz <je...@ebuilt.com>]

On Mon, Jul 16, 2001 at 01:54:14AM +0200, Luke Kenneth Casson Leighton wrote:
> ah, yes, but it _is_ supported by DCEthreads - see
> http://sourceforge.net/projects/freedce which provides,
> horror-of-horrors, thread cancellation *emulated* on
> top of POSIX threads.
[snip, snip]
> ... but what i am basically saying is, on NT and Unix,
> thread cancellation _is_ possible, it's just that i
> think you might not be too happy about the coding-route
> you might have to take to _do_ it :) :)

Possible, but not robust.  And, it would add 3.5 million lines 
of code to APR.  Well, we could the APROS in that many LOC.  =)
Kind of like Quake - have an OS inside of your web-server.

Thread cancellation just opens up a can of worms not because the OS
doesn't support them, but because their semantics are ill-defined.
We'd have to add cancellation points to our code.  Not going to
happen.  Well, I'm not going to do it.  I might even veto it out 
of pure spite.  =)  -- justin

Re: Pools in threads

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

On Sun, Jul 15, 2001 at 04:43:37PM -0700, Justin Erenkrantz wrote:
> On Mon, Jul 16, 2001 at 12:46:31AM +0200, Luke Kenneth Casson Leighton wrote:
> > > In order to provide a win against the current pool code in a threaded
> > > MPM, we *need* to have thread-specific SMS that have no locks or
> > > association to anything other than a simple unlocked (from APR's
> > > perspective) malloc/free (aka std) SMS.  -- justin
> > 
> > okay.
> > 
> > well... uhmmm... this is going to sound odd.  i'm not even sure
> > if it will help, because i am a bit out of my depth in understanding
> > the problem.
> > 
> > how about a 'pass-through' sms for threads?
> 
> I'm not sure I'm following you.  The thread SMS never needs locks.
 
i am not necessarily referring to locks.

if there is something _ever_ so slightly different about the
circumstances under which sms instances need to be used, then
what i am suggesting is that instead of doing this:

apr_sms_general_t
apr_sms_thread_capable_general_t
[which was what you were recommending ;)]
apr_sms_trivial_t
apr_sms_thread_capable_trivial_t

instead having a:
apr_sms_thread_wrapping_redirector_t

and use *that* to do the thread-capable-very-specific-things
that i suspect you will need to do for all-and-any SMSes,
pretty much, and that might not necessarily be locking,
which simplifies the coding needed.

> Sander was under the assumption that the parent of a thread can clean up
> a thread.  That isn't supported with POSIX pthreads (cleanly).
> Apparently, Win32 has some support for this.  This lack basically kills
> off the parent/child relationship between SMSes across threads.
 
ah, yes, but it _is_ supported by DCEthreads - see
http://sourceforge.net/projects/freedce which provides,
horror-of-horrors, thread cancellation *emulated* on
top of POSIX threads.

DCEthreads is an implementation of POSIX (draft 4) threads
which of course drastically changed from the final
release.

now, if you examine Windows NT threads, i _bet_ they
support thread cancellation properly, because they
do support thread cancellation via dce/rpc (and dcom,
which is basically an object-orientated interface on
top of dce/rpc).

why?  because, well, that's a long story.

also, take a lok at www.opengroup.org/dce, download the
88mb of the dce 1.22 source code, unpack it to its 430mb
uncompressed size [no, i'm not kidding: yes, that's right,
four hundred and thirty megabytes - 3.5 million lines of code,
i had to use xargs to do wc on them], and you will find that there is
DCEthreads supported for AIX, HP/UX, Solaris, DG/UX, SGI's Unix
(forgotten what it is!  IRIX *duh* :)

... but what i am basically saying is, on NT and Unix,
thread cancellation _is_ possible, it's just that i
think you might not be too happy about the coding-route
you might have to take to _do_ it :) :)


> pthread_cancel() gets us into all sorts of problems we don't want to 
> deal with.  It has been discussed years ago between Ryan, Dean and 
> Manoj - see around 08-1999 - "First in a long line of APR related 
> patches".  And, it isn't guaranteed to work, either (mutex acquires 
> are *not* interruptable).  In that same thread, Ryan also brings up a
> precursor to SMS.  Rather interesting thread, really.  (See
> www.apachelabs.org if you don't have a favorite new-httpd archive.)
> 
> Unless you have a clean way of killing off a thread from a parent,
> thread-based SMS must be directly parented from a basic non-locking 
> SMS (like std SMS).  
> 
> I'm open to any creative suggestions that work.  I'm just not aware 
> of any that work.  -- justin

:)

Re: Pools in threads

[Justin Erenkrantz <je...@ebuilt.com>]

On Mon, Jul 16, 2001 at 12:46:31AM +0200, Luke Kenneth Casson Leighton wrote:
> > In order to provide a win against the current pool code in a threaded
> > MPM, we *need* to have thread-specific SMS that have no locks or
> > association to anything other than a simple unlocked (from APR's
> > perspective) malloc/free (aka std) SMS.  -- justin
> 
> okay.
> 
> well... uhmmm... this is going to sound odd.  i'm not even sure
> if it will help, because i am a bit out of my depth in understanding
> the problem.
> 
> how about a 'pass-through' sms for threads?

I'm not sure I'm following you.  The thread SMS never needs locks.

Sander was under the assumption that the parent of a thread can clean up
a thread.  That isn't supported with POSIX pthreads (cleanly).
Apparently, Win32 has some support for this.  This lack basically kills
off the parent/child relationship between SMSes across threads.

pthread_cancel() gets us into all sorts of problems we don't want to 
deal with.  It has been discussed years ago between Ryan, Dean and 
Manoj - see around 08-1999 - "First in a long line of APR related 
patches".  And, it isn't guaranteed to work, either (mutex acquires 
are *not* interruptable).  In that same thread, Ryan also brings up a
precursor to SMS.  Rather interesting thread, really.  (See
www.apachelabs.org if you don't have a favorite new-httpd archive.)

Unless you have a clean way of killing off a thread from a parent,
thread-based SMS must be directly parented from a basic non-locking 
SMS (like std SMS).  

I'm open to any creative suggestions that work.  I'm just not aware 
of any that work.  -- justin

Re: Pools in threads

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

> In order to provide a win against the current pool code in a threaded
> MPM, we *need* to have thread-specific SMS that have no locks or
> association to anything other than a simple unlocked (from APR's
> perspective) malloc/free (aka std) SMS.  -- justin

okay.

well... uhmmm... this is going to sound odd.  i'm not even sure
if it will help, because i am a bit out of my depth in understanding
the problem.

how about a 'pass-through' sms for threads?

all it does is to create an apr_sms_thread_passthrough() is:

memcpy(&thr_pthru, &someothersms_api);
then OVERRIDE the create function, calling the *someothersms*->create.

you then do your 'thread-specific' stuff that you need to do,
e.g. locking _whatever_, i really don't know, as a wrapper around
the someothersms.

in other words, via this technique, you can turn a non-thread-capable
SMS into a thread-safe one.

... does that sound a bit mad, but useful, or am i way off the mark.

luke

Re: Pools in threads

[Justin Erenkrantz <je...@ebuilt.com>]

On Sun, Jul 15, 2001 at 07:52:46PM +0200, Sander Striker wrote:
> Can't we pass in a data structure to the thread function which has
> all the required information? The data the user wants to pass in will
> need to be in the data structure aswell ofcourse.

-0.9.  I don't like the idea of marshaling the data to the thread's
worker function.

I'm seeing the proposed marshaling like so:

struct {
 void *data;
 apr_pool_t *pool;
} apr_thread_starter_t;

void * worker_function(void *v)
{
 apr_thread_starter *ts = v;
 whatever_the_thread_really_wants_t *d = ts->data;
 ...
}

apr_thread_create(blah, blah, blah)
{
  apr_thread_starter *ts = apr_palloc(...);
  ts->data = the-data-that-user-really-wants-to-pass-in;  
  ts->sms = apr_pool_create(blah, blah, blah);

  thread_create(go call worker_function with ts as the sole argument)
}

Yuck.  

Since I'm seeing the thread SMS be completely separate from the parent
SMS, I'm just not sure why we even want to pass the sms along.  

Unless you can make a case that the thread's SMS needs to know about the
parent SMS.  If so, then we've got major problems in our SMS design.

In order to provide a win against the current pool code in a threaded
MPM, we *need* to have thread-specific SMS that have no locks or
association to anything other than a simple unlocked (from APR's
perspective) malloc/free (aka std) SMS.  -- justin