You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Charles Reitzel <cr...@rcn.com> on 2002/09/25 19:48:48 UTC
How Do I Create A Per-Worker Pool?
Hi All,
This is a thorny (to me) module development question. I have asked on the
module list and searched the archives, but have found only a partial
answer. Any pointers will be appreciated.
Objective: to create a mutex-free pool per worker in non-MPM-specific way.
I found this exchange in a June posting:
> > Another change we made, as I mentioned in a previous
> > Email, was using non-mutexing per-thread memory
> > pools (HeapCreate(HEAP_NO_SERIALIZE, ...)). > To get
> > best performance with Apache 2 we would really need
> > such a memory pool.
>
>And we already have it! Just do:
>
> apr_allocator_t *allocator;
> apr_allocator_create(&allocator);
> apr_pool_create_ex(&pool, parent_pool, abort_fn, allocator);
> apr_allocator_owner_set(allocator, pool);
>
>Now you have a mutexless allocator associated with a pool. All child pools
>of this pool will use the same allocator and will therefor also have no mutex.
Looks good. But I have questions. What is the correct place to put this
code and where do keep the pool pointer afterwards. I.e. how do you find
the pool from within a module handler?
<comment>
To my mind, the Apache module API is missing an important data structure:
worker_rec. It is obviously redundant (but does no harm) in the prefork
MPM, but is a necessary ingredient in _any_ threaded MPM. Such a structure
would allow modules to be written as thread-safe - independent of the MPM
currently configured by the user. The request record could and should
contain a pointer to the current worker for easy access. In the absence of
a worker_rec, however, some guidance on the above would be helpful.
</comment>
thanks,
Charlie
Re: How Do I Create A Per-Worker Pool?
Posted by Leonardo Javier Bel�n <lb...@afip.gov.ar>.
Thanks for the tip, but is this actually Ok if I try to use it on Apache 1.3
Leo.
----- Original Message -----
From: "Ian Holsman" <ia...@apache.org>
To: <de...@httpd.apache.org>
Sent: Wednesday, September 25, 2002 4:06 PM
Subject: Re: How Do I Create A Per-Worker Pool?
Leonardo Javier Belىn wrote:
> I think I have a question to ask associated to this thread: How can I
> implement a "global pool" of objects. Let me say it better, I want a
global
> pool of database connection, first because I have a limited number of
> connections and second, because I want to be able to dominate the dbms
> accesses. All developed as an "ansi C" apache module, but I thing is
the
> same. I'm running Apache 1.3. Is this the same thing? Leonardo Javier
Belen.
> AFIP-AR
Have a look at apr_reslist.c in the apr-util/misc directory.
>
> ----- Original Message -----
> From: <rb...@apache.org>
> To: <de...@httpd.apache.org>
> Sent: Wednesday, September 25, 2002 3:29 PM
> Subject: Re: How Do I Create A Per-Worker Pool?
>
>
>
>>On Wed, 25 Sep 2002, Charles Reitzel wrote:
>>
>>
>>>Hi All,
>>>
>>>This is a thorny (to me) module development question. I have asked
on
>>
>>the
>>
>>>module list and searched the archives, but have found only a partial
>>>answer. Any pointers will be appreciated.
>>>
>>>Objective: to create a mutex-free pool per worker in non-MPM-specific
>>
>>way.
>>
>>>I found this exchange in a June posting:
>>>
>>>>>Another change we made, as I mentioned in a previous
>>>>>Email, was using non-mutexing per-thread memory
>>>>>pools (HeapCreate(HEAP_NO_SERIALIZE, ...)). > To get
>>>>>best performance with Apache 2 we would really need
>>>>>such a memory pool.
>>>>
>>>>And we already have it! Just do:
>>>>
>>>> apr_allocator_t *allocator;
>>>> apr_allocator_create(&allocator);
>>>> apr_pool_create_ex(&pool, parent_pool, abort_fn, allocator);
>>>> apr_allocator_owner_set(allocator, pool);
>>>>
>>>>Now you have a mutexless allocator associated with a pool. All child
>>
>>pools
>>
>>>>of this pool will use the same allocator and will therefor also have
>>
>>no mutex.
>>
>>>Looks good. But I have questions. What is the correct place to put
>>
>>this
>>
>>>code and where do keep the pool pointer afterwards. I.e. how do you
>>
>>find
>>
>>>the pool from within a module handler?
>>>
>>><comment>
>>>To my mind, the Apache module API is missing an important data
>>
>>structure:
>>
>>>worker_rec. It is obviously redundant (but does no harm) in the
>>
>>prefork
>>
>>>MPM, but is a necessary ingredient in _any_ threaded MPM. Such a
>>
>>structure
>>
>>>would allow modules to be written as thread-safe - independent of the
>>
>>MPM
>>
>>>currently configured by the user. The request record could and
should
>>
>>>contain a pointer to the current worker for easy access. In the
>>
>>absence of
>>
>>>a worker_rec, however, some guidance on the above would be helpful.
>>></comment>
>>
>>The structures in Apache are designed around an HTTP request, not the
>>entity that is running the request. Every threaded MPM has a pool for
>>each thread, which does allow for thread-safe programming.
>>
>>Why exactly would you want a worker_rec? What problem are you having?
>>
>>Ryan
>>
>>______________________________________________________________________
__
>>_______
>>Ryan Bloom rbb@apache.org
>>550 Jean St
>>Oakland CA 94610
>>----------------------------------------------------------------------
--
>>-------
>
>
Re: How Do I Create A Per-Worker Pool?
Posted by Leonardo Javier Bel�n <lb...@afip.gov.ar>.
I was wondering that point (and thatr's what I asked Ian). So, the way is to
move to A2, but is it reliable as the 1.3 version, I mean is as stable as
the older brother, or is it under development? I heard it is rock solid, but
I coudnt check.
Thanks in advance. Leo.
----- Original Message -----
From: <rb...@apache.org>
To: <de...@httpd.apache.org>
Sent: Wednesday, September 25, 2002 4:13 PM
Subject: Re: How Do I Create A Per-Worker Pool?
> This message uses a character set that is not supported by the Internet
> Service. To view the original message content, open the attached
> message. If the text doesn't display correctly, save the attachment to
> disk, and then open it using a viewer that can display the original
> character set. <<message.txt>>
>
Re: How Do I Create A Per-Worker Pool?
Posted by rb...@apache.org.
On Wed, 25 Sep 2002, Ian Holsman wrote:
> Leonardo Javier Bel�n wrote:
> > I think I have a question to ask associated to this thread: How can I
> > implement a "global pool" of objects. Let me say it better, I want a global
> > pool of database connection, first because I have a limited number of
> > connections and second, because I want to be able to dominate the dbms
> > accesses. All developed as an "ansi C" apache module, but I thing is the
> > same. I'm running Apache 1.3. Is this the same thing? Leonardo Javier Belen.
> > AFIP-AR
>
> Have a look at apr_reslist.c in the apr-util/misc directory.
apr_reslist.c actually won't work in this case. He is trying to use
Apache 1.3, which means that the connections would need to be shared
across processes. In general, it just isn't worth doing a connection pool
with 1.3, because of the overhead of the process model.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
550 Jean St
Oakland CA 94610
-------------------------------------------------------------------------------
Re: How Do I Create A Per-Worker Pool?
Posted by Ian Holsman <ia...@apache.org>.
Leonardo Javier Belىn wrote:
> I think I have a question to ask associated to this thread: How can I
> implement a "global pool" of objects. Let me say it better, I want a global
> pool of database connection, first because I have a limited number of
> connections and second, because I want to be able to dominate the dbms
> accesses. All developed as an "ansi C" apache module, but I thing is the
> same. I'm running Apache 1.3. Is this the same thing? Leonardo Javier Belen.
> AFIP-AR
Have a look at apr_reslist.c in the apr-util/misc directory.
>
> ----- Original Message -----
> From: <rb...@apache.org>
> To: <de...@httpd.apache.org>
> Sent: Wednesday, September 25, 2002 3:29 PM
> Subject: Re: How Do I Create A Per-Worker Pool?
>
>
>
>>On Wed, 25 Sep 2002, Charles Reitzel wrote:
>>
>>
>>>Hi All,
>>>
>>>This is a thorny (to me) module development question. I have asked on
>>
>>the
>>
>>>module list and searched the archives, but have found only a partial
>>>answer. Any pointers will be appreciated.
>>>
>>>Objective: to create a mutex-free pool per worker in non-MPM-specific
>>
>>way.
>>
>>>I found this exchange in a June posting:
>>>
>>>>>Another change we made, as I mentioned in a previous
>>>>>Email, was using non-mutexing per-thread memory
>>>>>pools (HeapCreate(HEAP_NO_SERIALIZE, ...)). > To get
>>>>>best performance with Apache 2 we would really need
>>>>>such a memory pool.
>>>>
>>>>And we already have it! Just do:
>>>>
>>>> apr_allocator_t *allocator;
>>>> apr_allocator_create(&allocator);
>>>> apr_pool_create_ex(&pool, parent_pool, abort_fn, allocator);
>>>> apr_allocator_owner_set(allocator, pool);
>>>>
>>>>Now you have a mutexless allocator associated with a pool. All child
>>
>>pools
>>
>>>>of this pool will use the same allocator and will therefor also have
>>
>>no mutex.
>>
>>>Looks good. But I have questions. What is the correct place to put
>>
>>this
>>
>>>code and where do keep the pool pointer afterwards. I.e. how do you
>>
>>find
>>
>>>the pool from within a module handler?
>>>
>>><comment>
>>>To my mind, the Apache module API is missing an important data
>>
>>structure:
>>
>>>worker_rec. It is obviously redundant (but does no harm) in the
>>
>>prefork
>>
>>>MPM, but is a necessary ingredient in _any_ threaded MPM. Such a
>>
>>structure
>>
>>>would allow modules to be written as thread-safe - independent of the
>>
>>MPM
>>
>>>currently configured by the user. The request record could and should
>>
>>>contain a pointer to the current worker for easy access. In the
>>
>>absence of
>>
>>>a worker_rec, however, some guidance on the above would be helpful.
>>></comment>
>>
>>The structures in Apache are designed around an HTTP request, not the
>>entity that is running the request. Every threaded MPM has a pool for
>>each thread, which does allow for thread-safe programming.
>>
>>Why exactly would you want a worker_rec? What problem are you having?
>>
>>Ryan
>>
>>________________________________________________________________________
>>_______
>>Ryan Bloom rbb@apache.org
>>550 Jean St
>>Oakland CA 94610
>>------------------------------------------------------------------------
>>-------
>
>
Re: How Do I Create A Per-Worker Pool?
Posted by Leonardo Javier Bel�n <lb...@afip.gov.ar>.
I think I have a question to ask associated to this thread: How can I
implement a "global pool" of objects. Let me say it better, I want a global
pool of database connection, first because I have a limited number of
connections and second, because I want to be able to dominate the dbms
accesses. All developed as an "ansi C" apache module, but I thing is the
same. I'm running Apache 1.3. Is this the same thing? Leonardo Javier Belen.
AFIP-AR
----- Original Message -----
From: <rb...@apache.org>
To: <de...@httpd.apache.org>
Sent: Wednesday, September 25, 2002 3:29 PM
Subject: Re: How Do I Create A Per-Worker Pool?
> On Wed, 25 Sep 2002, Charles Reitzel wrote:
>
> > Hi All,
> >
> > This is a thorny (to me) module development question. I have asked on
> the
> > module list and searched the archives, but have found only a partial
> > answer. Any pointers will be appreciated.
> >
> > Objective: to create a mutex-free pool per worker in non-MPM-specific
> way.
> >
> > I found this exchange in a June posting:
> > > > Another change we made, as I mentioned in a previous
> > > > Email, was using non-mutexing per-thread memory
> > > > pools (HeapCreate(HEAP_NO_SERIALIZE, ...)). > To get
> > > > best performance with Apache 2 we would really need
> > > > such a memory pool.
> > >
> > >And we already have it! Just do:
> > >
> > > apr_allocator_t *allocator;
> > > apr_allocator_create(&allocator);
> > > apr_pool_create_ex(&pool, parent_pool, abort_fn, allocator);
> > > apr_allocator_owner_set(allocator, pool);
> > >
> > >Now you have a mutexless allocator associated with a pool. All child
> pools
> > >of this pool will use the same allocator and will therefor also have
> no mutex.
> >
> > Looks good. But I have questions. What is the correct place to put
> this
> > code and where do keep the pool pointer afterwards. I.e. how do you
> find
> > the pool from within a module handler?
> >
> > <comment>
> > To my mind, the Apache module API is missing an important data
> structure:
> > worker_rec. It is obviously redundant (but does no harm) in the
> prefork
> > MPM, but is a necessary ingredient in _any_ threaded MPM. Such a
> structure
> > would allow modules to be written as thread-safe - independent of the
> MPM
> > currently configured by the user. The request record could and should
>
> > contain a pointer to the current worker for easy access. In the
> absence of
> > a worker_rec, however, some guidance on the above would be helpful.
> > </comment>
>
> The structures in Apache are designed around an HTTP request, not the
> entity that is running the request. Every threaded MPM has a pool for
> each thread, which does allow for thread-safe programming.
>
> Why exactly would you want a worker_rec? What problem are you having?
>
> Ryan
>
> ________________________________________________________________________
> _______
> Ryan Bloom rbb@apache.org
> 550 Jean St
> Oakland CA 94610
> ------------------------------------------------------------------------
> -------
Re: How Do I Create A Per-Worker Pool?
Posted by rb...@apache.org.
On Wed, 25 Sep 2002, Charles Reitzel wrote:
> Hi All,
>
> This is a thorny (to me) module development question. I have asked on the
> module list and searched the archives, but have found only a partial
> answer. Any pointers will be appreciated.
>
> Objective: to create a mutex-free pool per worker in non-MPM-specific way.
>
> I found this exchange in a June posting:
> > > Another change we made, as I mentioned in a previous
> > > Email, was using non-mutexing per-thread memory
> > > pools (HeapCreate(HEAP_NO_SERIALIZE, ...)). > To get
> > > best performance with Apache 2 we would really need
> > > such a memory pool.
> >
> >And we already have it! Just do:
> >
> > apr_allocator_t *allocator;
> > apr_allocator_create(&allocator);
> > apr_pool_create_ex(&pool, parent_pool, abort_fn, allocator);
> > apr_allocator_owner_set(allocator, pool);
> >
> >Now you have a mutexless allocator associated with a pool. All child pools
> >of this pool will use the same allocator and will therefor also have no mutex.
>
> Looks good. But I have questions. What is the correct place to put this
> code and where do keep the pool pointer afterwards. I.e. how do you find
> the pool from within a module handler?
>
> <comment>
> To my mind, the Apache module API is missing an important data structure:
> worker_rec. It is obviously redundant (but does no harm) in the prefork
> MPM, but is a necessary ingredient in _any_ threaded MPM. Such a structure
> would allow modules to be written as thread-safe - independent of the MPM
> currently configured by the user. The request record could and should
> contain a pointer to the current worker for easy access. In the absence of
> a worker_rec, however, some guidance on the above would be helpful.
> </comment>
The structures in Apache are designed around an HTTP request, not the
entity that is running the request. Every threaded MPM has a pool for
each thread, which does allow for thread-safe programming.
Why exactly would you want a worker_rec? What problem are you having?
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
550 Jean St
Oakland CA 94610
-------------------------------------------------------------------------------