Mod_MEM_cache doesn't use Pools to allocate cache objects???

[luca regini <lu...@gmail.com>]

Taking a look at mod_mem_cache source code i have seen that it doesn't
use pools to allocate cache objects but i does so by means of
reference counting and simple calloc/free calls. I  have also seen
that this module requires a Threaded apr to work. I am wondering the
reasons of this design choices. I am developing a web service caching
module and now i have some issues with caching some web service
parameters correctly. I store these parameters in a per server config
object but it seems that stored data is valid only on a per connection
base. My developement version of Apache 2.0 is compiled in maintainer
mode with a prefork mpm. I would like to contribute the module to the
apache group if i manage to develop something usable.

Thanks in advance.
Luca

Re: Mod_MEM_cache doesn't use Pools to allocate cache objects???

[luca regini <lu...@gmail.com>]

I am going to use shared memory. I found a very good example in the
ssl_scache_shmht.c file. This implementation has everything i need of.
Now i am just working to remove dependencies that this code has on
other mod_ssl files, so to provide a generic
shared memory cache table object. I can release the source code to the
group if there is some interest in it. If i manage to build a generic
solution for the web service caching module i will release this source
code also.

Luca



On Tue, 08 Mar 2005 16:08:58 +0100, Matthieu Estrade
<me...@apache.org> wrote:
> luca regini wrote:
> 
> > Taking a look at mod_mem_cache source code i have seen that it doesn't
> > use pools to allocate cache objects but i does so by means of
> > reference counting and simple calloc/free calls. I  have also seen
> > that this module requires a Threaded apr to work.
> >
> If you use prefork, each child is a fork and has it's own memory space,
> so you are unable to share data between child -> bad idea
> The way you can do is code a cache module using shared memory but in
> this case, shm and rmm are not using pools, and i think there will be
> performances issues.
> 
> It depend on what you want to do with your cache module. Actually, in a
> threaded mpm, each child and the threads inside are sharing common data.
> So you have duplicated cache data in each forked child. The performance
> depend on how you setup your server. low child number and high thread
> number = few request on backend and only few duplicated data.
> 
> Coding a cache module using shared memory is huge work.
> Good luck if you choose to do it
> 
> Matthieu
> 
>

Re: Mod_MEM_cache doesn't use Pools to allocate cache objects???

[Matthieu Estrade <me...@apache.org>]

luca regini wrote:

> Taking a look at mod_mem_cache source code i have seen that it doesn't
> use pools to allocate cache objects but i does so by means of
> reference counting and simple calloc/free calls. I  have also seen
> that this module requires a Threaded apr to work.
>
If you use prefork, each child is a fork and has it's own memory space, 
so you are unable to share data between child -> bad idea
The way you can do is code a cache module using shared memory but in 
this case, shm and rmm are not using pools, and i think there will be 
performances issues.

It depend on what you want to do with your cache module. Actually, in a 
threaded mpm, each child and the threads inside are sharing common data. 
So you have duplicated cache data in each forked child. The performance 
depend on how you setup your server. low child number and high thread 
number = few request on backend and only few duplicated data.

Coding a cache module using shared memory is huge work.
Good luck if you choose to do it

Matthieu

Re: Mod_MEM_cache doesn't use Pools to allocate cache objects???

[Cliff Woolley <jw...@virginia.edu>]

On Tue, 8 Mar 2005, Bill Stoddard wrote:

> luca regini wrote:
> > Taking a look at mod_mem_cache source code i have seen that it doesn't
> > use pools to allocate cache objects but i does so by means of
> > reference counting and simple calloc/free calls. I  have also seen
> > that this module requires a Threaded apr to work. I am wondering the
> > reasons of this design choices.
>
> If you allocate the cache objects out of a pool, what happens to the
> memory when stale objects are garbage collected?

Bad Things(tm).  :)  However, it should be possible to use a custom
apr_allocator_t instead of directly calling calloc/free.  Might be a
little faster?

Re: Mod_MEM_cache doesn't use Pools to allocate cache objects???

[Bill Stoddard <bi...@wstoddard.com>]

luca regini wrote:
> Taking a look at mod_mem_cache source code i have seen that it doesn't
> use pools to allocate cache objects but i does so by means of
> reference counting and simple calloc/free calls. I  have also seen
> that this module requires a Threaded apr to work. I am wondering the
> reasons of this design choices. 

If you allocate the cache objects out of a pool, what happens to the memory when stale objects are garbage 
collected?