Re: mod_disk_cache read-while-caching patch

[Niklas Edmundsson <ni...@acc.umu.se>]

On Tue, 2 May 2006, Niklas Edmundsson wrote:

<a lot of things>

>>> In any case the patch is more or less finished, independent testing
>>> and auditing haven't been done yet but I can submit a preliminary
>>> jumbo-patch if people are interested in having a look at it now.
>> 
>> Post it, people can take a look.
>
> OK. It's attached. It has only had mild testing using the worker mpm with 
> mmap enabled, it needs a bit more testing and auditing before trusting it too 
> hard.
>
> Note that this patch fixes a whole slew of other issues along the way, the 
> most notable ones being LFS on 32bit arch, don't eat all your 32bit 
> memory/address space when caching a huge files, provide r->filename so %f in 
> LogFormat works, and other smaller issues.

Now it's a fair bit more complete, and tested quite a bit with the 
worker mpm at least. Most of the time has been spent trying to figure 
out what the apr*-api is doing. For example, when things go bad (a 
client hangs up a connection for example) memory seems to be freed 
before the cleanups are run, the segfault caused by that took quite 
some time to find.

This jumbo-patch is provided for those who wants to see what the 
result is and see the general way I solved things. It still has some 
FIXME's and debug messages. For those who like to try it, note that 
the disk format is changed (one file instead of header and body in 
separate files) so you'll have to clean your httpcache to get to a 
known state. Regarding stability, we trust it enough to deploy it for 
production use.

In general, it solves LFS, read-while-caching and "thundering herd" 
and touches quite a lot of code to get there since I wanted to avoid 
the locking mess.

I'll start breaking this down into smaller patches, but given the 
recent cache-cleanup-effort I'd like to know how to do it.

* Should I provide patches against httpd-2.2.2 or trunk?
* Should I just attach them to bug #39380 or post them here first?

I suspect that the cache-cleanup-effort will make some of this 
obsolete, but since mod_disk_cache is a dark hole of no error handling 
and doing checks at weird places I suspect that at least those patches 
will prove useful.


/Nikke - looking forward to see how it survives the Ubuntu Dapper
          release ...
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
  Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se      |     nikke@acc.umu.se
---------------------------------------------------------------------------
  I've seen the procedure hundreds of times. - Qwark
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=