Re: mod_cache responsibilities vs mod_xxx_cache provider responsibilities

[Graham Leggett <mi...@sharp.fm>]

On Wed, September 20, 2006 9:50 pm, Ruediger Pluem wrote:

> You can set a max cache file size (CacheMaxFileSize) which prevents
> caching files that are larger then
> a specfic size. This is checked after each bucket is written to the disk.
> If the
> stream is larger then the max file size the file gets deleted and caching
> of this request
> is stopped. So this also works with chunked responses.

Hmmm - this affects the case where another process/thread is delivering
from a still-being-cached entity.

If the lead thread decides to stop, and other threads are following, the
other following threads will deliver CacheMaxFileSize data, and cut the
request short.

One workaround for this problem is to have following threads ignore the
cached entity if the entity does not have a content length - something the
entity will have when caching is complete.

This means the backend server will still see a spike of traffic while the
object is being cached, but the cache will no try and cache multiple
entities until the first one wins, which happens now.

Regards,
Graham
--

Re: mod_cache responsibilities vs mod_xxx_cache provider responsibilities

[Niklas Edmundsson <ni...@acc.umu.se>]

On Thu, 21 Sep 2006, Graham Leggett wrote:

> Hmmm - this affects the case where another process/thread is delivering
> from a still-being-cached entity.
>
> If the lead thread decides to stop, and other threads are following, the
> other following threads will deliver CacheMaxFileSize data, and cut the
> request short.
>
> One workaround for this problem is to have following threads ignore the
> cached entity if the entity does not have a content length - something the
> entity will have when caching is complete.
>
> This means the backend server will still see a spike of traffic while the
> object is being cached, but the cache will no try and cache multiple
> entities until the first one wins, which happens now.

Our patch solves this by pausing read-threads while the object is 
being cached until there is a known length of the body, with a timeout 
to detect if the caching thread has died. Drawback is that you have to 
write the header twice, but that's cheap compared to caching an object 
N times. Seems to do the trick, but it haven't had nearly as much 
pounding as the cache-from-file case (the main use is on a ftp server, 
remember).

/Nikke - currently working on getting a machine up for testing
          the smaller patches before submitting them.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
  Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | nikke@acc.umu.se 
---------------------------------------------------------------------------
  The dog ate my .REP packet.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=