Improving cache coherency [Was: Idea of an apache module]

[Barrie Slaymaker <ba...@slaysys.com>]

aaron wrote:
> 
>  for example, in discussion software you have a very clear moment when you
>  want to invalidate specific pages: when a message arrives. now i don't want
>  squid or any other cache to even check w/ every request. i know darn well
>  when the cache is no longer valid!

I've been mulling over ways to better coordinate between a cache and the
back end web server(s) so that the staleness heuristics aren't needed.

I want a way to affect the state (fresh/stale/deleted) of cached resources.
I've been considering a couple of approaches, here's one I'm fairly happy
with.

Enable the cache to accept POST requests containing commands to expire or
refresh certain items, like so:

  ## Let's mark a few things as STALE
  expire //http_perl/index.html         ## a single item
  expire //http_doc/images/*            ## all resources in a directory
  expire //http_doc/images/**           ## all resources in a directory tree
  expire ** if mime-type =~ image/*     ## all images in all locations
  expire ** if last-modified > now-1day ## Anything too old.

  ## Let's modify the Expires: time for a few things (like renewing
  ## a library book the cache has checked out)
  expire //http_perl/maps/** at 2000/07/31 11:59:59

  ## Let's mark a few things stale and cause an immediate flurry
  ## of if-modified-since requests.
  refresh ** if mime-type =~ image/*
  refresh //*/~** if last-modified > now-1day

  ## Let's mark a few things stale and cause an immediate flurry
  ## of plain old GET requests.  Not sure why you'd want to do this,
  ## but hey...
  reload ** if mime-type =~ image/*
  reload //*/~** if last-modified > now-1day

This would allow me to have a database trigger, file upload or form
mod_perl handler (hey! this might even be on-topic :-), File::Find cron
job, cvs/perforce/... trigger, rsync script, etc. notify the cache that
it might have stale entries.  In many applications it would be better
than setting Expires headers in mod_perl-land.

Wildcards would only match URIs currently in the cache.  Explicit URLs
would (or atleast could) allow cache pre-loading.

Is there anything like this out there?

This is perhaps a bit off-topic for the mod_perl list, but let's say I
implement it as a perl module that cooperates with mod_cache.  Certainly
pieces of it could be used from mod_perl, and perhaps existing caching
systems written for mod_perl might be able to implement it.  Or something.

- Barrie