mod_cache_shared?

[Paul Querna <ch...@force-elite.com>]

I have been thinking about some ideas for allowing multiple 
mod_disk_cache instances to share their cache.

There are existing 'solutions' to 'this' problem:
   - HTCP, RFC 2756, http://www.htcp.org/
   - ICP, RFC 2186 and RFC 2187, http://icp.ircache.net/
   - CARP, http://icp.ircache.net/carp.txt

I've only done a quick look at each, and they all seem to be overly 
complicated.  If anyone has experience with these or other methods, 
please speak up :)

HTCP seems like the best pre-designed solution at this time.  It doesn't 
look like it got much adoption from any other proxy/cache servers. 
Anyone know why?

But, the 'lets do this quickly and make it work' part of me says, we can 
design something very simple and 'stupid', that uses the knowledge of 
how mod_cache works.

What we would do is create a new mod_cache backend, 'mod_cache_shared'.

It would run in the Quick Handler, after the other cache methods.  This 
mod_cache_shared would use some communication protocol to request this 
document from its peers.

If no peer replied in 5 ms, we would DECLINE the quick handler, and 
continue on the normal path of a cache miss. Of course, giving-up 
timeout would be configurable.

If a peer replied with 'yes we have this', we would fetch it from there, 
and *also* insert the required output filters, to add it to our local cache.

Thoughts?

-Paul

Re: mod_cache_shared?

[Brian Akins <ba...@web.turner.com>]

Justin Erenkrantz wrote:

> mod_disk_cache should work over NFS.  =)  -- justin
> 

NO!!! ;)


-- 
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: mod_cache_shared?

[Justin Erenkrantz <ju...@erenkrantz.com>]

On 3/11/06, Paul Querna <ch...@force-elite.com> wrote:
> But, the 'lets do this quickly and make it work' part of me says, we can
> design something very simple and 'stupid', that uses the knowledge of
> how mod_cache works.

mod_disk_cache should work over NFS.  =)  -- justin

Re: mod_cache_shared?

[Ian Holsman <li...@holsman.net>]

My only thoughts is that if you could integrate with ICP (which squid 
uses) it might make it easier to get mod_cache_shared installed, tested, 
and used in large organizations who already use a caching solution.

That way they could put a single mod_cache_shared into their existing 
pool and see how it works/compares to squid, and then slowly move their 
existing base to mod_cache_shared if it is better.

Inventing yet another protocol is not going to help people switch over. 
(and it is bound to get complex quickly as all the boundary cases get 
added into the protocol)

regards
Ian

(same thing can be applied to HTCP or CARP.. I'm protocol agnostic when 
it comes to choosing, but -0.5 on inventing another one)

Paul Querna wrote:
> I have been thinking about some ideas for allowing multiple 
> mod_disk_cache instances to share their cache.
> 
> There are existing 'solutions' to 'this' problem:
>   - HTCP, RFC 2756, http://www.htcp.org/
>   - ICP, RFC 2186 and RFC 2187, http://icp.ircache.net/
>   - CARP, http://icp.ircache.net/carp.txt
> 
> I've only done a quick look at each, and they all seem to be overly 
> complicated.  If anyone has experience with these or other methods, 
> please speak up :)
> 
> HTCP seems like the best pre-designed solution at this time.  It doesn't 
> look like it got much adoption from any other proxy/cache servers. 
> Anyone know why?
> 
> But, the 'lets do this quickly and make it work' part of me says, we can 
> design something very simple and 'stupid', that uses the knowledge of 
> how mod_cache works.
> 
> What we would do is create a new mod_cache backend, 'mod_cache_shared'.
> 
> It would run in the Quick Handler, after the other cache methods.  This 
> mod_cache_shared would use some communication protocol to request this 
> document from its peers.
> 
> If no peer replied in 5 ms, we would DECLINE the quick handler, and 
> continue on the normal path of a cache miss. Of course, giving-up 
> timeout would be configurable.
> 
> If a peer replied with 'yes we have this', we would fetch it from there, 
> and *also* insert the required output filters, to add it to our local 
> cache.
> 
> Thoughts?
> 
> -Paul
> 

Re: mod_cache_shared?

[Brian Akins <ba...@web.turner.com>]

Paul Querna wrote:
> I have been thinking about some ideas for allowing multiple 
> mod_disk_cache instances to share their cache.

Someone may have already suggested this:

If you could stack cache modules here is a solution I have.

Write a mod_cache_memcached (on my todo list).

Configure a cache so that it checks disk first, then memcached.  Have it 
store stuff in memcached.

It is much faster to serve from disk than the network.

This is very similar to how squid does it.


We discussed this on IRC a couple weeks ago.



-- 
Brian Akins
Lead Systems Engineer
CNN Internet Technologies