You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Laurent Perez <ha...@gmail.com> on 2006/01/24 23:16:44 UTC

mod_disk_cache behaviour with Vary: header

Hello

I apologize for posting this to dev@httpd but I think my question is
related to mod_cache module developers, that's why I'm using this
list, after reading bugzilla and source code available on
svn.apache.org.

I'm having troubles whenever I want to use the Vary: header to let
mod_cache (I'm using mod_disk_cache, together with mod_proxy on
apache2.2.0-54, debian packages) handle cached variations based on
that header.

The header I vary on is User-Agent, and when several different agents
hit the cache, I thought mod_cache would create a separate couple from
.header + .data files for each and every cached request. What is
happening is that mod_cache is correctly detecting the agent variation
(LogLevel debug directive reports "Vary header mismatch" from
cache_storage.c), but instead of creating a new hash couple on disk,
it is always rewriting the same .header and .data files.

This reduces interest of the Vary: header, because for example when
two different agents hit mod_cache, the backend webapp proxied by
mod_proxy is queried twice, because the resulting hash variation is
stored in the same files, and not in four different files. Of course
if the same agent hits the cache twice, with no other agent hit
in-between, the correct cached response is obviously well served.

Is it an intentional cache behaviour, for example to avoid too many
files to be created from mod_disk_cache only because of multiple
probable variations ? Or is it a bug/todo feature that's being still
worked on ?

Thanks for any advice

laurent

--
<a href="http://in-pocket.blogspot.com">http://in-pocket.blogspot.com
- Mobile world, technology and more</a>

Re: mod_disk_cache behaviour with Vary: header

Posted by Laurent Perez <ha...@gmail.com>.
Sorry to up this thread again, but another wrong behaviour appeared
after I switched to 2.2 (source from
http://apache.crihan.fr/dist/httpd/httpd-2.2.0.tar.gz) :-(

The varying now works fine, with the .vary folder filled with
variations. But now, the cached files do not seem to be checked on the
disk anymore, so the cache is always being rewritten from backend
webapp queries. I have apache2.0 and apache2.2 on ports 80 and 81,
both proxying the same single webapp, and here's the following ouput I
get from debugs when I make the same request on the two servers :

Apache 2.0.54, first request :
------------------------------------------
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/proxy/proxy_http.c(67):
proxy: HTTP: canonicalising URL //mydomain.com:8080/
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/proxy/mod_proxy.c(418):
Trying to run scheme_handler
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/proxy/proxy_http.c(1075):
proxy: HTTP: serving URL http://mydomain.com:8080/
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/proxy/proxy_http.c(186):
proxy: HTTP connecting http://mydomain.com:8080/ to mydomain.com:8080
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/proxy/proxy_util.c(1139):
proxy: HTTP: fam 2 socket created to connect to mydomain.com
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/proxy/proxy_http.c(336):
proxy: socket is connected
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/proxy/proxy_http.c(370):
proxy: connection complete to 192.168.0.170:8080 (mydomain.com)
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/proxy/proxy_http.c(909):
proxy: start body send
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/experimental/mod_cache.c(556):
cache: Caching url: /
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/experimental/mod_disk_cache.c(645):
disk_cache: Stored headers for URL mydomain.com/?
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/experimental/mod_disk_cache.c(747):
disk_cache: Body for URL mydomain.com/? cached.
[Wed Jan 25 11:16:58 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/proxy/proxy_http.c(969):
proxy: end body send
------------------------------------------
Apache 2.0.54, second request :
------------------------------------------
[Wed Jan 25 11:17:02 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/experimental/mod_disk_cache.c(371):
disk_cache: Recalled cached URL info header mydomain.com/?
[Wed Jan 25 11:17:02 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/experimental/mod_disk_cache.c(502):
disk_cache: Recalled headers for URL mydomain.com/?
[Wed Jan 25 11:17:02 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/experimental/mod_cache.c(220):
cache: running CACHE_OUT filter
[Wed Jan 25 11:17:02 2006] [debug]
/home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/experimental/mod_cache.c(229):
cache: serving /

Clearly, the page is served from the cache.

Apache 2.2.0, first request :
------------------------------------------
[Wed Jan 25 11:19:01 2006] [debug] mod_cache.c(129): Adding CACHE_SAVE
filter for /
[Wed Jan 25 11:19:01 2006] [debug] mod_cache.c(136): Adding
CACHE_REMOVE_URL filter for /
[Wed Jan 25 11:19:01 2006] [debug] mod_proxy_http.c(54): proxy: HTTP:
canonicalising URL //mydomain.com:8080/
[Wed Jan 25 11:19:01 2006] [debug] proxy_util.c(1373): [client
192.168.0.170] proxy: http: found worker http://mydomain.com:8080/ for
http://mydomain.com:8080/
[Wed Jan 25 11:19:01 2006] [debug] mod_proxy.c(736): Running scheme
http handler (attempt 0)
[Wed Jan 25 11:19:01 2006] [debug] mod_proxy_http.c(1661): proxy:
HTTP: serving URL http://mydomain.com:8080/
[Wed Jan 25 11:19:01 2006] [debug] proxy_util.c(1754): proxy: HTTP:
has acquired connection for (mydomain.com)
[Wed Jan 25 11:19:01 2006] [debug] proxy_util.c(1811): proxy:
connecting http://mydomain.com:8080/ to mydomain.com:8080
[Wed Jan 25 11:19:01 2006] [debug] proxy_util.c(1911): proxy:
connected / to mydomain.com:8080
[Wed Jan 25 11:19:01 2006] [debug] proxy_util.c(2005): proxy: HTTP:
fam 2 socket created to connect to mydomain.com
[Wed Jan 25 11:19:01 2006] [debug] proxy_util.c(2101): proxy: HTTP:
connection complete to 192.168.0.170:8080 (mydomain.com)
[Wed Jan 25 11:19:01 2006] [debug] mod_proxy_http.c(1443): proxy:
start body send
[Wed Jan 25 11:19:01 2006] [debug] mod_cache.c(602): cache: Caching url: /
[Wed Jan 25 11:19:01 2006] [debug] mod_cache.c(608): cache: Removing
CACHE_REMOVE_URL filter.
[Wed Jan 25 11:19:01 2006] [debug] mod_disk_cache.c(954): disk_cache:
Stored headers for URL http://_default_:81/?
[Wed Jan 25 11:19:01 2006] [debug] mod_disk_cache.c(1035): disk_cache:
Body for URL http://_default_:81/? cached.
[Wed Jan 25 11:19:01 2006] [debug] mod_proxy_http.c(1530): proxy: end body send
[Wed Jan 25 11:19:01 2006] [debug] proxy_util.c(1769): proxy: HTTP:
has released connection for (mydomain.com)
------------------------------------------
Apache 2.2.0, second request :
------------------------------------------
[Wed Jan 25 11:19:14 2006] [debug] mod_cache.c(129): Adding CACHE_SAVE
filter for /
[Wed Jan 25 11:19:14 2006] [debug] mod_cache.c(136): Adding
CACHE_REMOVE_URL filter for /
[Wed Jan 25 11:19:14 2006] [debug] mod_proxy_http.c(54): proxy: HTTP:
canonicalising URL //mydomain.com:8080/
[Wed Jan 25 11:19:14 2006] [debug] proxy_util.c(1373): [client
192.168.0.170] proxy: http: found worker http://mydomain.com:8080/ for
http://mydomain.com:8080/
[Wed Jan 25 11:19:14 2006] [debug] mod_proxy.c(736): Running scheme
http handler (attempt 0)
[Wed Jan 25 11:19:14 2006] [debug] mod_proxy_http.c(1661): proxy:
HTTP: serving URL http://mydomain.com:8080/
[Wed Jan 25 11:19:14 2006] [debug] proxy_util.c(1754): proxy: HTTP:
has acquired connection for (mydomain.com)
[Wed Jan 25 11:19:14 2006] [debug] proxy_util.c(1811): proxy:
connecting http://mydomain.com:8080/ to mydomain.com:8080
[Wed Jan 25 11:19:14 2006] [debug] proxy_util.c(1911): proxy:
connected / to mydomain.com:8080
[Wed Jan 25 11:19:14 2006] [debug] proxy_util.c(2005): proxy: HTTP:
fam 2 socket created to connect to mydomain.com
[Wed Jan 25 11:19:14 2006] [debug] proxy_util.c(2101): proxy: HTTP:
connection complete to 192.168.0.170:8080 (mydomain.com)
[Wed Jan 25 11:19:14 2006] [debug] mod_proxy_http.c(1443): proxy:
start body send
[Wed Jan 25 11:19:14 2006] [debug] mod_cache.c(602): cache: Caching url: /
[Wed Jan 25 11:19:14 2006] [debug] mod_cache.c(608): cache: Removing
CACHE_REMOVE_URL filter.
[Wed Jan 25 11:19:14 2006] [debug] mod_disk_cache.c(954): disk_cache:
Stored headers for URL http://_default_:81/?

Here, the second request is the same : it "re-caches" the page,
hitting my webapp. I'm not an httpd expert but what's the meaning of
CACHE_REMOVE_URL, and isn't CACHE_OUT supposed to appear in 2.2.0 ?

For clarification, the headers I receive from my webapp are in both cases :

HTTP/1.1 200 OK
Date: Wed, 25 Jan 2006 10:47:08 GMT
Server: Apache-Coyote/1.1
X-Cocoon-Version: 2.1.5
Last-Modified: Wed, 25 Jan 2006 10:47:07 GMT
Expires: Sat, 28 Jan 2006 10:47:08 GMT
Cache-Control: max-age=259200
Vary: User-Agent
Content-Type: text/html
Connection: close

The directives I use are :

    CacheRoot "/var/cache/apache220/proxy/mydomain"
    CacheIgnoreCacheControl On
    CacheEnable disk /
    CacheDisable /site/images
    CacheDirLevels 5
    CacheDirLength 3

Is it a known problem ?

Thanks
laurent

Re: mod_disk_cache behaviour with Vary: header

Posted by Laurent Perez <ha...@gmail.com>.
> Wait a minute. Are you using 2.2.0 or are you using 2.0.54?  AFAIK,
> Debian has not imported 2.2.xx at all yet.

oh my.. sorry, the logs messages I had were [Tue Jan 24 17:53:50 2006]
[debug] /home/adconrad/build/apache2/security/sarge/apache2-2.0.54/build-tree/apache2/modules/experimental/mod_cache.c(446)
and I mixed 2-2 with 2.0.

> In 2.2.0, it uses a .vary directory, with the same base hash, and then
> all varied contents are re-hashed underneath that base <hash>.vary
> directory.  It gives you the correct behavior with varied contents.

yea, in fact I was reading MARC mailing list archives and just found
out about this.

thanks for the answer Paul, now I guess I'll have to convince my
hosting company to switch to 2.2 :p


--
<a href="http://in-pocket.blogspot.com">http://in-pocket.blogspot.com
- Mobile world, technology and more</a>

Re: mod_disk_cache behaviour with Vary: header

Posted by Paul Querna <ch...@force-elite.com>.
Laurent Perez wrote:
> Hello
> 
> I apologize for posting this to dev@httpd but I think my question is
> related to mod_cache module developers, that's why I'm using this
> list, after reading bugzilla and source code available on
> svn.apache.org.
> 
> I'm having troubles whenever I want to use the Vary: header to let
> mod_cache (I'm using mod_disk_cache, together with mod_proxy on
> apache2.2.0-54, debian packages) handle cached variations based on
> that header.

Wait a minute. Are you using 2.2.0 or are you using 2.0.54?  AFAIK, 
Debian has not imported 2.2.xx at all yet.


> The header I vary on is User-Agent, and when several different agents
> hit the cache, I thought mod_cache would create a separate couple from
> .header + .data files for each and every cached request. What is
> happening is that mod_cache is correctly detecting the agent variation
> (LogLevel debug directive reports "Vary header mismatch" from
> cache_storage.c), but instead of creating a new hash couple on disk,
> it is always rewriting the same .header and .data files.
> 
> This reduces interest of the Vary: header, because for example when
> two different agents hit mod_cache, the backend webapp proxied by
> mod_proxy is queried twice, because the resulting hash variation is
> stored in the same files, and not in four different files. Of course
> if the same agent hits the cache twice, with no other agent hit
> in-between, the correct cached response is obviously well served.
> 
> Is it an intentional cache behaviour, for example to avoid too many
> files to be created from mod_disk_cache only because of multiple
> probable variations ? Or is it a bug/todo feature that's being still
> worked on ?

This was fixed in 2.2.0. Upgrade and rejoice :)

In 2.2.0, it uses a .vary directory, with the same base hash, and then 
all varied contents are re-hashed underneath that base <hash>.vary 
directory.  It gives you the correct behavior with varied contents.

-Paul