Duplicate headers in apache 2 disk-cache

[Eric Prud'hommeaux <er...@w3.org>]

Resources served by proxy after consulting a stale cache have
duplicates of the non-specially handled headers. This is because
mod_disk_cache's read_headers reads the cached headers into the
err_headers_out and ap_http_header_filter merges them with the
ones read from a proxy call.

-----Details-----

In server/util_script.c:498 ap_scan_script_header_err_core takes a
table of headers that is has read, in my case, from a proxy cache
file, and merges them onto r->err_headers_out.

 apr_table_overlap(r->err_headers_out, merge,
  APR_OVERLAP_TABLES_MERGE);		/* It doesn't seem that an error has arisen. */

During the process of reading in the headers, it has set several
fields in the request:
  content_type, status, status_line, last_modified, cookies

and set some particular headers in the r->headers_out:
  Location, Content-Length, Transfer-Encoding

In the case of disk caching, read_headers (mod_disk_cache:465) calls
ap_scan_script_header_err_core (via ap_scan_script_header_err) and
sets the Content-Type from the previously set r->content_type. Only
a few headers get set this way:

(gdb) dump_table r->headers_out
[0] 'Last-Modified'='Fri, 26 Jul 2002 22:55:45 GMT'
[1] 'Content-Type'='multipart/mixed; boundary=3a6c43a796c1d96d'

The rest are merged into err_headers_out:

(gdb) dump_table r->err_headers_out
[0] 'Date'='Sat, 27 Jul 2002 16:17:34 GMT'
[1] 'Server'='Apache/2.0.40-dev (Unix) DAV/2'
[2] 'Cache-Control'='max-age=21600'
[3] 'Expires'='Sat, 27 Jul 2002 22:17:34 GMT'
[4] 'ETag'='"201-158f-ac96fe40"'
[5] 'Accept-Ranges'='bytes'
[6] '00-Package-Language'='http://www.w3.org/1999/02/22-rdf-syntax-ns#'
[7] 'Vary'='00-Package-Language,Man'
[8] '00-Content-Type'='text/html; charset=iso-8859-1'
[9] '00-Content-Length'='5519'
[10] 'Man'='"http://www.w3.org/2001/07/07-MetaPackage/"; ns=00'
[11] 'Via'='1.1 localhost:8003'

Since the query is not served from a cache, the proxy gets new headers
from the document server and stores them in r->headers_out.
ap_http_header_filter (modules/http/http_protocol.c:1522) merges
headers_out and err_headers_out onto a new headers_out:

    if (!apr_is_empty_table(r->err_headers_out)) {
        r->headers_out = apr_table_overlay(r->pool, r->err_headers_out,
                                           r->headers_out);
    }
This happens at
#0  ap_http_header_filter (f=0x81e1020, b=0x81e6000) at http_protocol.c:1519
#1  0x080d3e03 in ap_pass_brigade (next=0x81e1020, bb=0x81e6000) at util_filter.c:540
#2  0x080bdb93 in _httpExt_genExtensionHeaders (f=0x81e1dc0, b=0x81e6000) at mod_http_ext.c:231
#3  0x080d3e03 in ap_pass_brigade (next=0x81e1dc0, bb=0x81e6000) at util_filter.c:540
#4  0x080d714d in ap_content_length_filter (f=0x81e1008, b=0x81e6000) at protocol.c:1355
#5  0x080d3e03 in ap_pass_brigade (next=0x81e1008, bb=0x81e6000) at util_filter.c:540
#6  0x080872de in ap_byterange_filter (f=0x81e0ff0, bb=0x81e6000) at http_protocol.c:2803
#7  0x080d3e03 in ap_pass_brigade (next=0x81e0ff0, bb=0x81e6000) at util_filter.c:540
#8  0x08063fa8 in cache_in_filter (f=0x81d3c50, in=0x81e6000) at mod_cache.c:744
#9  0x080d3e03 in ap_pass_brigade (next=0x81d3c50, bb=0x81e6000) at util_filter.c:540
#10 0x0808185f in ap_proxy_http_process_response (p=0x81c8100, r=0x81e0360, p_conn=0x81e6050, origin=0x81e6248, backend=0x81c8658, 
    conf=0x811f230, bb=0x81e6000, server_portstr=0xbffff6a4 ":8003") at proxy_http.c:926
#11 0x08081ca9 in ap_proxy_http_handler (r=0x81e0360, conf=0x811f230, url=0x81e6140 "/2001/10/22-HttpExt-apache/Overview.html", 
    proxyname=0x0, proxyport=0) at proxy_http.c:1088
#12 0x08077d54 in proxy_run_scheme_handler (r=0x81e0360, conf=0x811f230, 
    url=0x81d3dfe "http://localhost:8003/2001/10/22-HttpExt-apache/Overview.html", proxyhost=0x0, proxyport=0) at mod_proxy.c:1121
#13 0x080767f5 in proxy_handler (r=0x81e0360) at mod_proxy.c:463
#14 0x080c4571 in ap_run_handler (r=0x81e0360) at config.c:193
#15 0x080c4dae in ap_invoke_handler (r=0x81e0360) at config.c:400
#16 0x080880bc in ap_process_request (r=0x81e0360) at http_request.c:257
#17 0x0808229a in ap_process_http_connection (c=0x81c8208) at http_core.c:293
#18 0x080d1081 in ap_run_process_connection (c=0x81c8208) at connection.c:85
#19 0x080d146c in ap_process_connection (c=0x81c8208, csd=0x81c8138) at connection.c:207
#20 0x080c2d37 in child_main (child_num_arg=0) at prefork.c:696
#21 0x080c2e1d in make_child (s=0x811e4d0, slot=0) at prefork.c:736
#22 0x080c2f54 in startup_children (number_to_start=5) at prefork.c:808
#23 0x080c3376 in ap_mpm_run (_pconf=0x8119a30, plog=0x8163b58, s=0x811e4d0) at prefork.c:1024
#24 0x080ca023 in main (argc=2, argv=0xbffffab4) at main.c:645
#25 0x4016714f in __libc_start_main () from /lib/libc.so.6

This leaves duplicate headers as the the err_headers_out was set by
reading an expired response from the disk and the headers_out come
from reading the proxied headers_in.
HTTP/1.1 200 OK
Date: Tue, 30 Jul 2002 03:58:24 GMT
Server: Apache/2.0.40-dev (Unix) DAV/2
Cache-Control: max-age=1
Expires: Tue, 30 Jul 2002 01:43:35 GMT
ETag: "201-158f-ac96fe40"
Accept-Ranges: bytes
Via: 1.1 localhost:8003
Cache-Control: max-age=1
Expires: Tue, 30 Jul 2002 04:00:30 GMT
Last-Modified: Fri, 26 Jul 2002 22:55:45 GMT
ETag: "201-158f-ac96fe40"
Accept-Ranges: bytes
Content-Type: multipart/mixed; boundary=3a6f647f7bea04f3c
Via: 1.1 localhost:8003
Transfer-Encoding: chunked

Note the duplicates of Cache-Control, Expires, Etag and Accept-Ranges.
The rest are handled specially by the intervening code.

I nuked the err_headers_out when the decision was made to use a proxy,
cache_url_handler (modules/experimental/mod_cache.c:246)
        else {
+	    r->err_headers_out = apr_table_make(r->pool, 3);
            if (lookup) {
                return DECLINED;
            }
but perhaps the intent was to serve a stale document if the fresh one
was not available?

Thoughts? Issues? Reflexes?
-- 
-eric

(eric@w3.org)
Feel free to forward this message to any list for any purpose other than
email address distribution.