You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Eric Prud'hommeaux <er...@w3.org> on 2002/08/16 15:55:16 UTC
patches for cache_pqueue.h break cache_cache.c
This possibly follows an earlier message:
<Pi...@bistromath.cs.virginia.edu>
2002/08/14 00:07:44 commit of cache_pqueue.h changed typedefs for
callbacks from function declarations to function pointers. Following
this, cache_cache(_make_money_fast).c failed to compile. I could fix
either, but being strongly in favor of typedefs of function
definitions, I patched cache_pqueue.{h,c}.
Argument for function definitions typedefs:
You can use the typedef to declare your function.
someLib.h:
typedef int (adder)(int origValue);
void registerAddr(adder* addMe);
myCode.c
adder MyAdder;
int main () {registerAddr(&MyAddr);}
int MyAdder (int origValue) {return origValue+7;}
If adder were a function pointer
typedef int (adder)(int origValue);
I wouldn't be able to use it in forward declarations/sanity checks like
adder MyAdder;
This mattered to me as I was testing some disk caching proxy patches.
These are included, but not tested. I should get to that in the next
couple of days.
--
-eric
(eric@w3.org)
Feel free to forward this message to any list for any purpose other than
email address distribution.
Re: patches for cache_pqueue.h break cache_cache.c
Posted by Eric Prud'hommeaux <er...@w3.org>.
On Fri, Aug 16, 2002 at 09:55:16AM -0400, Eric Prud'hommeaux wrote:
> This possibly follows an earlier message:
> <Pi...@bistromath.cs.virginia.edu>
>
> 2002/08/14 00:07:44 commit of cache_pqueue.h changed typedefs for
> callbacks from function declarations to function pointers. Following
> this, cache_cache(_make_money_fast).c failed to compile. I could fix
> either, but being strongly in favor of typedefs of function
> definitions, I patched cache_pqueue.{h,c}.
>
> Argument for function definitions typedefs:
> You can use the typedef to declare your function.
> someLib.h:
> typedef int (adder)(int origValue);
> void registerAddr(adder* addMe);
> myCode.c
> adder MyAdder;
> int main () {registerAddr(&MyAddr);}
> int MyAdder (int origValue) {return origValue+7;}
> If adder were a function pointer
> typedef int (adder)(int origValue);
oops, make that:
typedef int (*adder)(int origValue);
> I wouldn't be able to use it in forward declarations/sanity checks like
> adder MyAdder;
>
> This mattered to me as I was testing some disk caching proxy patches.
> These are included, but not tested. I should get to that in the next
> couple of days.
> --
> -eric
>
> (eric@w3.org)
> Feel free to forward this message to any list for any purpose other than
> email address distribution.
Content-Description: cache_pqueue.{h,c} patches
> Index: httpd-2.0/modules/experimental/cache_pqueue.c
> ===================================================================
> RCS file: /home/cvspublic/httpd-2.0/modules/experimental/cache_pqueue.c,v
> retrieving revision 1.12
> diff -u -r1.12 cache_pqueue.c
> --- httpd-2.0/modules/experimental/cache_pqueue.c 14 Aug 2002 01:24:16 -0000 1.12
> +++ httpd-2.0/modules/experimental/cache_pqueue.c 16 Aug 2002 13:19:28 -0000
> @@ -81,9 +81,9 @@
> apr_ssize_t size;
> apr_ssize_t avail;
> apr_ssize_t step;
> - cache_pqueue_get_priority pri;
> - cache_pqueue_getpos get;
> - cache_pqueue_setpos set;
> + cache_pqueue_get_priority* pri;
> + cache_pqueue_getpos* get;
> + cache_pqueue_setpos* set;
> void **d;
> };
>
> Index: httpd-2.0/modules/experimental/cache_pqueue.h
> ===================================================================
> RCS file: /home/cvspublic/httpd-2.0/modules/experimental/cache_pqueue.h,v
> retrieving revision 1.4
> diff -u -r1.4 cache_pqueue.h
> --- httpd-2.0/modules/experimental/cache_pqueue.h 14 Aug 2002 00:07:44 -0000 1.4
> +++ httpd-2.0/modules/experimental/cache_pqueue.h 16 Aug 2002 13:19:28 -0000
> @@ -78,21 +78,21 @@
> * @param a the element
> * @return the score (the lower the score the longer it is kept int the queue)
> */
> -typedef long (*cache_pqueue_set_priority)(long queue_clock, void *a);
> -typedef long (*cache_pqueue_get_priority)(void *a);
> +typedef long (cache_pqueue_set_priority)(long queue_clock, void *a);
> +typedef long (cache_pqueue_get_priority)(void *a);
>
> /** callback function to get a position of a element */
> -typedef apr_ssize_t (*cache_pqueue_getpos)(void *a);
> +typedef apr_ssize_t (cache_pqueue_getpos)(void *a);
>
> /**
> * callback function to set a position of a element
> * @param a the element
> * @param pos the position to set it to
> */
> -typedef void (*cache_pqueue_setpos)(void *a, apr_ssize_t pos);
> +typedef void (cache_pqueue_setpos)(void *a, apr_ssize_t pos);
>
> /** debug callback function to print a entry */
> -typedef void (*cache_pqueue_print_entry)(FILE *out, void *a);
> +typedef void (cache_pqueue_print_entry)(FILE *out, void *a);
>
> /**
> * initialize the queue
Content-Description: disk_cache patches (not tested)
> Index: httpd-2.0/modules/experimental/cache_storage.c
> ===================================================================
> RCS file: /home/cvspublic/httpd-2.0/modules/experimental/cache_storage.c,v
> retrieving revision 1.25
> diff -u -r1.25 cache_storage.c
> --- httpd-2.0/modules/experimental/cache_storage.c 23 Jun 2002 06:10:00 -0000 1.25
> +++ httpd-2.0/modules/experimental/cache_storage.c 1 Aug 2002 06:38:22 -0000
> @@ -154,6 +154,16 @@
> return 1;
> }
>
> +static apr_status_t _failCache (request_rec *r, cache_request_rec *cache) {
> + /* headers do not match, so Vary failed */
> + ap_log_error(APLOG_MARK, APLOG_INFO, APR_SUCCESS, r->server,
> + "cache_select_url(): Vary header mismatch - Cached document cannot be used. \n");
> + apr_table_clear(r->headers_out);
> + r->status_line = NULL;
> + cache->handle = NULL;
> + return DECLINED;
> +}
> +
> /*
> * select a specific URL entity in the cache
> *
> @@ -209,7 +219,7 @@
> *
> * RFC2616 13.6 and 14.44 describe the Vary mechanism.
> */
> - vary = apr_pstrdup(r->pool, apr_table_get(r->headers_out, "Vary"));
> + vary = apr_pstrdup(r->pool, apr_table_get(r->err_headers_out, "Vary"));
> while (vary && *vary) {
> char *name = vary;
> const char *h1, *h2;
> @@ -222,12 +232,14 @@
> ++vary;
> }
>
> - /*
> - * is this header in the request and the header in the cached
> - * request identical? If not, we give up and do a straight get
> - */
> - h1 = apr_table_get(r->headers_in, name);
> - h2 = apr_table_get(h->req_hdrs, name);
> + {
> + /*
> + * Are this header in the request and the header in the cached
> + * request identical? If not, we give up and do a straight GET.
> + */
> + h1 = apr_table_get(r->headers_in, name);
> + h2 = apr_table_get(h->req_hdrs, name);
> + }
> if (h1 == h2) {
> /* both headers NULL, so a match - do nothing */
> }
> @@ -235,13 +247,7 @@
> /* both headers exist and are equal - do nothing */
> }
> else {
> - /* headers do not match, so Vary failed */
> - ap_log_error(APLOG_MARK, APLOG_INFO, APR_SUCCESS, r->server,
> - "cache_select_url(): Vary header mismatch - Cached document cannot be used. \n");
> - apr_table_clear(r->headers_out);
> - r->status_line = NULL;
> - cache->handle = NULL;
> - return DECLINED;
> + return _failCache(r, cache);
> }
> }
> return OK;
> @@ -283,7 +289,8 @@
> return rv;
> }
>
> - r->filename = apr_pstrdup(r->pool, info->filename );
> + if (info->filename) /* EGP: bug report Message-Id: <20...@w3.org> */
> + r->filename = apr_pstrdup(r->pool, info->filename );
>
> return APR_SUCCESS;
> }
> Index: httpd-2.0/modules/experimental/mod_cache.c
> ===================================================================
> RCS file: /home/cvspublic/httpd-2.0/modules/experimental/mod_cache.c,v
> retrieving revision 1.49
> diff -u -r1.49 mod_cache.c
> --- httpd-2.0/modules/experimental/mod_cache.c 24 Jul 2002 20:47:28 -0000 1.49
> +++ httpd-2.0/modules/experimental/mod_cache.c 1 Aug 2002 06:38:25 -0000
> @@ -118,7 +118,7 @@
> "cache: URL exceeds length threshold: %s", url);
> return DECLINED;
> }
> - /* DECLINE urls ending in / */
> + /* DECLINE urls ending in / ??? EGP: why? */
> if (url[urllen-1] == '/') {
> return DECLINED;
> }
> @@ -244,6 +244,7 @@
> return OK;
> }
> else {
> + r->err_headers_out = apr_table_make(r->pool, 3);
> /* stale data available */
> if (lookup) {
> return DECLINED;
> Index: httpd-2.0/modules/experimental/mod_disk_cache.c
> ===================================================================
> RCS file: /home/cvspublic/httpd-2.0/modules/experimental/mod_disk_cache.c,v
> retrieving revision 1.36
> diff -u -r1.36 mod_disk_cache.c
> --- httpd-2.0/modules/experimental/mod_disk_cache.c 17 Jul 2002 14:52:36 -0000 1.36
> +++ httpd-2.0/modules/experimental/mod_disk_cache.c 1 Aug 2002 06:38:30 -0000
> @@ -237,7 +237,7 @@
> if ((temp = strchr(&urlbuff[0], '\n')) != NULL) /* trim off new line character */
> *temp = '\0'; /* overlay it with the null terminator */
>
> - if (!apr_date_checkmask(urlbuff, "&&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&&")) {
> + if (!apr_date_checkmask(urlbuff, "&&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&&")) {
> return APR_EGENERAL;
> }
>
> @@ -246,6 +246,10 @@
> info->expire = ap_cache_hex2usec(urlbuff + offset);
> offset += (sizeof(info->expire)*2) + 1;
> dobj->version = ap_cache_hex2usec(urlbuff + offset);
> + offset += (sizeof(info->expire)*2) + 1;
> + info->request_time = ap_cache_hex2usec(urlbuff + offset);
> + offset += (sizeof(info->expire)*2) + 1;
> + info->response_time = ap_cache_hex2usec(urlbuff + offset);
>
> /* check that we have the same URL */
> rv = apr_file_gets(&urlbuff[0], urllen, fd);
> @@ -276,6 +280,8 @@
> char dateHexS[sizeof(apr_time_t) * 2 + 1];
> char expireHexS[sizeof(apr_time_t) * 2 + 1];
> char verHexS[sizeof(apr_time_t) * 2 + 1];
> + char requestHexS[sizeof(apr_time_t) * 2 + 1];
> + char responseHexS[sizeof(apr_time_t) * 2 + 1];
> cache_info *info = &(h->cache_obj->info);
> disk_cache_object_t *dobj = (disk_cache_object_t *) h->cache_obj->vobj;
>
> @@ -287,7 +293,9 @@
> ap_cache_usec2hex(info->date, dateHexS);
> ap_cache_usec2hex(info->expire, expireHexS);
> ap_cache_usec2hex(dobj->version++, verHexS);
> - buf = apr_pstrcat(r->pool, dateHexS, " ", expireHexS, " ", verHexS, "\n", NULL);
> + ap_cache_usec2hex(info->request_time, requestHexS);
> + ap_cache_usec2hex(info->response_time, responseHexS);
> + buf = apr_pstrcat(r->pool, dateHexS, " ", expireHexS, " ", verHexS, " ", requestHexS, " ", responseHexS, "\n", NULL);
> amt = strlen(buf);
> rc = apr_file_write(fd, buf, &amt);
> if (rc != APR_SUCCESS) {
> @@ -448,6 +456,7 @@
> char urlbuff[1034];
> int urllen = sizeof(urlbuff);
> disk_cache_object_t *dobj = (disk_cache_object_t *) h->cache_obj->vobj;
> + apr_table_t * tmp;
>
> /* This case should not happen... */
> if (!dobj->fd || !dobj->hfd) {
> @@ -486,6 +495,17 @@
>
> r->status_line = apr_pstrdup(r->pool, urlbuff); /* Save status line into request rec */
>
> + h->req_hdrs = apr_table_make(r->pool, 20);
> +
> + /*
> + * Call routine to read the header lines/status line
> + */
> + tmp = r->err_headers_out;
> + r->err_headers_out = h->req_hdrs;
> + rv = apr_file_gets(&urlbuff[0], urllen, dobj->hfd); /* Read status */
> + ap_scan_script_header_err(r, dobj->hfd, NULL);
> + r->err_headers_out = tmp;
> +
> apr_file_close(dobj->hfd);
>
> ap_log_error(APLOG_MARK, APLOG_INFO, 0, r->server,
> @@ -585,6 +605,24 @@
> buf = apr_pstrcat(r->pool, CRLF, NULL);
> amt = strlen(buf);
> apr_file_write(hfd, buf, &amt);
> +
> + /* Parse the vary header and dump those fields from the headers_in. */
> + /* Make call to the same thing cache_select_url calls to crack Vary. */
> + /* @@@ Some day, not today. */
> + if (r->headers_in) {
> + int i;
> + apr_table_entry_t *elts = (apr_table_entry_t *) apr_table_elts(r->headers_in)->elts;
> + for (i = 0; i < apr_table_elts(r->headers_in)->nelts; ++i) {
> + if (elts[i].key != NULL) {
> + buf = apr_pstrcat(r->pool, elts[i].key, ": ", elts[i].val, CRLF, NULL);
> + amt = strlen(buf);
> + apr_file_write(hfd, buf, &amt);
> + }
> + }
> + buf = apr_pstrcat(r->pool, CRLF, NULL);
> + amt = strlen(buf);
> + apr_file_write(hfd, buf, &amt);
> + }
> apr_file_close(hfd); /* flush and close */
> }
> else {
--
-eric
(eric@w3.org)
Feel free to forward this message to any list for any purpose other than
email address distribution.
Re: disk caching patches now tested
Posted by Eric Prud'hommeaux <er...@w3.org>.
I have to screw at least one thing up per post.
Here are the promised attachments...
--
-eric
(eric@w3.org)
Feel free to forward this message to any list for any purpose other than
email address distribution.
RE: disk caching patches now tested
Posted by Bill Stoddard <bi...@wstoddard.com>.
Eric,
If no beats me to it, I'll review and commit your patches later this
weekend.
Thanks.
Bill
> This documents tests and provides a test harnes for patches given in
> Messge-ID: <20...@w3.org>.
>
> Using my good buddies patchPanel and xterm, I tested the disk caching
> with regards to observing the Vary header. The attached tarball is
> intended to be expanded in the directory containing httpd-2.0/.
>
> ALL EXCITED TO PLAY:
> If all goes oddly well, you should be able to
> make -f diskCache/Makefile
> and go take a leak. When you come back, you should have three xterms
> proxy server
> manual server
> manual client
> Anything you type into manual client will go to proxy server and be
> relayed to a newly running apache proxy server running on port 9003.
> The proxy server will relay its requests on to whatever is specified
> in the GET and HOST lines. The sample request sends the requests on
> to the manual server where you will have to provide the response.
> You can GET stuff from a real server, but this gives you a bit more
> control.
>
> SPECIFIC TESTS:
> You enter a forest of xterms, each with a different title. The one
> labeld "connected 3 9004" should have
> connected 3 to 9004
> in the top. Enter a '/' followed by the command
> cat diskCache/proxyTest-request.http 1
> This will send a request to the proxy server which will relay it to
> the manual server. Now you have to play the role of manual server
> so you go to that window, enter a '/' and give the command
> cat diskCache/proxyTest-response.http 2
> to give a response back to the proxy and client.
>
> The proxy should create the file
> diskCache/ProxyServerRoot/proxy/4o/0q/Jx/@_IyiXLoI94OuiXg.header
> which holds the headers for the request and response.
>
> You can mark where you are in each patchPanel with
> /echo virgin cache
>
> The proxy server should have seen
>
> GET http://localhost:9005/doc1 HTTP/1.1
> Host: localhost:9005
> Header1: Value1
> Header2: Value2
>
> HTTP/1.1 200 OK
> Date: Fri, 16 Aug 2002 21:12:14 GMT
> Server: Apache/2.0.41-dev (Unix)
> Content-Location: doc1.xhtml
> Vary: Header1,Header2
> Last-Modified: Mon, 05 Aug 2002 08:32:14 GMT
> ETag: "4301b8-102-c6d39f80;fb6d5700"
> Accept-Ranges: bytes
> Cache-Control: max-age=600000
> Expires: Fri, 16 Aug 2003 20:26:20 GMT
> Content-Type: text/plain
> Via: 1.1 127.0.0.1:9003
> Content-Length: 14
>
> line 1
> line 2
>
> Entering the same request in (quickly, before you lose your the proxy
> server's attention) should result in the same reply, without touching
> the manual server. Changing one of the vary fields:
> GET http://localhost:9005/doc1 HTTP/1.1
> Host: localhost:9005
> Header1: Value1
> Header2: Value2b
>
> should result in the request going back to the manual server. Thus I
> declare the vary support a success.
>
> BORED NOW:
> After you get tired of screwing around with this stuff
> make -f diskCache/Makefile kill
> will kill off the proxy server and you can
> /quit
> in each of the patchPanel windows.
>
> TODO:
> -There are a few code paths with conditional caching that I haven't
> looked at.
>
> -It would be cool to retrieve the Vary headers for foo (by the current
> mechanism), compute a new hash from the request values of those
> headers, and look for a cached variant matching that request. This
> could be done only when the first cache fails the varies tests. Then
> it would only create work when would have had to fail anyways.
>
> -Add HTTP Extensions support to the cache validity calculation. I have
> patches for this, but I'll wait 'till I've got fewer outstanding
> patches (2 currently, typedefFuncs-20020816.patch and
> diskCache-20020816.patch).
>
> Despite these todos, I believe it is a good idea to integrate this
> patches as disk caching doesn't really work now so they at least
> improve the situation.
> --
> -eric
>
> (eric@w3.org)
> Feel free to forward this message to any list for any purpose other than
> email address distribution.
>
disk caching patches now tested
Posted by Eric Prud'hommeaux <er...@w3.org>.
This documents tests and provides a test harnes for patches given in
Messge-ID: <20...@w3.org>.
Using my good buddies patchPanel and xterm, I tested the disk caching
with regards to observing the Vary header. The attached tarball is
intended to be expanded in the directory containing httpd-2.0/.
ALL EXCITED TO PLAY:
If all goes oddly well, you should be able to
make -f diskCache/Makefile
and go take a leak. When you come back, you should have three xterms
proxy server
manual server
manual client
Anything you type into manual client will go to proxy server and be
relayed to a newly running apache proxy server running on port 9003.
The proxy server will relay its requests on to whatever is specified
in the GET and HOST lines. The sample request sends the requests on
to the manual server where you will have to provide the response.
You can GET stuff from a real server, but this gives you a bit more
control.
SPECIFIC TESTS:
You enter a forest of xterms, each with a different title. The one
labeld "connected 3 9004" should have
connected 3 to 9004
in the top. Enter a '/' followed by the command
cat diskCache/proxyTest-request.http 1
This will send a request to the proxy server which will relay it to
the manual server. Now you have to play the role of manual server
so you go to that window, enter a '/' and give the command
cat diskCache/proxyTest-response.http 2
to give a response back to the proxy and client.
The proxy should create the file
diskCache/ProxyServerRoot/proxy/4o/0q/Jx/@_IyiXLoI94OuiXg.header
which holds the headers for the request and response.
You can mark where you are in each patchPanel with
/echo virgin cache
The proxy server should have seen
GET http://localhost:9005/doc1 HTTP/1.1
Host: localhost:9005
Header1: Value1
Header2: Value2
HTTP/1.1 200 OK
Date: Fri, 16 Aug 2002 21:12:14 GMT
Server: Apache/2.0.41-dev (Unix)
Content-Location: doc1.xhtml
Vary: Header1,Header2
Last-Modified: Mon, 05 Aug 2002 08:32:14 GMT
ETag: "4301b8-102-c6d39f80;fb6d5700"
Accept-Ranges: bytes
Cache-Control: max-age=600000
Expires: Fri, 16 Aug 2003 20:26:20 GMT
Content-Type: text/plain
Via: 1.1 127.0.0.1:9003
Content-Length: 14
line 1
line 2
Entering the same request in (quickly, before you lose your the proxy
server's attention) should result in the same reply, without touching
the manual server. Changing one of the vary fields:
GET http://localhost:9005/doc1 HTTP/1.1
Host: localhost:9005
Header1: Value1
Header2: Value2b
should result in the request going back to the manual server. Thus I
declare the vary support a success.
BORED NOW:
After you get tired of screwing around with this stuff
make -f diskCache/Makefile kill
will kill off the proxy server and you can
/quit
in each of the patchPanel windows.
TODO:
-There are a few code paths with conditional caching that I haven't
looked at.
-It would be cool to retrieve the Vary headers for foo (by the current
mechanism), compute a new hash from the request values of those
headers, and look for a cached variant matching that request. This
could be done only when the first cache fails the varies tests. Then
it would only create work when would have had to fail anyways.
-Add HTTP Extensions support to the cache validity calculation. I have
patches for this, but I'll wait 'till I've got fewer outstanding
patches (2 currently, typedefFuncs-20020816.patch and
diskCache-20020816.patch).
Despite these todos, I believe it is a good idea to integrate this
patches as disk caching doesn't really work now so they at least
improve the situation.
--
-eric
(eric@w3.org)
Feel free to forward this message to any list for any purpose other than
email address distribution.
Re: patches for cache_pqueue.h break cache_cache.c
Posted by Cliff Woolley <jw...@virginia.edu>.
On Fri, 16 Aug 2002, Eric Prud'hommeaux wrote:
> This possibly follows an earlier message:
> <Pi...@bistromath.cs.virginia.edu>
Yes, it does. I meant to fix this yesterday but then the power went out
in my building. :-( I'll fix it today.
--Cliff