You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modproxy-dev@apache.org by Christian von Roques <ro...@mti.ag> on 2000/08/12 19:53:27 UTC
[PATCH] mod_proxy talking HTTP/1.1 to clients
I'm in need of a non-caching HTTP-proxy which is talking HTTPS/1.1
including persistent connections to its clients and is compatible with
HTTP/1.0 content servers. For this I thought about using Apache-1.3,
mod_ssl, and mod_proxy. This combination as released does not support
persistent client connections when serving a page through mod_proxy.
Do you know of a better / easier solution to this problem than to fix
Apache's mod_proxy?
To make this work, I started with Graham Leggett's http/1.1 patch for
mod_proxy, which adds the logic behind all of the new headers of
http/1.1. I then rewrote mod_proxy's communication with the client to
use the machinery the Apache core provides, which looks much cleaner
and makes it HTTP/1.1-compatible including persistent connections.
The patch below mostly consists of a slightly cleaned up version of
Graham Leggett's http/1.1 patch plus my changes to the mod_proxy <->
client communication. The patch is against Apache-1.3.13-dev from CVS
as of today. It works for me, but I haven't really tested it in a
caching configuration.
Is anybody else interested in this kind of changes? If this patch
proves to be to huge to be accepted, or to broken to be fixed in a
reasonable amount of time, if used in a caching configuration, I will
try to determine if it's feasible to make it talk HTTP/1.1 only in a
non-caching configuration and leave the caching case to the old code.
Please, give me some feedback. I didn't receive one single answer to
last weeks email about graceful degradation of service instead of
locking out clients. Isn't anybody interested? I half expected Dean
to suggest the other way of implementing busy_servers().
Christian.
diff -wurdX nodiff.pats apache-1.3.13-dev/src/main/alloc.c apache-1.3.13-dev+http1.1/src/main/alloc.c
--- apache-1.3.13-dev/src/main/alloc.c Mon Jul 17 10:13:04 2000
+++ apache-1.3.13-dev+http1.1/src/main/alloc.c Thu Aug 10 13:18:31 2000
@@ -1457,6 +1457,24 @@
return res;
}
+/* overlay one table on another - keys in base will be replaced by keys in overlay */
+API_EXPORT(int) ap_replace_tables(table *base, table *overlay)
+{
+ table_entry *elts = (table_entry *) overlay->a.elts;
+ int i, q = 0;
+ const char *val;
+
+ for (i = 0; i < overlay->a.nelts; ++i) {
+ val = ap_table_get(base, elts[i].key);
+ if (!val || strcmp(val, elts[i].val))
+ q = 1;
+ ap_table_set(base, elts[i].key, elts[i].val);
+ }
+
+ return q;
+}
+
+
/* And now for something completely abstract ...
* For each key value given as a vararg:
diff -wurdX nodiff.pats apache-1.3.13-dev/src/main/http_protocol.c apache-1.3.13-dev+http1.1/src/main/http_protocol.c
--- apache-1.3.13-dev/src/main/http_protocol.c Sat Feb 19 21:14:47 2000
+++ apache-1.3.13-dev+http1.1/src/main/http_protocol.c Thu Aug 10 13:26:23 2000
@@ -1389,12 +1389,10 @@
if (!r->status_line)
r->status_line = status_lines[ap_index_of_response(r->status)];
- /* mod_proxy is only HTTP/1.0, so avoid sending HTTP/1.1 error response;
- * kluge around broken browsers when indicated by force-response-1.0
+ /* kluge around broken browsers when indicated by force-response-1.0
*/
- if (r->proxyreq != NOT_PROXY
- || (r->proto_num == HTTP_VERSION(1,0)
- && ap_table_get(r->subprocess_env, "force-response-1.0"))) {
+ if (r->proto_num == HTTP_VERSION(1,0)
+ && ap_table_get(r->subprocess_env, "force-response-1.0")) {
protocol = "HTTP/1.0";
r->connection->keepalive = -1;
diff -wurdX nodiff.pats apache-1.3.13-dev/src/modules/proxy/mod_proxy.c apache-1.3.13-dev+http1.1/src/modules/proxy/mod_proxy.c
--- apache-1.3.13-dev/src/modules/proxy/mod_proxy.c Thu Jun 1 19:42:26 2000
+++ apache-1.3.13-dev+http1.1/src/modules/proxy/mod_proxy.c Thu Aug 10 14:43:10 2000
@@ -323,6 +323,7 @@
if (p == NULL)
return HTTP_BAD_REQUEST;
+ /* Try serving the request from the cache. If we suceed, we leave. */
rc = ap_proxy_cache_check(r, url, &conf->cache, &cr);
if (rc != DECLINED)
return rc;
diff -wurdX nodiff.pats apache-1.3.13-dev/src/modules/proxy/mod_proxy.h apache-1.3.13-dev+http1.1/src/modules/proxy/mod_proxy.h
--- apache-1.3.13-dev/src/modules/proxy/mod_proxy.h Fri Jun 2 10:49:59 2000
+++ apache-1.3.13-dev+http1.1/src/modules/proxy/mod_proxy.h Fri Aug 11 13:52:49 2000
@@ -182,6 +182,13 @@
#define DEFAULT_CACHE_LMFACTOR (0.1)
#define DEFAULT_CACHE_COMPLETION (0.9)
+#ifndef MAX
+#define MAX(a,b) ((a) > (b) ? (a) : (b))
+#endif
+#ifndef MIN
+#define MIN(a,b) ((a) < (b) ? (a) : (b))
+#endif
+
/* static information about the local cache */
struct cache_conf {
const char *root; /* the location of the cache directory */
@@ -237,29 +244,33 @@
char *url; /* the URL requested */
char *filename; /* name of the cache file, or NULL if no cache */
char *tempfile; /* name of the temporary file, of NULL if not caching */
- time_t ims; /* if-modified-since date of request; -1 if no header */
+ time_t ims; /* If-Modified-Since date of request; -1 if no header */
+ time_t ius; /* If-Unmodified-Since date of request; -1 if no header */
+ const char *im; /* If-Match etag of request; NULL if no header */
+ const char *inm; /* If-None-Match etag of request; NULL if no header */
BUFF *fp; /* the cache file descriptor if the file is cached
and may be returned, or NULL if the file is
not cached (or must be reloaded) */
+ BUFF *origfp; /* the old cache file descriptor if the file has
+ been revalidated and is being rewritten to
+ disk */
time_t expire; /* calculated expire date of cached entity */
time_t lmod; /* last-modified date of cached entity */
time_t date; /* the date the cached file was last touched */
+ time_t req_time; /* the time the request started */
+ time_t resp_time; /* the time the response was received */
int version; /* update count of the file */
off_t len; /* content length */
char *protocol; /* Protocol, and major/minor number, e.g. HTTP/1.1 */
int status; /* the status of the cached file */
unsigned int written; /* total *content* bytes written to cache */
float cache_completion; /* specific to this request */
- char *resp_line; /* the whole status like (protocol, code + message) */
- table *hdrs; /* the HTTP headers of the file */
+ char *resp_line; /* the whole status line (protocol, code + message) */
+ table *req_hdrs; /* the original request headers when it was made */
+ table *hdrs; /* the original HTTP response headers of the file */
+ char *xcache; /* the X-Cache header value to be sent to client */
} cache_req;
-/* Additional information passed to the function called by ap_table_do() */
-struct tbl_do_args {
- request_rec *req;
- cache_req *cache;
-};
-
struct per_thread_data {
struct hostent hpbuf;
u_long ipaddr;
@@ -303,9 +314,9 @@
char **passwordp, char **hostp, int *port);
const char *ap_proxy_date_canon(pool *p, const char *x);
table *ap_proxy_read_headers(request_rec *r, char *buffer, int size, BUFF *f);
-long int ap_proxy_send_fb(BUFF *f, request_rec *r, cache_req *c);
-void ap_proxy_send_headers(request_rec *r, const char *respline, table *hdrs);
-int ap_proxy_liststr(const char *list, const char *val);
+long int ap_proxy_send_fb(BUFF *f, request_rec *r, cache_req *c, off_t len, int nowrite);
+void ap_proxy_write_headers(cache_req *c, const char *respline, table *t);
+int ap_proxy_liststr(const char *list, const char *key, char **val);
void ap_proxy_hash(const char *it, char *val, int ndepth, int nlength);
int ap_proxy_hex2sec(const char *x);
void ap_proxy_sec2hex(int t, char *y);
@@ -321,5 +332,9 @@
/* This function is called by ap_table_do() for all header lines */
int ap_proxy_send_hdr_line(void *p, const char *key, const char *value);
unsigned ap_proxy_bputs2(const char *data, BUFF *client, cache_req *cache);
+time_t ap_proxy_current_age(cache_req *c, const time_t age_value);
+BUFF *ap_proxy_open_cachefile(request_rec *r, char *filename);
+BUFF *ap_proxy_create_cachefile(request_rec *r, char *filename);
+void ap_proxy_clear_connection(pool *p, table *headers);
#endif /*MOD_PROXY_H*/
diff -wurdX nodiff.pats apache-1.3.13-dev/src/modules/proxy/proxy_cache.c apache-1.3.13-dev+http1.1/src/modules/proxy/proxy_cache.c
--- apache-1.3.13-dev/src/modules/proxy/proxy_cache.c Mon Jul 17 10:13:04 2000
+++ apache-1.3.13-dev+http1.1/src/modules/proxy/proxy_cache.c Thu Aug 10 15:30:25 2000
@@ -61,6 +61,7 @@
#include "http_conf_globals.h"
#include "http_log.h"
#include "http_main.h"
+#include "http_core.h"
#include "util_date.h"
#ifdef WIN32
#include <sys/utime.h>
@@ -412,7 +413,7 @@
static int sub_garbage_coll(request_rec *r, array_header *files,
const char *cachebasedir, const char *cachesubdir)
{
- char line[27];
+ char line[17*(3)];
char cachedir[HUGE_STRING_LEN];
struct stat buf;
int fd, i;
@@ -566,7 +567,7 @@
}
#endif
- i = read(fd, line, 26);
+ i = read(fd, line, 17*(3)-1);
close(fd);
if (i == -1) {
ap_log_error(APLOG_MARK, APLOG_ERR, r->server,
@@ -574,8 +575,8 @@
continue;
}
line[i] = '\0';
- garbage_expire = ap_proxy_hex2sec(line + 18);
- if (!ap_checkmask(line, "&&&&&&&& &&&&&&&& &&&&&&&&") ||
+ garbage_expire = ap_proxy_hex2sec(line + 17*(2));
+ if (!ap_checkmask(line, "&&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&&") ||
garbage_expire == BAD_DATE) {
/* bad file */
if (garbage_now != -1 && buf.st_atime > garbage_now + SEC_ONE_DAY &&
@@ -613,21 +614,45 @@
}
+
/*
- * read a cache file;
+ * Read a cache file;
* returns 1 on success,
* 0 on failure (bad file or wrong URL)
* -1 on UNIX error
+ *
+ * We read the cache hex header, then the message response line and
+ * response headers, and finally we return with the filepointer
+ * pointing at the start of the message body itself, ready to be
+ * shipped to the client later on, if appropriate.
*/
static int rdcache(request_rec *r, BUFF *cachefp, cache_req *c)
{
- char urlbuff[1034], *strp;
+ char urlbuff[HUGE_STRING_LEN], *strp;
int len;
+
/* read the data from the cache file */
-/* format
- * date SP lastmod SP expire SP count SP content-length CRLF
- * dates are stored as hex seconds since 1970
+
+ /* Format:
+ *
+ * The cache needs to keep track of the following information:
+ * - Date, LastMod, Version, ReqTime, RespTime, ContentLength
+ * - The original request headers (for Vary)
+ * - The original response headers (for returning with a cached response)
+ * - The body of the message
+ *
+ * date SP lastmod SP expire SP count SP request-time SP response-time SP content-lengthCRLF
+ * (dates are stored as hex seconds since 1970)
+ * Original URLCRLF
+ * Original Request Headers
+ * CRLF
+ * Original Response Headers
+ * CRLF
+ * Body
+ *
*/
+
+ /* retrieve cachefile information values */
len = ap_bgets(urlbuff, sizeof urlbuff, cachefp);
if (len == -1)
return -1;
@@ -636,14 +661,16 @@
urlbuff[len - 1] = '\0';
if (!ap_checkmask(urlbuff,
- "&&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&&"))
+ "&&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&& &&&&&&&&&&&&&&&&"))
return 0;
- c->date = ap_proxy_hex2sec(urlbuff);
- c->lmod = ap_proxy_hex2sec(urlbuff + 9);
- c->expire = ap_proxy_hex2sec(urlbuff + 18);
- c->version = ap_proxy_hex2sec(urlbuff + 27);
- c->len = ap_proxy_hex2sec(urlbuff + 36);
+ c->date = ap_proxy_hex2sec(urlbuff + 17*(0));
+ c->lmod = ap_proxy_hex2sec(urlbuff + 17*(1));
+ c->expire = ap_proxy_hex2sec(urlbuff + 17*(2));
+ c->version = ap_proxy_hex2sec(urlbuff + 17*(3));
+ c->req_time = ap_proxy_hex2sec(urlbuff + 17*(4));
+ c->resp_time = ap_proxy_hex2sec(urlbuff + 17*(5));
+ c->len = ap_proxy_hex2sec(urlbuff + 17*(6));
/* check that we have the same URL */
len = ap_bgets(urlbuff, sizeof urlbuff, cachefp);
@@ -656,7 +683,12 @@
if (strcmp(urlbuff + 7, c->url) != 0)
return 0;
-/* What follows is the message */
+ /* then the original request headers */
+ c->req_hdrs = ap_proxy_read_headers(r, urlbuff, sizeof urlbuff, cachefp);
+ if (c->req_hdrs == NULL)
+ return -1;
+
+ /* then the original response headers */
len = ap_bgets(urlbuff, sizeof urlbuff, cachefp);
if (len == -1)
return -1;
@@ -682,6 +714,195 @@
return 1;
}
+/*
+ * Call this to check the possible conditional status of
+ * the client request, and return the response from the cache
+ *
+ * Conditionals include If-Modified-Since, If-Match, If-Unmodified-Since
+ * and If-None-Match.
+ *
+ * We don't yet understand If-Range, but we will...
+ */
+int ap_proxy_cache_conditional(request_rec *r, cache_req *c, BUFF *cachefp)
+{
+ const char *etag, *wetag;
+
+ /* get etag */
+ if ((etag = ap_table_get(c->hdrs, "Etag"))) {
+ wetag = ap_pstrcat(r->pool, "W/", etag, NULL);
+ }
+
+ /* check for If-Match, If-Unmodified-Since
+ *
+ */
+ while (1) {
+
+ /* check If-Match and If-Unmodified-Since exist
+ *
+ * If neither of these exist, the request is not conditional, and
+ * we serve it normally
+ */
+ if (!c->im && BAD_DATE == c->ius) {
+ break;
+ }
+
+ /* check If-Match
+ *
+ * we check if the Etag on the cached file is in the list of Etags
+ * in the If-Match field. The comparison must be a strong comparison,
+ * so the Etag cannot be marked as weak. If the comparision fails
+ * we return 412 Precondition Failed.
+ *
+ * if If-Match is specified AND
+ * If-Match is not a "*" AND
+ * Etag is missing or weak or not in the list THEN
+ * return 412 Precondition Failed
+ */
+
+ if (c->im) {
+ if (strcmp(c->im, "*") &&
+ (!etag || (strlen(etag) > 1 && 'W' == etag[0] && '/' == etag[1]) || !ap_proxy_liststr(c->im, etag, NULL))) {
+ Explain0("If-Match specified, and it didn't - return 412");
+ }
+ else {
+ Explain0("If-Match specified, and it matched");
+ break;
+ }
+ }
+
+ /* check If-Unmodified-Since
+ *
+ * if If-Unmodified-Since is specified AND
+ * Last-Modified is specified somewhere AND
+ * If-Unmodified-Since is in the past compared to Last-Modified THEN
+ * return 412 Precondition Failed
+ */
+ if (BAD_DATE != c->ius && BAD_DATE != c->lmod) {
+ if (c->ius < c->lmod) {
+ Explain0("If-Unmodified-Since specified, but it wasn't - return 412");
+ }
+ else {
+ Explain0("If-Unmodified-Since specified, and it was unmodified");
+ break;
+ }
+ }
+
+ /* if cache file is being updated */
+ if (c->origfp) {
+ ap_proxy_write_headers(c, c->resp_line, c->hdrs);
+ ap_proxy_send_fb(c->origfp, r, c, c->len, 1);
+ ap_pclosef(r->pool, ap_bfileno(c->origfp, B_WR));
+ ap_proxy_cache_tidy(c);
+ }
+ else
+ ap_pclosef(r->pool, ap_bfileno(cachefp, B_WR));
+
+ Explain0("Use your cached copy, conditional precondition failed.");
+ return HTTP_PRECONDITION_FAILED;
+ }
+
+
+ /* check for If-None-Match, If-Modified-Since
+ *
+ */
+ while (1) {
+
+ /* check for existance of If-None-Match and If-Modified-Since
+ *
+ * if neither of these headers have been set, then the request
+ * is not conditional, and we just send the cached response and
+ * be done with it.
+ */
+ if (!c->inm && BAD_DATE == c->ims) {
+ break;
+ }
+
+ /* check If-None-Match
+ *
+ * we check if the Etag on the cached file is in the list of Etags
+ * in the If-None-Match field. The comparison must be a strong comparison,
+ * so the Etag cannot be marked as weak. If the comparision fails
+ * we return 412 Precondition Failed.
+ *
+ * if If-None-Match is specified:
+ * if If-None-Match is a "*" THEN 304
+ * else if Etag is specified AND we get a match THEN 304
+ * else if Weak Etag is specified AND we get a match THEN 304
+ * else sent the original object
+ */
+ if (c->inm) {
+ if (!strcmp(c->inm, "*")) {
+ Explain0("If-None-Match: * specified, return 304");
+ }
+ else if (etag && ap_proxy_liststr(c->inm, etag, NULL)) {
+ Explain0("If-None-Match: specified and we got a strong match - return 304");
+ }
+ else if (wetag && ap_proxy_liststr(c->inm, wetag, NULL)) {
+ Explain0("If-None-Match specified, and we got a weak match - return 304");
+ }
+ else
+ break;
+ }
+
+ /* check If-Modified-Since
+ *
+ * if If-Modified-Since is specified AND
+ * Last-Modified is specified somewhere:
+ * if last modification date is earlier than If-Modified-Since THEN 304
+ * else send the original object
+ */
+ if (BAD_DATE != c->ims && BAD_DATE != c->lmod) {
+ if (c->ims >= c->lmod) {
+ Explain0("If-Modified-Since specified and not modified, try return 304");
+ }
+ else
+ break;
+ }
+
+
+ /* are we updating the cache file? */
+ if (c->origfp) {
+ ap_proxy_write_headers(c, c->resp_line, c->hdrs);
+ ap_proxy_send_fb(c->origfp, r, c, c->len, 1);
+ ap_pclosef(r->pool, ap_bfileno(c->origfp, B_WR));
+ ap_proxy_cache_tidy(c);
+ }
+ else
+ ap_pclosef(r->pool, ap_bfileno(cachefp, B_WR));
+
+ Explain0("Use local copy, cached file hasn't changed");
+ return HTTP_NOT_MODIFIED;
+ }
+
+
+ /* No conditional - just send it cousin! */
+ Explain0("Local copy modified, send it");
+ r->status_line = strchr(c->resp_line, ' ') + 1;
+ r->status = c->status;
+
+ /* Prepare and send headers to client */
+ ap_overlap_tables(r->headers_out, c->hdrs, AP_OVERLAP_TABLES_SET);
+ ap_table_setn(r->headers_out, "X-Cache", c->xcache);
+ r->content_type = ap_table_get(r->headers_out, "Content-Type");
+ ap_send_http_header(r);
+
+ /* are we rewriting the cache file? */
+ if (c->origfp) {
+ ap_proxy_write_headers(c, c->resp_line, c->hdrs);
+ ap_proxy_send_fb(c->origfp, r, c, c->len, r->header_only);
+ ap_pclosef(r->pool, ap_bfileno(c->origfp, B_WR));
+ ap_proxy_cache_tidy(c);
+ return OK;
+ }
+
+ /* no, we not */
+ if (!r->header_only)
+ ap_proxy_send_fb(cachefp, r, NULL, c->len, 0);
+
+ ap_pclosef(r->pool, ap_bfileno(cachefp, B_WR));
+ return OK;
+}
+
/*
* Call this to test for a resource in the cache
@@ -693,7 +914,7 @@
* if cached file is not expired then
* if last modified after if-modified-since then send body
* else send 304 Not modified
- * else
+ * else if cached file is expired then
* if last modified after if-modified-since then add
* last modified date to request
*/
@@ -701,65 +922,114 @@
cache_req **cr)
{
char hashfile[66];
- const char *imstr, *pragma, *auth;
+ const char *datestr, *pragma_req = NULL, *pragma_cresp = NULL, *cc_req = NULL, *cc_cresp = NULL, *vary = NULL;
cache_req *c;
time_t now;
BUFF *cachefp;
- int cfd, i;
- const long int zero = 0L;
+ int i;
void *sconf = r->server->module_config;
proxy_server_conf *pconf =
(proxy_server_conf *) ap_get_module_config(sconf, &proxy_module);
+ const char *agestr = NULL;
+ char *val;
+ time_t age_c = 0;
+ time_t age, maxage_req, maxage_cresp, maxage, smaxage, maxstale, minfresh;
c = ap_pcalloc(r->pool, sizeof(cache_req));
*cr = c;
c->req = r;
c->url = ap_pstrdup(r->pool, url);
+ c->filename = NULL;
+ c->tempfile = NULL;
+ c->fp = NULL;
+ c->origfp = NULL;
+ c->version = 0;
+ c->len = -1;
+ c->req_hdrs = NULL;
+ c->hdrs = NULL;
+ c->xcache = NULL;
-/* get the If-Modified-Since date of the request */
+ /* get the If-Modified-Since date of the request, if it exists */
c->ims = BAD_DATE;
- imstr = ap_table_get(r->headers_in, "If-Modified-Since");
- if (imstr != NULL) {
+ datestr = ap_table_get(r->headers_in, "If-Modified-Since");
+ if (datestr != NULL) {
/* this may modify the value in the original table */
- imstr = ap_proxy_date_canon(r->pool, imstr);
- c->ims = ap_parseHTTPdate(imstr);
+ datestr = ap_proxy_date_canon(r->pool, datestr);
+ c->ims = ap_parseHTTPdate(datestr);
if (c->ims == BAD_DATE) /* bad or out of range date; remove it */
ap_table_unset(r->headers_in, "If-Modified-Since");
}
+ /* get the If-Unmodified-Since date of the request, if it exists */
+ c->ius = BAD_DATE;
+ datestr = ap_table_get(r->headers_in, "If-Unmodified-Since");
+ if (datestr != NULL) {
+ /* this may modify the value in the original table */
+ datestr = ap_proxy_date_canon(r->pool, datestr);
+ c->ius = ap_parseHTTPdate(datestr);
+ if (c->ius == BAD_DATE) /* bad or out of range date; remove it */
+ ap_table_unset(r->headers_in, "If-Unmodified-Since");
+ }
+
+ /* get the If-Match of the request, if it exists */
+ c->im = ap_table_get(r->headers_in, "If-Match");
+
+ /* get the If-None-Match of the request, if it exists */
+ c->inm = ap_table_get(r->headers_in, "If-None-Match");
+
/* find the filename for this cache entry */
ap_proxy_hash(url, hashfile, pconf->cache.dirlevels, pconf->cache.dirlength);
if (conf->root != NULL)
c->filename = ap_pstrcat(r->pool, conf->root, "/", hashfile, NULL);
- else
+ else {
+ c->filename = NULL;
+ c->fp = NULL;
+ Explain0("No CacheRoot, so no caching. Declining.");
+ return DECLINED;
+ }
+
+ /* find certain cache controlling headers */
+ pragma_req = ap_table_get(r->headers_in, "Pragma");
+ cc_req = ap_table_get(r->headers_in, "Cache-Control");
+
+
+ /* first things first - does the request allow us to return
+ * cached information at all? If not, just decline the request.
+ *
+ * Note that there is a big difference between not being allowed
+ * to cache a request (no-store) and not being allowed to return
+ * a cached request without revalidation (max-age=0).
+ *
+ * Caching is forbidden under the following circumstances:
+ *
+ * - RFC2616 14.9.2 Cache-Control: no-store
+ * we are not supposed to store this request at all. Behave as a tunnel.
+ *
+ */
+ if (ap_proxy_liststr(cc_req, "no-store", NULL)) {
+
+ /* delete the previously cached file */
+ if (c->filename)
+ unlink(c->filename);
+ c->fp = NULL;
c->filename = NULL;
+ Explain0("no-store forbids caching. Declining.");
+ return DECLINED;
+ }
+
+ /* if the cache file exists, open it */
cachefp = NULL;
-/* find out about whether the request can access the cache */
- pragma = ap_table_get(r->headers_in, "Pragma");
- auth = ap_table_get(r->headers_in, "Authorization");
- Explain5("Request for %s, pragma=%s, auth=%s, ims=%ld, imstr=%s", url,
- pragma, auth, (long)c->ims, imstr);
+ Explain3("Request for %s, pragma_req=%s, ims=%ld", url,
+ pragma_req, c->ims);
if (c->filename != NULL && r->method_number == M_GET &&
- strlen(url) < 1024 && !ap_proxy_liststr(pragma, "no-cache") &&
- auth == NULL) {
- Explain1("Check file %s", c->filename);
- cfd = open(c->filename, O_RDWR | O_BINARY);
- if (cfd != -1) {
- ap_note_cleanups_for_fd(r->pool, cfd);
- cachefp = ap_bcreate(r->pool, B_RD | B_WR);
- ap_bpushfd(cachefp, cfd, cfd);
- }
- else if (errno != ENOENT)
- ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
- "proxy: error opening cache file %s",
- c->filename);
-#ifdef EXPLAIN
- else
- Explain1("File %s not found", c->filename);
-#endif
+ strlen(url) < 1024 ) {
+
+ cachefp = ap_proxy_open_cachefile(r, c->filename);
}
+
+ /* if a cache file exists, try read body and headers from cache file */
if (cachefp != NULL) {
i = rdcache(r, cachefp, c);
if (i == -1)
@@ -773,68 +1043,237 @@
ap_pclosef(r->pool, ap_bfileno(cachefp, B_WR));
cachefp = NULL;
}
+ if (c->hdrs) {
+ cc_cresp = ap_table_get(c->hdrs, "Cache-Control");
+ pragma_cresp = ap_table_get(c->hdrs, "Pragma");
+ vary = ap_table_get(c->hdrs, "Vary");
+ if ((agestr = ap_table_get(c->hdrs, "Age"))) {
+ age_c = atoi(agestr);
+ }
}
+ }
+
+ /* if a cache file does not exist, create empty header array */
/* fixed? in this case, we want to get the headers from the remote server
it will be handled later if we don't do this (I hope ;-)
+
if (cachefp == NULL)
c->hdrs = ap_make_table(r->pool, 20);
*/
/* FIXME: Shouldn't we check the URL somewhere? */
- now = time(NULL);
-/* Ok, have we got some un-expired data? */
- if (cachefp != NULL && c->expire != BAD_DATE && now < c->expire) {
- Explain0("Unexpired data available");
-/* check IMS */
- if (c->lmod != BAD_DATE && c->ims != BAD_DATE && c->ims >= c->lmod) {
-/* has the cached file changed since this request? */
- if (c->date == BAD_DATE || c->date > c->ims) {
-/* No, but these header values may have changed, so we send them with the
- * 304 HTTP_NOT_MODIFIED response
+
+
+
+ /* Check Content-Negotiation - Vary
+ *
+ * At this point we need to make sure that the object we found in the cache
+ * is the same object that would be delivered to the client, when the
+ * effects of content negotiation are taken into effect.
+ *
+ * In plain english, we want to make sure that a language-negotiated
+ * document in one language is not given to a client asking for a
+ * language negotiated document in a different language by mistake.
+ *
+ * RFC2616 13.6 and 14.44 describe the Vary mechanism.
*/
- const char *q;
+ if (c->hdrs && c->req_hdrs) {
+ char *vary = ap_pstrdup(r->pool, ap_table_get(c->hdrs, "Vary"));
- if ((q = ap_table_get(c->hdrs, "Expires")) != NULL)
- ap_table_set(r->headers_out, "Expires", q);
+ while (vary && *vary) {
+ char *name = vary;
+ const char *h1, *h2;
+
+ /* isolate header name */
+ while (*vary && !ap_isspace(*vary) && (*vary != ','))
+ ++vary;
+ while (*vary && (ap_isspace(*vary) || (*vary == ','))) {
+ *vary = '\0';
+ ++vary;
}
- ap_pclosef(r->pool, ap_bfileno(cachefp, B_WR));
- Explain0("Use local copy, cached file hasn't changed");
- return HTTP_NOT_MODIFIED;
+
+ /* is this header in the request and the header in the cached
+ * request identical? If not, we give up and do a straight get */
+ h1 = ap_table_get(r->headers_in, name);
+ h2 = ap_table_get(c->req_hdrs, name);
+ if (h1 == h2) {
+ /* both headers NULL, so a match - do nothing */
}
+ else if (h1 && h2 && !strcmp(h1, h2)) {
+ /* both headers exist and are equal - do nothing */
+ }
+ else {
+
+ /* headers do not match, so Vary failed */
+ c->fp = cachefp;
+ Explain0("Vary header mismatch - object must be fetched from scratch. Declining.");
+ return DECLINED;
-/* Ok, has been modified */
- Explain0("Local copy modified, send it");
- r->status_line = strchr(c->resp_line, ' ') + 1;
- r->status = c->status;
- if (!r->assbackwards) {
- ap_soft_timeout("proxy send headers", r);
- ap_proxy_send_headers(r, c->resp_line, c->hdrs);
- ap_kill_timeout(r);
}
- ap_bsetopt(r->connection->client, BO_BYTECT, &zero);
- r->sent_bodyct = 1;
- if (!r->header_only)
- ap_proxy_send_fb(cachefp, r, NULL);
- ap_pclosef(r->pool, ap_bfileno(cachefp, B_WR));
- return OK;
+ }
}
-/* if we already have data and a last-modified date, and it is not a head
- * request, then add an If-Modified-Since
+
+ /* We now want to check if our cached data is still fresh. This depends
+ * on a few things, in this order:
+ *
+ * - RFC2616 14.9.4 End to end reload, Cache-Control: no-cache
+ * no-cache in either the request or the cached response means that
+ * we must revalidate the request unconditionally, overriding any
+ * expiration mechanism. It's equivalent to max-age=0,must-revalidate.
+ *
+ * - RFC2616 14.32 Pragma: no-cache
+ * This is treated the same as Cache-Control: no-cache.
+ *
+ * - RFC2616 14.9.3 Cache-Control: max-stale, must-revalidate, proxy-revalidate
+ * if the max-stale request header exists, modify the stale calculations
+ * below so that an object can be at most <max-stale> seconds stale before
+ * we request a revalidation, _UNLESS_ a must-revalidate or
+ * proxy-revalidate cached response header exists to stop us doing this.
+ *
+ * - RFC2616 14.9.3 Cache-Control: s-maxage
+ * the origin server specifies the maximum age an object can be before
+ * it is considered stale. This directive has the effect of proxy|must
+ * revalidate, which in turn means simple ignore any max-stale setting.
+ *
+ * - RFC2616 14.9.4 Cache-Control: max-age
+ * this header can appear in both requests and responses. If both are
+ * specified, the smaller of the two takes priority.
+ *
+ * - RFC2616 14.21 Expires:
+ * if this request header exists in the cached entity, and it's value is
+ * in the past, it has expired.
+ *
*/
- if (cachefp != NULL && c->lmod != BAD_DATE && !r->header_only) {
-/*
- * use the later of the one from the request and the last-modified date
- * from the cache
+ /* calculate age of object */
+ age = ap_proxy_current_age(c, age_c);
+
+ /* extract s-maxage */
+ if (cc_cresp && ap_proxy_liststr(cc_cresp, "s-maxage", &val))
+ smaxage = atoi(val);
+ else
+ smaxage = -1;
+
+ /* extract max-age from request */
+ if (cc_cresp && ap_proxy_liststr(cc_req, "max-age", &val))
+ maxage_req = atoi(val);
+ else
+ maxage_req = -1;
+
+ /* extract max-age from response */
+ if (cc_cresp && ap_proxy_liststr(cc_cresp, "max-age", &val))
+ maxage_cresp = atoi(val);
+ else
+ maxage_cresp = -1;
+
+ /* if both maxage request and response, the smaller one takes priority */
+ if (-1 == maxage_req)
+ maxage = maxage_cresp;
+ else if (-1 == maxage_cresp)
+ maxage = maxage_req;
+ else
+ maxage = MIN(maxage_req, maxage_cresp);
+
+ /* extract max-stale */
+ if (cc_req && ap_proxy_liststr(cc_req, "max-stale", &val))
+ maxstale = atoi(val);
+ else
+ maxstale = 0;
+
+ /* extract min-fresh */
+ if (cc_req && ap_proxy_liststr(cc_req, "min-fresh", &val))
+ minfresh = atoi(val);
+ else
+ minfresh = 0;
+
+ /* override maxstale if must-revalidate or proxy-revalidate */
+ if (maxstale && ( (cc_cresp && ap_proxy_liststr(cc_cresp, "must-revalidate", NULL)) || (cc_cresp && ap_proxy_liststr(cc_cresp, "proxy-revalidate", NULL)) ))
+ maxstale = 0;
+
+ now = time(NULL);
+ if (cachefp != NULL &&
+
+ /* handle no-cache */
+ !( (cc_req && ap_proxy_liststr(cc_req, "no-cache", NULL)) ||
+ (pragma_req && ap_proxy_liststr(pragma_req, "no-cache", NULL)) ||
+ (cc_cresp && ap_proxy_liststr(cc_cresp, "no-cache", NULL)) ||
+ (pragma_cresp && ap_proxy_liststr(pragma_cresp, "no-cache", NULL)) ) &&
+
+ /* handle expiration */
+ ( (-1 < smaxage && age < (smaxage - minfresh)) ||
+ (-1 < maxage && age < (maxage + maxstale - minfresh)) ||
+ (c->expire != BAD_DATE && age < (c->expire - c->date + maxstale - minfresh)) )
+
+ ) {
+
+ /* it's fresh darlings... */
+
+ Explain0("Unexpired data available");
+
+ /* set age header on response */
+ ap_table_set(c->hdrs, "Age",
+ ap_psprintf(r->pool, "%lu", (unsigned long)age));
+
+ /* add warning if maxstale overrode freshness calculation */
+ if (!( (-1 < smaxage && age < smaxage) ||
+ (-1 < maxage && age < maxage) ||
+ (c->expire != BAD_DATE && (c->expire - c->date) > age) )) {
+ ap_table_set(c->hdrs, "Warning", "110 Response is stale");
+ }
+
+ /* check conditionals (If-Modified-Since, etc) */
+ c->xcache = ap_pstrcat(r->pool, "HIT from ", ap_get_server_name(r), NULL);
+ return ap_proxy_cache_conditional(r, c, cachefp);
+
+
+ }
+
+ /* at this point we have determined our cached data needs revalidation
+ * but first - we check 1 thing:
+ *
+ * RFC2616 14.9.4 - if "only-if-cached" specified, send a
+ * 504 Gateway Timeout - we're not allowed to revalidate the object
+ */
+ if (ap_proxy_liststr(cc_req, "only-if-cached", NULL)) {
+ if (cachefp)
+ ap_pclosef(r->pool, ap_bfileno(cachefp, B_WR));
+ return HTTP_GATEWAY_TIME_OUT;
+ }
+
+
+ /* If we already have cached data and a last-modified date, and it is
+ * not a head request, then add an If-Modified-Since.
+ *
+ * If we also have an Etag, then the object must have come from
+ * an HTTP/1.1 server. Add an If-None-Match as well.
+ *
+ * See RFC2616 13.3.4
*/
+
+ if (cachefp != NULL && !r->header_only) {
+
+ const char *etag = ap_table_get(c->hdrs, "Etag");
+
+ /* If-Modified-Since */
+ if (c->lmod != BAD_DATE) {
+ /* use the later of the one from the request and the last-modified date
+ * from the cache */
if (c->ims == BAD_DATE || c->ims < c->lmod) {
const char *q;
if ((q = ap_table_get(c->hdrs, "Last-Modified")) != NULL)
- ap_table_set(r->headers_in, "If-Modified-Since",
- (char *) q);
+ ap_table_set(r->headers_in, "If-Modified-Since", (char *) q);
+ }
}
+
+ /* If-None-Match */
+ if (etag) {
+ ap_table_set(r->headers_in, "If-None-Match", etag);
}
+
+ }
+
+
c->fp = cachefp;
Explain0("Local copy not present or expired. Declining.");
@@ -862,30 +1301,30 @@
#endif
request_rec *r = c->req;
char *p;
- int i;
const char *expire, *lmods, *dates, *clen;
time_t expc, date, lmod, now;
- char buff[46];
+ char buff[17*7+1];
void *sconf = r->server->module_config;
proxy_server_conf *conf =
(proxy_server_conf *) ap_get_module_config(sconf, &proxy_module);
- const long int zero = 0L;
+ const char *cc_resp;
+ table *req_hdrs;
+
+ cc_resp = ap_table_get(resp_hdrs, "Cache-Control");
c->tempfile = NULL;
-/* we've received the response */
+ /* we've received the response from the origin server */
+
/* read expiry date; if a bad date, then leave it so the client can
- * read it
- */
+ * read it */
expire = ap_table_get(resp_hdrs, "Expires");
if (expire != NULL)
expc = ap_parseHTTPdate(expire);
else
expc = BAD_DATE;
-/*
- * read the last-modified date; if the date is bad, then delete it
- */
+ /* read the last-modified date; if the date is bad, then delete it */
lmods = ap_table_get(resp_hdrs, "Last-Modified");
if (lmods != NULL) {
lmod = ap_parseHTTPdate(lmods);
@@ -897,40 +1336,87 @@
else
lmod = BAD_DATE;
+
/*
* what responses should we not cache?
- * Unknown status responses and those known to be uncacheable
- * 304 HTTP_NOT_MODIFIED response when we have no valid cache file, or
- * 200 HTTP_OK response from HTTP/1.0 and up without a Last-Modified header, or
- * HEAD requests, or
- * requests with an Authorization header, or
- * protocol requests nocache (e.g. ftp with user/password)
- */
-/* @@@ XXX FIXME: is the test "r->status != HTTP_MOVED_PERMANENTLY" correct?
- * or shouldn't it be "ap_is_HTTP_REDIRECT(r->status)" ? -MnKr */
- if ((r->status != HTTP_OK && r->status != HTTP_MOVED_PERMANENTLY && r->status != HTTP_NOT_MODIFIED) ||
+ *
+ * At this point we decide based on the response headers whether it
+ * is appropriate _NOT_ to cache the data from the server. There are
+ * a whole lot of conditions that prevent us from caching this data.
+ * They are tested here one by one to be clear and unambiguous. */
+
+ /* RFC2616 13.4 we are allowed to cache 200, 203, 206, 300, 301 or 410
+ * We don't cache 206, because we don't (yet) cache partial responses.
+ * We include 304 Not Modified here too as this is the origin server
+ * telling us to serve the cached copy. */
+ if ((r->status != HTTP_OK && r->status != HTTP_NON_AUTHORITATIVE && r->status != HTTP_MULTIPLE_CHOICES && r->status != HTTP_MOVED_PERMANENTLY && r->status != HTTP_NOT_MODIFIED) ||
+
+ /* if a broken Expires header is present, don't cache it */
(expire != NULL && expc == BAD_DATE) ||
+
+ /* if the server said 304 Not Modified but we have no cache file - pass
+ * this untouched to the user agent, it's not for us. */
(r->status == HTTP_NOT_MODIFIED && (c == NULL || c->fp == NULL)) ||
+
+ /* 200 OK response from HTTP/1.0 and up without a Last-Modified header */
(r->status == HTTP_OK && lmods == NULL && is_HTTP1) ||
+
+ /* HEAD requests */
r->header_only ||
- ap_table_get(r->headers_in, "Authorization") != NULL ||
+
+ /* RFC2616 14.9.2 Cache-Control: no-store response indicating do not
+ * cache, or stop now if you are trying to cache it */
+ ap_proxy_liststr(cc_resp, "no-store", NULL) ||
+
+ /* RFC2616 14.9.1 Cache-Control: private
+ * this object is marked for this user's eyes only. Behave as a tunnel. */
+ ap_proxy_liststr(cc_resp, "private", NULL) ||
+
+ /* RFC2616 14.8 Authorisation:
+ * if authorisation is included in the request, we don't cache, but we
+ * can cache if the following exceptions are true:
+ * 1) If Cache-Control: s-maxage is included
+ * 2) If Cache-Control: must-revalidate is included
+ * 3) If Cache-Control: public is included
+ */
+ (ap_table_get(r->headers_in, "Authorization") != NULL
+
+ && !(ap_proxy_liststr(cc_resp, "s-maxage", NULL) || ap_proxy_liststr(cc_resp, "must-revalidate", NULL) || ap_proxy_liststr(cc_resp, "public", NULL))
+ ) ||
+
+ /* or we've been asked not to cache it above */
nocache) {
+
Explain1("Response is not cacheable, unlinking %s", c->filename);
+
/* close the file */
if (c->fp != NULL) {
ap_pclosef(r->pool, ap_bfileno(c->fp, B_WR));
c->fp = NULL;
}
+
/* delete the previously cached file */
if (c->filename)
unlink(c->filename);
return DECLINED; /* send data to client but not cache */
}
-/* otherwise, we are going to cache the response */
-/*
- * Read the date. Generate one if one is not supplied
+
+
+
+
+ /* It's safe to cache the response.
+ *
+ * We now want to update the cache file header information with
+ * the new date, last modified, expire and content length and write
+ * it away to our cache file. First, we determine these values from
+ * the response, using heuristics if appropriate.
+ *
+ * In addition, we make HTTP/1.1 age calculations and write them away
+ * too.
*/
+
+ /* Read the date. Generate one if one is not supplied */
dates = ap_table_get(resp_hdrs, "Date");
if (dates != NULL)
date = ap_parseHTTPdate(dates);
@@ -949,14 +1435,18 @@
Explain0("Added date header");
}
+ /* set response_time for HTTP/1.1 age calculations */
+ c->resp_time = now;
+
/* check last-modified date */
- if (lmod != BAD_DATE && lmod > date)
/* if its in the future, then replace by date */
+ if (lmod != BAD_DATE && lmod > date)
{
lmod = date;
lmods = dates;
Explain0("Last modified is in the future, replacing with now");
}
+
/* if the response did not contain the header, then use the cached version */
if (lmod == BAD_DATE && c->fp != NULL) {
lmod = c->lmod;
@@ -969,6 +1459,7 @@
if (expire != NULL)
expc = ap_parseHTTPdate(expire);
}
+
/* so we now have the expiry date */
/* if no expiry date then
* if lastmod
@@ -976,7 +1467,7 @@
* else
* expire date = now + defaultexpire
*/
- Explain1("Expiry date is %ld", (long)expc);
+ Explain1("Expiry date is %ld", expc);
if (expc == BAD_DATE) {
if (lmod != BAD_DATE) {
double x = (double) (date - lmod) * conf->cache.lmfactor;
@@ -997,121 +1488,168 @@
else
c->len = atoi(clen);
- ap_proxy_sec2hex(date, buff);
- buff[8] = ' ';
- ap_proxy_sec2hex(lmod, buff + 9);
- buff[17] = ' ';
- ap_proxy_sec2hex(expc, buff + 18);
- buff[26] = ' ';
- ap_proxy_sec2hex(c->version++, buff + 27);
- buff[35] = ' ';
- ap_proxy_sec2hex(c->len, buff + 36);
- buff[44] = '\n';
- buff[45] = '\0';
+ /* we have all the header information we need - write it to the cache file */
+ c->version++;
+ ap_proxy_sec2hex(date, buff + 17*(0));
+ buff[17*(1)-1] = ' ';
+ ap_proxy_sec2hex(lmod, buff + 17*(1));
+ buff[17*(2)-1] = ' ';
+ ap_proxy_sec2hex(expc, buff + 17*(2));
+ buff[17*(3)-1] = ' ';
+ ap_proxy_sec2hex(c->version, buff + 17*(3));
+ buff[17*(4)-1] = ' ';
+ ap_proxy_sec2hex(c->req_time, buff + 17*(4));
+ buff[17*(5)-1] = ' ';
+ ap_proxy_sec2hex(c->resp_time, buff + 17*(5));
+ buff[17*(6)-1] = ' ';
+ ap_proxy_sec2hex(c->len, buff + 17*(6));
+ buff[17*(7)-1] = '\n';
+ buff[17*(7)] = '\0';
-/* if file not modified */
+
+ /* Was the server response a 304 Not Modified?
+ *
+ * If it was, it means that we requested a revalidation, and that
+ * the result of that revalidation was that the object was fresh.
+ *
+ */
+
+ /* if response from server 304 not modified */
if (r->status == HTTP_NOT_MODIFIED) {
- if (c->ims != BAD_DATE && lmod != BAD_DATE && lmod <= c->ims) {
-/* set any changed headers somehow */
-/* update dates and version, but not content-length */
- if (lmod != c->lmod || expc != c->expire || date != c->date) {
- off_t curpos = lseek(ap_bfileno(c->fp, B_WR), 0, SEEK_SET);
- if (curpos == -1)
- ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
- "proxy: error seeking on cache file %s",
- c->filename);
- else if (write(ap_bfileno(c->fp, B_WR), buff, 35) == -1)
- ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
- "proxy: error updating cache file %s",
- c->filename);
- }
- ap_pclosef(r->pool, ap_bfileno(c->fp, B_WR));
- Explain0("Remote document not modified, use local copy");
- /* CHECKME: Is this right? Shouldn't we check IMS again here? */
- return HTTP_NOT_MODIFIED;
- }
- else {
-/* return the whole document */
- Explain0("Remote document updated, sending");
- r->status_line = strchr(c->resp_line, ' ') + 1;
- r->status = c->status;
- if (!r->assbackwards) {
- ap_soft_timeout("proxy send headers", r);
- ap_proxy_send_headers(r, c->resp_line, c->hdrs);
- ap_kill_timeout(r);
- }
- ap_bsetopt(r->connection->client, BO_BYTECT, &zero);
- r->sent_bodyct = 1;
- if (!r->header_only)
- ap_proxy_send_fb(c->fp, r, NULL);
-/* set any changed headers somehow */
-/* update dates and version, but not content-length */
- if (lmod != c->lmod || expc != c->expire || date != c->date) {
- off_t curpos = lseek(ap_bfileno(c->fp, B_WR), 0, SEEK_SET);
- if (curpos == -1)
- ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
- "proxy: error seeking on cache file %s",
- c->filename);
- else if (write(ap_bfileno(c->fp, B_WR), buff, 35) == -1)
- ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
- "proxy: error updating cache file %s",
- c->filename);
+ /* Have the headers changed?
+ *
+ * if not - we fulfil the request and return now.
+ */
+
+ if (c->hdrs) {
+ if (!ap_replace_tables(c->hdrs, resp_hdrs)) {
+ c->xcache = ap_pstrcat(r->pool, "HIT from ", ap_get_server_name(r), " (with revalidation)", NULL);
+ return ap_proxy_cache_conditional(r, c, c->fp);
}
- ap_pclosef(r->pool, ap_bfileno(c->fp, B_WR));
- return OK;
}
+ else
+ c->hdrs = resp_hdrs;
+
+
+ /* if we get here - the headers have changed. Go through the motions
+ * of creating a new temporary cache file below, we'll then serve
+ * the request like we would have in ap_proxy_cache_conditional()
+ * above, and at the same time we will also rewrite the contents
+ * to the new temporary file.
+ */
+
}
-/* new or modified file */
+
+
+
+ /*
+ * Ok - lets prepare and open the cached file
+ *
+ * If a cached file (in c->fp) is already open, then we want to
+ * update that cached file. Copy the c->fp to c->origfp and open
+ * up a new one.
+ *
+ * If the cached file (in c->fp) is NULL, we must open a new cached
+ * file from scratch.
+ *
+ * The new cache file will be moved to it's final location in the
+ * directory tree later, overwriting the old cache file should it exist.
+ */
+
+ /* if a cache file was already open */
if (c->fp != NULL) {
- ap_pclosef(r->pool, ap_bfileno(c->fp, B_WR));
+ c->origfp = c->fp;
}
- c->version = 0;
- ap_proxy_sec2hex(0, buff + 27);
- buff[35] = ' ';
-/* open temporary file */
-#if !defined(TPF) && !defined(NETWARE)
+ while (1) {
+ /* create temporary filename */
+#ifndef TPF
#define TMPFILESTR "/tmpXXXXXX"
- if (conf->cache.root == NULL)
- return DECLINED;
+ if (conf->cache.root == NULL) {
+ c = ap_proxy_cache_error(c);
+ break;
+ }
c->tempfile = ap_palloc(r->pool, strlen(conf->cache.root) + sizeof(TMPFILESTR));
strcpy(c->tempfile, conf->cache.root);
strcat(c->tempfile, TMPFILESTR);
#undef TMPFILESTR
p = mktemp(c->tempfile);
#else
- if (conf->cache.root == NULL)
- return DECLINED;
+ if (conf->cache.root == NULL) {
+ c = ap_proxy_cache_error(c);
+ break;
+ }
c->tempfile = ap_palloc(r->pool, strlen(conf->cache.root) +1+ L_tmpnam);
strcpy(c->tempfile, conf->cache.root);
strcat(c->tempfile, "/");
p = tmpnam(NULL);
strcat(c->tempfile, p);
#endif
- if (p == NULL)
- return DECLINED;
+ if (p == NULL) {
+ c = ap_proxy_cache_error(c);
+ break;
+ }
Explain1("Create temporary file %s", c->tempfile);
- i = open(c->tempfile, O_WRONLY | O_CREAT | O_EXCL | O_BINARY, 0622);
- if (i == -1) {
- ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
- "proxy: error creating cache file %s",
- c->tempfile);
- return DECLINED;
+ /* create the new file */
+ c->fp = ap_proxy_create_cachefile(r, c->tempfile);
+ if (NULL == c->fp) {
+ c = ap_proxy_cache_error(c);
+ break;
}
- ap_note_cleanups_for_fd(r->pool, i);
- c->fp = ap_bcreate(r->pool, B_WR);
- ap_bpushfd(c->fp, -1, i);
+ /* write away the cache header and the URL */
if (ap_bvputs(c->fp, buff, "X-URL: ", c->url, "\n", NULL) == -1) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
"proxy: error writing cache file(%s)", c->tempfile);
- ap_pclosef(r->pool, ap_bfileno(c->fp, B_WR));
- unlink(c->tempfile);
- c->fp = NULL;
+ c = ap_proxy_cache_error(c);
+ break;
+ }
+
+
+ /* get original request headers */
+ if (c->req_hdrs)
+ req_hdrs = ap_copy_table(r->pool, c->req_hdrs);
+ else
+ req_hdrs = ap_copy_table(r->pool, r->headers_in);
+
+ /* remove hop-by-hop headers */
+ ap_proxy_clear_connection(r->pool, req_hdrs);
+
+ /* save original request headers */
+ if (c->req_hdrs)
+ ap_table_do(ap_proxy_send_hdr_line, c, c->req_hdrs, NULL);
+ else
+ ap_table_do(ap_proxy_send_hdr_line, c, r->headers_in, NULL);
+ if (ap_bputs(CRLF, c->fp) == -1) {
+ ap_log_rerror(APLOG_MARK, APLOG_ERR, c->req,
+ "proxy: error writing request headers terminating CRLF to %s", c->tempfile);
+ c = ap_proxy_cache_error(c);
+ break;
+ }
+
+ break;
}
+
+
+ /* Was the server response a 304 Not Modified?
+ *
+ * If so, we have some work to do that we didn't do when we first
+ * checked above. We need to fulfil the request, and we need to
+ * copy the body from the old object to the new one.
+ */
+
+ /* if response from server 304 not modified */
+ if (r->status == HTTP_NOT_MODIFIED) {
+
+ /* fulfil the request */
+ c->xcache = ap_pstrcat(r->pool, "HIT from ", ap_get_server_name(r), " (with revalidation)", NULL);
+ return ap_proxy_cache_conditional(r, c, c->fp);
+
+ }
+
return DECLINED;
}
@@ -1147,17 +1685,17 @@
*/
else {
/* update content-length of file */
- char buff[9];
+ char buff[17];
off_t curpos;
c->len = bc;
ap_bflush(c->fp);
ap_proxy_sec2hex(c->len, buff);
- curpos = lseek(ap_bfileno(c->fp, B_WR), 36, SEEK_SET);
+ curpos = lseek(ap_bfileno(c->fp, B_WR), 17*6, SEEK_SET);
if (curpos == -1)
ap_log_error(APLOG_MARK, APLOG_ERR, s,
"proxy: error seeking on cache file %s", c->tempfile);
- else if (write(ap_bfileno(c->fp, B_WR), buff, 8) == -1)
+ else if (write(ap_bfileno(c->fp, B_WR), buff, sizeof(buff) - 1) == -1)
ap_log_error(APLOG_MARK, APLOG_ERR, s,
"proxy: error updating cache file %s", c->tempfile);
}
diff -wurdX nodiff.pats apache-1.3.13-dev/src/modules/proxy/proxy_ftp.c apache-1.3.13-dev+http1.1/src/modules/proxy/proxy_ftp.c
--- apache-1.3.13-dev/src/modules/proxy/proxy_ftp.c Tue Feb 29 10:24:27 2000
+++ apache-1.3.13-dev+http1.1/src/modules/proxy/proxy_ftp.c Sat Aug 12 13:35:48 2000
@@ -460,9 +460,7 @@
BUFF *data = NULL;
pool *p = r->pool;
int one = 1;
- const long int zero = 0L;
NET_SIZE_T clen;
- struct tbl_do_args tdo;
void *sconf = r->server->module_config;
proxy_server_conf *conf =
@@ -610,6 +608,9 @@
strerror(errno), NULL));
}
+ /* record request_time for HTTP/1.1 age calculation */
+ c->req_time = time(NULL);
+
f = ap_bcreate(p, B_RDWR | B_SOCKET);
ap_bpushfd(f, sock, sock);
/* shouldn't we implement telnet control options here? */
@@ -1197,40 +1198,30 @@
ap_bpushfd(data, dsock, dsock);
}
- ap_hard_timeout("proxy receive", r);
/* send response */
-/* write status line */
- if (!r->assbackwards)
- ap_rvputs(r, "HTTP/1.0 ", r->status_line, CRLF, NULL);
- if (c != NULL && c->fp != NULL
- && ap_bvputs(c->fp, "HTTP/1.0 ", r->status_line, CRLF, NULL) == -1) {
- ap_log_rerror(APLOG_MARK, APLOG_ERR, c->req,
- "proxy: error writing CRLF to %s", c->tempfile);
- c = ap_proxy_cache_error(c);
- }
+ /* write status line and headers to the cache file */
+ ap_proxy_write_headers(c, ap_pstrcat(p, "HTTP/1.1 ", r->status_line, NULL), resp_hdrs);
-/* send headers */
- tdo.req = r;
- tdo.cache = c;
- ap_table_do(ap_proxy_send_hdr_line, &tdo, resp_hdrs, NULL);
+ /* Setup the headers for our client from upstreams response-headers */
+ ap_overlap_tables(r->headers_out, resp_hdrs, AP_OVERLAP_TABLES_SET);
+ /* Add X-Cache header */
+ ap_table_setn(r->headers_out, "X-Cache",
+ ap_pstrcat(r->pool, "MISS from ",
+ ap_get_server_name(r), NULL));
+ /* The Content-Type of this response is the upstream one. */
+ r->content_type = ap_table_get (r->headers_out, "Content-Type");
+ /* finally output the headers to the client */
+ ap_send_http_header(r);
- if (!r->assbackwards)
- ap_rputs(CRLF, r);
- if (c != NULL && c->fp != NULL && ap_bputs(CRLF, c->fp) == -1) {
- ap_log_rerror(APLOG_MARK, APLOG_ERR, c->req,
- "proxy: error writing CRLF to %s", c->tempfile);
- c = ap_proxy_cache_error(c);
- }
+ ap_hard_timeout("proxy receive", r);
- ap_bsetopt(r->connection->client, BO_BYTECT, &zero);
- r->sent_bodyct = 1;
/* send body */
if (!r->header_only) {
if (parms[0] != 'd') {
/* we need to set this for ap_proxy_send_fb()... */
if (c != NULL)
c->cache_completion = 0;
- ap_proxy_send_fb(data, r, c);
+ ap_proxy_send_fb(data, r, c, -1, 0);
} else
send_dir(data, r, c, cwd);
diff -wurdX nodiff.pats apache-1.3.13-dev/src/modules/proxy/proxy_http.c apache-1.3.13-dev+http1.1/src/modules/proxy/proxy_http.c
--- apache-1.3.13-dev/src/modules/proxy/proxy_http.c Tue Feb 29 10:24:27 2000
+++ apache-1.3.13-dev+http1.1/src/modules/proxy/proxy_http.c Sat Aug 12 13:36:07 2000
@@ -112,6 +112,7 @@
return OK;
}
+/* handle the conversion of URLs in the ProxyPassReverse function */
static const char *proxy_location_reverse_map(request_rec *r, const char *url)
{
void *sconf;
@@ -134,29 +135,6 @@
return url;
}
-/* Clear all connection-based headers from the incoming headers table */
-static void clear_connection(pool *p, table *headers)
-{
- const char *name;
- char *next = ap_pstrdup(p, ap_table_get(headers, "Connection"));
-
- ap_table_unset(headers, "Proxy-Connection");
- if (!next)
- return;
-
- while (*next) {
- name = next;
- while (*next && !ap_isspace(*next) && (*next != ','))
- ++next;
- while (*next && (ap_isspace(*next) || (*next == ','))) {
- *next = '\0';
- ++next;
- }
- ap_table_unset(headers, name);
- }
- ap_table_unset(headers, "Connection");
-}
-
/*
* This handles http:// URLs, and other URLs using a remote proxy over http
* If proxyhost is NULL, then contact the server directly, otherwise
@@ -173,9 +151,9 @@
char *strp2;
const char *err, *desthost;
int i, j, sock, len, backasswards;
+ table *req_hdrs, *resp_hdrs;
array_header *reqhdrs_arr;
- table *resp_hdrs;
- table_entry *reqhdrs;
+ table_entry *reqhdrs_elts;
struct sockaddr_in server;
struct in_addr destaddr;
struct hostent server_hp;
@@ -183,12 +161,10 @@
char buffer[HUGE_STRING_LEN];
char portstr[32];
pool *p = r->pool;
- const long int zero = 0L;
int destport = 0;
char *destportstr = NULL;
const char *urlptr = NULL;
- const char *datestr;
- struct tbl_do_args tdo;
+ const char *datestr, *urlstr;
void *sconf = r->server->module_config;
proxy_server_conf *conf =
@@ -251,6 +227,10 @@
return ap_proxyerror(r, HTTP_INTERNAL_SERVER_ERROR, err);
}
+
+ /* we have worked out who exactly we are going to connect to, now
+ * make that connection...
+ */
sock = ap_psocket(p, PF_INET, SOCK_STREAM, IPPROTO_TCP);
if (sock == -1) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
@@ -300,8 +280,20 @@
strerror(errno), NULL));
}
- clear_connection(r->pool, r->headers_in); /* Strip connection-based headers */
+ /* record request_time for HTTP/1.1 age calculation */
+ c->req_time = time(NULL);
+
+ /* build upstream-request headers by stripping r->headers_in from
+ * connection specific headers.
+ * We must not remove the Connection: header from r->headers_in,
+ * we still have to react to Connection: close
+ */
+ req_hdrs = ap_copy_table(r->pool, r->headers_in);
+ ap_proxy_clear_connection(r->pool, req_hdrs);
+ /* At this point, we start sending the HTTP/1.1 request to the
+ * remote server (proxy or otherwise).
+ */
f = ap_bcreate(p, B_RDWR | B_SOCKET);
ap_bpushfd(f, sock, sock);
@@ -315,7 +307,7 @@
if (conf->viaopt == via_block) {
/* Block all outgoing Via: headers */
- ap_table_unset(r->headers_in, "Via");
+ ap_table_unset(req_hdrs, "Via");
} else if (conf->viaopt != via_off) {
/* Create a "Via:" request header entry and merge it */
i = ap_get_server_port(r);
@@ -325,7 +317,7 @@
ap_snprintf(portstr, sizeof portstr, ":%d", i);
}
/* Generate outgoing Via: header with/without server comment: */
- ap_table_mergen(r->headers_in, "Via",
+ ap_table_mergen(req_hdrs, "Via",
(conf->viaopt == via_full)
? ap_psprintf(p, "%d.%d %s%s (%s)",
HTTP_VERSION_MAJOR(r->proto_num),
@@ -339,24 +331,46 @@
);
}
- reqhdrs_arr = ap_table_elts(r->headers_in);
- reqhdrs = (table_entry *) reqhdrs_arr->elts;
+ /* Add X-Forwarded-For: so that the upstream has a chance to
+ determine, where the original request came from. */
+ ap_table_mergen(req_hdrs, "X-Forwarded-For", r->connection->remote_ip);
+
+ /* we don't yet support keepalives - but we will soon, I promise! */
+ ap_table_set(req_hdrs, "Connection", "close");
+
+ reqhdrs_arr = ap_table_elts(req_hdrs);
+ reqhdrs_elts = (table_entry *) reqhdrs_arr->elts;
for (i = 0; i < reqhdrs_arr->nelts; i++) {
- if (reqhdrs[i].key == NULL || reqhdrs[i].val == NULL
- /* Clear out headers not to send */
- || !strcasecmp(reqhdrs[i].key, "Host") /* Already sent */
+ if (reqhdrs_elts[i].key == NULL || reqhdrs_elts[i].val == NULL
+
+ /* Clear out hop-by-hop request headers not to send:
+ * RFC2616 13.5.1 says we should strip these headers:
+ */
+ || !strcasecmp(reqhdrs_elts[i].key, "Host") /* Already sent */
+ || !strcasecmp(reqhdrs_elts[i].key, "Keep-Alive")
+ || !strcasecmp(reqhdrs_elts[i].key, "TE")
+ || !strcasecmp(reqhdrs_elts[i].key, "Trailer")
+ || !strcasecmp(reqhdrs_elts[i].key, "Transfer-Encoding")
+ || !strcasecmp(reqhdrs_elts[i].key, "Upgrade")
+
/* XXX: @@@ FIXME: "Proxy-Authorization" should *only* be
* suppressed if THIS server requested the authentication,
* not when a frontend proxy requested it!
+ *
+ * The solution to this problem is probably to strip out
+ * the Proxy-Authorisation header in the authorisation
+ * code itself, not here. This saves us having to signal
+ * somehow whether this request was authenticated or not.
*/
- || !strcasecmp(reqhdrs[i].key, "Proxy-Authorization"))
+ || !strcasecmp(reqhdrs_elts[i].key, "Proxy-Authorization"))
continue;
- ap_bvputs(f, reqhdrs[i].key, ": ", reqhdrs[i].val, CRLF, NULL);
+ ap_bvputs(f, reqhdrs_elts[i].key, ": ", reqhdrs_elts[i].val, CRLF, NULL);
}
+ /* the obligatory empty line to mark the end of the headers */
ap_bputs(CRLF, f);
-/* send the request data, if any. */
+ /* send the request data, if any. */
if (ap_should_client_block(r)) {
while ((i = ap_get_client_block(r, buffer, sizeof buffer)) > 0)
ap_bwrite(f, buffer, i);
@@ -364,6 +378,9 @@
ap_bflush(f);
ap_kill_timeout(r);
+
+ /* Right - now it's time to listen for a response.
+ */
ap_hard_timeout("proxy receive", r);
len = ap_bgets(buffer, sizeof buffer - 1, f);
@@ -382,12 +399,16 @@
"Document contains no data");
}
-/* Is it an HTTP/1 response? This is buggy if we ever see an HTTP/1.10 */
+ /* Is it an HTTP/1 response?
+ * Do some sanity checks on the response.
+ * (This is buggy if we ever see an HTTP/1.10)
+ */
if (ap_checkmask(buffer, "HTTP/#.# ###*")) {
int major, minor;
if (2 != sscanf(buffer, "HTTP/%u.%u", &major, &minor)) {
+ /* if no response, default to HTTP/1.1 - is this correct? */
major = 1;
- minor = 0;
+ minor = 1;
}
/* If not an HTTP/1 message or if the status line was > 8192 bytes */
@@ -404,7 +425,7 @@
buffer[12] = ' ';
r->status_line = ap_pstrdup(p, &buffer[9]);
-/* read the headers. */
+ /* read the response headers. */
/* N.B. for HTTP/1.0 clients, we have to fold line-wrapped headers */
/* Also, take care with headers with multiple occurences. */
@@ -417,6 +438,7 @@
nocache = 1; /* do not cache this broken file */
}
+ /* handle Via header in the response */
if (conf->viaopt != via_off && conf->viaopt != via_block) {
/* Create a "Via:" response header entry and merge it */
i = ap_get_server_port(r);
@@ -437,7 +459,8 @@
);
}
- clear_connection(p, resp_hdrs); /* Strip Connection hdrs */
+ /* strip hop-by-hop headers defined by Connection */
+ ap_proxy_clear_connection(p, resp_hdrs);
}
else {
/* an http/0.9 response */
@@ -449,12 +472,10 @@
resp_hdrs = ap_make_table(p, 20);
}
- c->hdrs = resp_hdrs;
-
ap_kill_timeout(r);
/*
- * HTTP/1.0 requires us to accept 3 types of dates, but only generate
+ * HTTP/1.1 requires us to accept 3 types of dates, but only generate
* one type
*/
if ((datestr = ap_table_get(resp_hdrs, "Date")) != NULL)
@@ -464,10 +485,13 @@
if ((datestr = ap_table_get(resp_hdrs, "Expires")) != NULL)
ap_table_set(resp_hdrs, "Expires", ap_proxy_date_canon(p, datestr));
- if ((datestr = ap_table_get(resp_hdrs, "Location")) != NULL)
- ap_table_set(resp_hdrs, "Location", proxy_location_reverse_map(r, datestr));
- if ((datestr = ap_table_get(resp_hdrs, "URI")) != NULL)
- ap_table_set(resp_hdrs, "URI", proxy_location_reverse_map(r, datestr));
+ /* handle the ProxyPassReverse mappings */
+ if ((urlstr = ap_table_get(resp_hdrs, "Location")) != NULL)
+ ap_table_set(resp_hdrs, "Location", proxy_location_reverse_map(r, urlstr));
+ if ((urlstr = ap_table_get(resp_hdrs, "URI")) != NULL)
+ ap_table_set(resp_hdrs, "URI", proxy_location_reverse_map(r, urlstr));
+ if ((urlstr = ap_table_get(resp_hdrs, "Content-Location")) != NULL)
+ ap_table_set(resp_hdrs, "Content-Location", proxy_location_reverse_map(r, urlstr));
/* check if NoCache directive on this host */
for (i = 0; i < conf->nocaches->nelts; i++) {
@@ -476,49 +500,45 @@
nocache = 1;
}
+ /* update the cache file, possibly even fulfilling the request if
+ * it turns out a conditional allowed us to serve the object from the
+ * cache...
+ */
i = ap_proxy_cache_update(c, resp_hdrs, !backasswards, nocache);
if (i != DECLINED) {
ap_bclose(f);
return i;
}
- ap_hard_timeout("proxy receive", r);
-
-/* write status line */
- if (!r->assbackwards)
- ap_rvputs(r, "HTTP/1.0 ", r->status_line, CRLF, NULL);
- if (c != NULL && c->fp != NULL &&
- ap_bvputs(c->fp, "HTTP/1.0 ", r->status_line, CRLF, NULL) == -1) {
- ap_log_rerror(APLOG_MARK, APLOG_ERR, c->req,
- "proxy: error writing status line to %s", c->tempfile);
- c = ap_proxy_cache_error(c);
- }
-
-/* send headers */
- tdo.req = r;
- tdo.cache = c;
- ap_table_do(ap_proxy_send_hdr_line, &tdo, resp_hdrs, NULL);
+ /* write status line and headers to the cache file */
+ ap_proxy_write_headers(c, ap_pstrcat(p, "HTTP/1.1 ", r->status_line, NULL), resp_hdrs);
- if (!r->assbackwards)
- ap_rputs(CRLF, r);
- if (c != NULL && c->fp != NULL && ap_bputs(CRLF, c->fp) == -1) {
- ap_log_rerror(APLOG_MARK, APLOG_ERR, c->req,
- "proxy: error writing CRLF to %s", c->tempfile);
- c = ap_proxy_cache_error(c);
- }
+ /* Setup the headers for our client from upstreams response-headers */
+ ap_overlap_tables(r->headers_out, resp_hdrs, AP_OVERLAP_TABLES_SET);
+ /* Add X-Cache header */
+ ap_table_setn(r->headers_out, "X-Cache",
+ ap_pstrcat(r->pool, "MISS from ",
+ ap_get_server_name(r), NULL));
+ /* The Content-Type of this response is the upstream one. */
+ r->content_type = ap_table_get (r->headers_out, "Content-Type");
+ Explain1("Content-Type: %s", r->content_type);
+ /* finally output the headers to the client */
+ ap_send_http_header(r);
- ap_bsetopt(r->connection->client, BO_BYTECT, &zero);
- r->sent_bodyct = 1;
-/* Is it an HTTP/0.9 respose? If so, send the extra data */
+ /* Is it an HTTP/0.9 respose? If so, send the extra data we read
+ from upstream as the start of the reponse to client */
if (backasswards) {
+ ap_hard_timeout("proxy send assbackward", r);
+
ap_bwrite(r->connection->client, buffer, len);
if (c != NULL && c->fp != NULL && ap_bwrite(c->fp, buffer, len) != len) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, c->req,
"proxy: error writing extra data to %s", c->tempfile);
c = ap_proxy_cache_error(c);
}
- }
ap_kill_timeout(r);
+ }
+
#ifdef CHARSET_EBCDIC
/* What we read/write after the header should not be modified
@@ -531,10 +551,17 @@
/* send body */
/* if header only, then cache will be NULL */
/* HTTP/1.0 tells us to read to EOF, rather than content-length bytes */
+/* XXX CHANGEME: We want to eventually support keepalives, which means
+ * we must read content-length bytes... */
if (!r->header_only) {
/* we need to set this for ap_proxy_send_fb()... */
c->cache_completion = conf->cache.cache_completion;
- ap_proxy_send_fb(f, r, c);
+
+/* XXX CHECKME: c->len should be the expected content length, or -1 if the
+ * content length is not known. We need to make 100% sure c->len is always
+ * set correctly before we get here to correctly do keepalive.
+ */
+ ap_proxy_send_fb(f, r, c, c->len, 0);
}
ap_proxy_cache_tidy(c);
diff -wurdX nodiff.pats apache-1.3.13-dev/src/modules/proxy/proxy_util.c apache-1.3.13-dev+http1.1/src/modules/proxy/proxy_util.c
--- apache-1.3.13-dev/src/modules/proxy/proxy_util.c Fri Jun 2 14:01:59 2000
+++ apache-1.3.13-dev+http1.1/src/modules/proxy/proxy_util.c Sat Aug 12 13:35:00 2000
@@ -489,7 +489,12 @@
return resp_hdrs;
}
-long int ap_proxy_send_fb(BUFF *f, request_rec *r, cache_req *c)
+/* read data from f, write it to:
+ * - c->fp, if it is open
+ * - r->connection->client, if nowrite == 0
+ */
+
+long int ap_proxy_send_fb(BUFF *f, request_rec *r, cache_req *c, off_t len, int nowrite)
{
int ok;
char buf[IOBUFSIZE];
@@ -545,7 +550,12 @@
ap_hard_timeout("proxy recv body from upstream server", r);
/* Read block from server */
+ if (-1 == len) {
n = ap_bread(f, buf, IOBUFSIZE);
+ }
+ else {
+ n = ap_bread(f, buf, MIN(IOBUFSIZE, len - total_bytes_rcvd));
+ }
if (alternate_timeouts)
ap_kill_timeout(r);
@@ -560,10 +570,10 @@
}
break;
}
+ total_bytes_rcvd += n;
if (n == 0)
break; /* EOF */
o = 0;
- total_bytes_rcvd += n;
/* Write to cache first. */
/*@@@ XXX FIXME: Assuming that writing the cache file won't time out?!!? */
@@ -578,7 +588,7 @@
}
/* Write the block to the client, detect aborted transfers */
- while (!con->aborted && n > 0) {
+ while (!nowrite && !con->aborted && n > 0) {
if (alternate_timeouts)
ap_soft_timeout("proxy send body", r);
@@ -612,6 +622,11 @@
n -= w;
o += w;
} /* while client alive and more data to send */
+
+ /* if we've received everything, leave now */
+ if (total_bytes_rcvd == len)
+ break;
+
} /* loop and ap_bread while "ok" */
if (!con->aborted)
@@ -622,28 +637,30 @@
}
/*
- * Sends response line and headers. Uses the client fd and the
- * headers_out array from the passed request_rec to talk to the client
- * and to properly set the headers it sends for things such as logging.
+ * Writes response line and headers to the cache file.
*
- * A timeout should be set before calling this routine.
+ * If respline is NULL, no response line will be written.
*/
-void ap_proxy_send_headers(request_rec *r, const char *respline, table *t)
+void ap_proxy_write_headers(cache_req *c, const char *respline, table *t)
{
- int i;
- BUFF *fp = r->connection->client;
- table_entry *elts = (table_entry *) ap_table_elts(t)->elts;
+ /* write status line */
+ if (respline && c->fp != NULL &&
+ ap_bvputs(c->fp, respline, CRLF, NULL) == -1) {
+ ap_log_rerror(APLOG_MARK, APLOG_ERR, c->req,
+ "proxy: error writing status line to %s", c->tempfile);
+ c = ap_proxy_cache_error(c);
+ return;
+ }
- ap_bvputs(fp, respline, CRLF, NULL);
+ /* write response headers to the cache file */
+ ap_table_do(ap_proxy_send_hdr_line, c, t, NULL);
- for (i = 0; i < ap_table_elts(t)->nelts; ++i) {
- if (elts[i].key != NULL) {
- ap_bvputs(fp, elts[i].key, ": ", elts[i].val, CRLF, NULL);
- ap_table_addn(r->headers_out, elts[i].key, elts[i].val);
- }
+ /* write terminating CRLF */
+ if (c->fp != NULL && ap_bputs(CRLF, c->fp) == -1) {
+ ap_log_rerror(APLOG_MARK, APLOG_ERR, c->req,
+ "proxy: error writing CRLF to %s", c->tempfile);
+ c = ap_proxy_cache_error(c);
}
-
- ap_bputs(CRLF, fp);
}
@@ -653,12 +670,14 @@
* The return returns 1 if the token val is found in the list, or 0
* otherwise.
*/
-int ap_proxy_liststr(const char *list, const char *val)
+int ap_proxy_liststr(const char *list, const char *key, char **val)
{
int len, i;
const char *p;
+ char valbuf[HUGE_STRING_LEN];
+ valbuf[sizeof(valbuf)-1] = 0; /* safety terminating zero */
- len = strlen(val);
+ len = strlen(key);
while (list != NULL) {
p = strchr(list, ',');
@@ -673,8 +692,22 @@
while (i > 0 && ap_isspace(list[i - 1]))
i--;
- if (i == len && strncasecmp(list, val, len) == 0)
+ if (i == len && strncasecmp(list, key, len) == 0) {
+ if (val) {
+ p = strchr(list, ',');
+ while (ap_isspace(*list)) {
+ list++;
+ }
+ if ('=' == list[0])
+ list++;
+ while (ap_isspace(*list)) {
+ list++;
+ }
+ strncpy(valbuf, list, MIN(p-list, sizeof(valbuf)-1));
+ *val = valbuf;
+ }
return 1;
+ }
list = p;
}
return 0;
@@ -784,14 +817,14 @@
#endif /* CASE_BLIND_FILESYSTEM */
/*
- * Converts 8 hex digits to a time integer
+ * Converts 16 hex digits to a time integer
*/
int ap_proxy_hex2sec(const char *x)
{
int i, ch;
unsigned int j;
- for (i = 0, j = 0; i < 8; i++) {
+ for (i = 0, j = 0; i < 16; i++) {
ch = x[i];
j <<= 4;
if (ap_isdigit(ch))
@@ -801,21 +834,27 @@
else
j |= ch - ('a' - 10);
}
- if (j == 0xffffffff)
- return -1; /* so that it works with 8-byte ints */
- else
+/* no longer necessary, as the source hex is 8-byte int */
+/* if (j == 0xffffffff)*/
+/* return -1;*/ /* so that it works with 8-byte ints */
+/* else */
return j;
}
/*
- * Converts a time integer to 8 hex digits
+ * Converts a time integer to 16 hex digits
*/
void ap_proxy_sec2hex(int t, char *y)
{
int i, ch;
unsigned int j = t;
- for (i = 7; i >= 0; i--) {
+ if (-1 == t) {
+ strcpy(y, "FFFFFFFFFFFFFFFF");
+ return;
+ }
+
+ for (i = 15; i >= 0; i--) {
ch = j & 0xF;
j >>= 4;
if (ch >= 10)
@@ -823,7 +862,7 @@
else
y[i] = ch + '0';
}
- y[8] = '\0';
+ y[16] = '\0';
}
@@ -834,7 +873,12 @@
ap_pclosef(c->req->pool, ap_bfileno(c->fp, B_WR));
c->fp = NULL;
}
- if (c->tempfile) unlink(c->tempfile);
+ if (c->origfp != NULL) {
+ ap_pclosef(c->req->pool, ap_bfileno(c->origfp, B_WR));
+ c->origfp = NULL;
+ }
+ if (c->tempfile)
+ unlink(c->tempfile);
}
return NULL;
}
@@ -1259,22 +1303,22 @@
return i;
}
-/* This function is called by ap_table_do() for all header lines */
-/* (from proxy_http.c and proxy_ftp.c) */
-/* It is passed a table_do_args struct pointer and a MIME field and value pair */
+/* This function is called by ap_table_do() for all header lines
+ * (from proxy_http.c and proxy_ftp.c)
+ * It is passed a cache_req struct pointer and a MIME field and value pair
+ */
int ap_proxy_send_hdr_line(void *p, const char *key, const char *value)
{
- struct tbl_do_args *parm = (struct tbl_do_args *)p;
+ cache_req *c = (cache_req *)p;
if (key == NULL || value == NULL || value[0] == '\0')
return 1;
- if (!parm->req->assbackwards)
- ap_rvputs(parm->req, key, ": ", value, CRLF, NULL);
- if (parm->cache != NULL && parm->cache->fp != NULL &&
- ap_bvputs(parm->cache->fp, key, ": ", value, CRLF, NULL) == -1) {
- ap_log_rerror(APLOG_MARK, APLOG_ERR, parm->cache->req,
- "proxy: error writing header to %s", parm->cache->tempfile);
- parm->cache = ap_proxy_cache_error(parm->cache);
+ if (c->fp != NULL &&
+ ap_bvputs(c->fp, key, ": ", value, CRLF, NULL) == -1) {
+ ap_log_rerror(APLOG_MARK, APLOG_ERR, c->req,
+ "proxy: error writing header to %s", c->tempfile);
+ c = ap_proxy_cache_error(c);
+ return 0; /* no need to continue, it failed already */
}
return 1; /* tell ap_table_do() to continue calling us for more headers */
}
@@ -1288,6 +1332,93 @@
return len;
}
+/* do a HTTP/1.1 age calculation */
+time_t ap_proxy_current_age(cache_req *c, const time_t age_value)
+{
+ time_t apparent_age, corrected_received_age, response_delay, corrected_initial_age, resident_time, current_age;
+
+ /* Perform an HTTP/1.1 age calculation. (RFC2616 13.2.3) */
+
+ apparent_age = MAX(0, c->resp_time - c->date);
+ corrected_received_age = MAX(apparent_age, age_value);
+ response_delay = c->resp_time - c->req_time;
+ corrected_initial_age = corrected_received_age + response_delay;
+ resident_time = time(NULL) - c->resp_time;
+ current_age = corrected_initial_age + resident_time;
+
+ return (current_age);
+}
+
+/* open a cache file and return a pointer to a BUFF */
+BUFF *ap_proxy_open_cachefile(request_rec *r, char *filename)
+{
+ BUFF *cachefp = NULL;
+ int cfd;
+
+ if (filename != NULL) {
+ cfd = open(filename, O_RDWR | O_BINARY);
+ if (cfd != -1) {
+ ap_note_cleanups_for_fd(r->pool, cfd);
+ cachefp = ap_bcreate(r->pool, B_RD | B_WR);
+ ap_bpushfd(cachefp, cfd, cfd);
+ }
+ else if (errno != ENOENT)
+ ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
+ "proxy: error opening cache file %s",
+ filename);
+#ifdef EXPLAIN
+ else
+ Explain1("File %s not found", filename);
+#endif
+
+ }
+ return cachefp;
+}
+
+/* create a cache file and return a pointer to a BUFF */
+BUFF *ap_proxy_create_cachefile(request_rec *r, char *filename)
+{
+ BUFF *cachefp = NULL;
+ int cfd;
+
+ if (filename != NULL) {
+ cfd = open(filename, O_WRONLY | O_CREAT | O_EXCL | O_BINARY, 0622);
+ if (cfd != -1) {
+ ap_note_cleanups_for_fd(r->pool, cfd);
+ cachefp = ap_bcreate(r->pool, B_WR);
+ ap_bpushfd(cachefp, -1, cfd);
+ }
+ else if (errno != ENOENT)
+ ap_log_rerror(APLOG_MARK, APLOG_ERR, r,
+ "proxy: error creating cache file %s",
+ filename);
+ }
+ return cachefp;
+}
+
+/* Clear all connection-based headers from headers table */
+void ap_proxy_clear_connection(pool *p, table *headers)
+{
+ const char *name;
+ char *next = ap_pstrdup(p, ap_table_get(headers, "Connection"));
+
+ ap_table_unset(headers, "Proxy-Connection");
+ if (!next)
+ return;
+
+ while (*next) {
+ name = next;
+ while (*next && !ap_isspace(*next) && (*next != ','))
+ ++next;
+ while (*next && (ap_isspace(*next) || (*next == ','))) {
+ *next = '\0';
+ ++next;
+ }
+ ap_table_unset(headers, name);
+ }
+ ap_table_unset(headers, "Connection");
+}
+
#if defined WIN32
static DWORD tls_index;
--------------E0896E3F99CC46BA583363B1--
Re: [PATCH] mod_proxy talking HTTP/1.1 to clients
Posted by Graham Leggett <mi...@sharp.fm>.
Chuck Murcko wrote:
> The first part of the porting is done (builds/runs under 2.0, but is
> butt ugly). I have some cleanups from Sam Magnuson to commit when I am
> back from vacation, then all that's left is HTTP/1.1. That was supposed
> to be the easiest to do, but there were complaints about the table
> replace functions first time I started to commit it.
Was there a resolution to this? I remember a question being raised about
the table functions, an answer being given explaining why the table
functions were necessary, and then silence...
I am keen to jump into developing mod_proxy again, but I'd rather wait
until a working base is built into 2.0 that we can start on, that's up
to date with the HTTP/1.1 and the Keepalive patches. How far are we away
from this?
> Also, modproxy-dev list is set up and I plan to start posting this stuff
> there, as there is a smidgeon more interest in this work than there used
> to be. You might have received some mail about this. 8^)
Will look out for it :)
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: [PATCH] mod_proxy talking HTTP/1.1 to clients
Posted by Chuck Murcko <ch...@topsail.org>.
Graham Leggett wrote:
>
> Eli Marmor wrote:
>
> > I think that it is very important that somebody (Christian and
> > Graham) finally spend some effort in this module. It looked like
> > an "orphan" recently, and some people even suggested that because
> > nobody takes it under his/her responsibility, to axe out the
> > forward proxy stuff, and to leave only the reverse-proxy. There
> > are too many users who need this module and can't migrate to squid,
> > and with the new features (e.g. filtering) there will be so many
> > possibilities, so it is a too good thing to give away.
>
> I've been taking a bit of a break for a few months, but I'm certainly
> still around, so mod_proxy still has people who care about it :)
>
> What's happened with the development of mod_proxy is that it seems to
> have outrun itself - people are planning things and designing things
> without taking it to the next logical step - putting the mod_proxy (with
> HTTP/1.1 and Keepalive) in Apache v2.0 as it stands now.
>
> A lot of discussion has gone into the caching in mod_proxy - however I
> believe that caching is a general issue that is a project within itself.
> I'd like to see a genaral shared memory caching mechanism that works
> Apache wide, to replace mod_mmap_static with a layer that can cache
> anything in shared memory, not just files, using the already exisiting
> Cache-Control header mechanisms in RFC2616. But - this effort should not
> get in the way of moving the existing proxy code to Apache v2.0.
>
> So, the next step is porting mod_proxy to v2.0. I know some attempts
> were made to do so in the past, but I would like to see happen is that
> the existing mod_proxy should be ported lock stock and barrel to v2.0 -
> without the distractions of trying to "improve" it along the way. If we
> keep doing that, mod_proxy in v2.0 will never happen.
>
> Does a "how to port to v2.0" HOWTO exist anywhere? Specifically focused
> on how to incorporate a new module into the ./configure build system?
>
The first part of the porting is done (builds/runs under 2.0, but is
butt ugly). I have some cleanups from Sam Magnuson to commit when I am
back from vacation, then all that's left is HTTP/1.1. That was supposed
to be the easiest to do, but there were complaints about the table
replace functions first time I started to commit it.
Also, modproxy-dev list is set up and I plan to start posting this stuff
there, as there is a smidgeon more interest in this work than there used
to be. You might have received some mail about this. 8^)
--
Chuck
Chuck Murcko
Topsail Group
chuck@topsail.org
Re: [PATCH] mod_proxy talking HTTP/1.1 to clients
Posted by Graham Leggett <mi...@sharp.fm>.
Eli Marmor wrote:
> I think that it is very important that somebody (Christian and
> Graham) finally spend some effort in this module. It looked like
> an "orphan" recently, and some people even suggested that because
> nobody takes it under his/her responsibility, to axe out the
> forward proxy stuff, and to leave only the reverse-proxy. There
> are too many users who need this module and can't migrate to squid,
> and with the new features (e.g. filtering) there will be so many
> possibilities, so it is a too good thing to give away.
I've been taking a bit of a break for a few months, but I'm certainly
still around, so mod_proxy still has people who care about it :)
What's happened with the development of mod_proxy is that it seems to
have outrun itself - people are planning things and designing things
without taking it to the next logical step - putting the mod_proxy (with
HTTP/1.1 and Keepalive) in Apache v2.0 as it stands now.
A lot of discussion has gone into the caching in mod_proxy - however I
believe that caching is a general issue that is a project within itself.
I'd like to see a genaral shared memory caching mechanism that works
Apache wide, to replace mod_mmap_static with a layer that can cache
anything in shared memory, not just files, using the already exisiting
Cache-Control header mechanisms in RFC2616. But - this effort should not
get in the way of moving the existing proxy code to Apache v2.0.
So, the next step is porting mod_proxy to v2.0. I know some attempts
were made to do so in the past, but I would like to see happen is that
the existing mod_proxy should be ported lock stock and barrel to v2.0 -
without the distractions of trying to "improve" it along the way. If we
keep doing that, mod_proxy in v2.0 will never happen.
Does a "how to port to v2.0" HOWTO exist anywhere? Specifically focused
on how to incorporate a new module into the ./configure build system?
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: [PATCH] mod_proxy talking HTTP/1.1 to clients
Posted by Eli Marmor <ma...@elmar.co.il>.
> Please, give me some feedback. I didn't receive one single answer to
> last weeks email about graceful degradation of service instead of
> locking out clients. Isn't anybody interested? I half expected Dean
> to suggest the other way of implementing busy_servers().
I think that it is very important that somebody (Christian and
Graham) finally spend some effort in this module. It looked like
an "orphan" recently, and some people even suggested that because
nobody takes it under his/her responsibility, to axe out the
forward proxy stuff, and to leave only the reverse-proxy. There
are too many users who need this module and can't migrate to squid,
and with the new features (e.g. filtering) there will be so many
possibilities, so it is a too good thing to give away.
Anyway, I think that the right direction to go is to focus the
effort in the tree of 2.0, and not 1.3.13. I started to play with
the proxy of 2.0 a little, and even started to patch it, but my
patches are completely out of synch when taking into account the
huge progress that Christian and Graham achieved.
I suggest to wait a few days, till the big patches of the filtering
will be checked in and stabilize a little, and then to build a new
module, based on the tree of 2.0. I can't vote, but I pray for you
that the members here will love this stuff, and hopefully, it will
go into the 2.0a6, together with the filtering code. The un-
imaginable possibilities of this combination are starting to excite
me!
--
Eli Marmor
Re: [PATCH] mod_proxy talking HTTP/1.1 to clients
Posted by Graham Leggett <mi...@sharp.fm>.
Christian von Roques wrote:
> The patch below mostly consists of a slightly cleaned up version of
> Graham Leggett's http/1.1 patch plus my changes to the mod_proxy <->
> client communication. The patch is against Apache-1.3.13-dev from CVS
> as of today. It works for me, but I haven't really tested it in a
> caching configuration.
I just built and tested the patch against apache_1.3.13-dev.
I tested the server with the caching switched on, looking specifically
for segfault errors, and stuck connections or broken data. I used the
Apache documentation as a test case - first using simple browsing using
Netscape. Following this I tried doing repeated websucks using wget as a
mirroring tool, looking again for segfaults or other errors. So far
everything has worked correctly.
I then tested the server with enormous files (the test case a CDROM
image weighing in at about 640MB). The first test downloaded using
Netscape came in 6kb short, with the missing data at around the 390MB
mark - but I couldn't then duplicate this. I tried to download the same
huge file using wget, and then using Netscape a second time, in both
subsequent cases the download came through without a problem. So far it
looks like Netscape may have been the problem, as the proxy server
reported the correct number of bytes transferred. I'd appreciate it if
some other people can test this - I only have access to my laptop right
now, which isn't much of a test environment in terms of performance.
I'd feel better if someone could test this patch against a real server
with some traffic on it - I don't have access to such a system at the moment...
Regards,
Graham
--