You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by rb...@covalent.net on 2001/01/17 15:36:43 UTC

[PATCH] bucket problems.

This patch addresses Dean's problems with the bucket code.  This basically
introduces a simple buffer into the brigade, which can be used to coalesce
data before it is even put into a bucket.  This also adds a couple of
functions to just append data to the end of the brigade, using this
buffer.  There are going to be more of these functions, but this is what
was required to make mod_autoindex work.  That is the other half of this
patch, I have modified mod_autoindex to use those functions instead of
ap_r* to actually write the data for the request.

It is also important to realize that this really doesn't do anything for
filter writers.  These functions are really only useful for generating
data, because they always append the data that is written.  The only
solution I have for this is to create a second brigade, and merge the two
brigades.

Finally, module writers can use both ap_brigade_* functions and
ap_bucket_create_* functions at the same time, but they must be
diligent.  Before using an ap_bucket_create_* function, the module writer
will need to call ap_brigade_standardize.  This will take any data that is
in the buffer and put it at the end of the brigade.  I have not yet found
a way around this, other than to add another couple functions that do all
of this for you, which I dislike.  It is still important to be able to
create a bucket without a brigade, so it won't work to put the logic in
ap_bucket_create_*.

Please review ASAP, I would like to commit VERY quickly.

Ryan

Index: include/util_script.h
===================================================================
RCS file: /home/cvs/httpd-2.0/include/util_script.h,v
retrieving revision 1.13
diff -u -d -b -w -u -r1.13 util_script.h
--- include/util_script.h	2000/10/16 06:04:35	1.13
+++ include/util_script.h	2001/01/17 14:15:20
@@ -166,10 +166,10 @@
  * string sent to the client may be in bytes, kb, or MB, depending on the size
  * of the file.
  * @param size The size of the file
- * @param r The currnt request
- * @deffunc void ap_send_size(apr_ssize_t size, request_rec *r)
+ * @param b The current bucket brigade
+ * @deffunc void ap_send_size(apr_ssize_t size, ap_bucket_brigade *b)
  */
-AP_DECLARE(void) ap_send_size(apr_ssize_t size, request_rec *r);
+AP_DECLARE(void) ap_send_size(apr_ssize_t size, ap_bucket_brigade *b);
 
 #ifdef __cplusplus
 }
Index: modules/generators/mod_autoindex.c
===================================================================
RCS file: /home/cvs/httpd-2.0/modules/generators/mod_autoindex.c,v
retrieving revision 1.51
diff -u -d -b -w -u -r1.51 mod_autoindex.c
--- modules/generators/mod_autoindex.c	2001/01/08 23:54:52	1.51
+++ modules/generators/mod_autoindex.c	2001/01/17 14:15:21
@@ -202,9 +202,9 @@
  * We include the DOCTYPE because we may be using features therefrom (i.e.,
  * HEIGHT and WIDTH attributes on the icons if we're FancyIndexing).
  */
-static void emit_preamble(request_rec *r, char *title)
+static void emit_preamble(ap_bucket_brigade *b, char *title)
 {
-    ap_rvputs(r, DOCTYPE_HTML_3_2,
+    ap_brigade_putstrs(b, DOCTYPE_HTML_3_2,
 	      "<HTML>\n <HEAD>\n  <TITLE>Index of ", title,
 	      "</TITLE>\n </HEAD>\n <BODY>\n", NULL);
 }
@@ -908,14 +908,14 @@
 /*
  * emit a plain text file
  */
-static void do_emit_plain(request_rec *r, apr_file_t *f)
+static void do_emit_plain(ap_bucket_brigade *e, apr_file_t *f)
 {
     char buf[IOBUFSIZE + 1];
     int i, c, ch;
     apr_size_t n;
     apr_status_t stat;
 
-    ap_rputs("<PRE>\n", r);
+    ap_brigade_puts(e, "<PRE>\n");
     while (!apr_eof(f)) {
 	do {
             n = sizeof(char) * IOBUFSIZE;
@@ -935,20 +935,20 @@
 	    }
 	    ch = buf[i];
 	    buf[i] = '\0';
-	    ap_rputs(&buf[c], r);
+	    ap_brigade_puts(e, &buf[c]);
 	    if (ch == '<') {
-		ap_rputs("&lt;", r);
+		ap_brigade_puts(e, "&lt;");
 	    }
 	    else if (ch == '>') {
-		ap_rputs("&gt;", r);
+		ap_brigade_puts(e, "&gt;");
 	    }
 	    else if (ch == '&') {
-		ap_rputs("&amp;", r);
+		ap_brigade_puts(e, "&amp;");
 	    }
 	    c = i + 1;
 	}
     }
-    ap_rputs("</PRE>\n", r);
+    ap_brigade_puts(e, "</PRE>\n");
 }
 
 /*
@@ -960,8 +960,8 @@
  * instead of a text document, meaning nothing will be displayed, but
  * oh well.
  */
-static void emit_head(request_rec *r, char *header_fname, int suppress_amble,
-		      char *title)
+static void emit_head(request_rec *r, ap_bucket_brigade *e, char *header_fname,
+                      int suppress_amble, char *title)
 {
     apr_file_t *f = NULL;
     request_rec *rr = NULL;
@@ -991,7 +991,7 @@
 		emit_H1 = 0;
 
 		if (! suppress_amble) {
-		    emit_preamble(r, title);
+		    emit_preamble(e, title);
 		}
 		/*
 		 * If there's a problem running the subrequest, display the
@@ -1013,9 +1013,9 @@
 		 */
 		if (apr_open(&f, rr->filename, APR_READ,
                             APR_OS_DEFAULT, r->pool) == APR_SUCCESS) {
-		    emit_preamble(r, title);
+		    emit_preamble(e, title);
 		    emit_amble = 0;
-		    do_emit_plain(r, f);
+		    do_emit_plain(e, f);
 		    apr_close(f);
 		    emit_H1 = 0;
 		}
@@ -1024,10 +1024,10 @@
     }
 
     if (emit_amble) {
-	emit_preamble(r, title);
+	emit_preamble(e, title);
     }
     if (emit_H1) {
-	ap_rvputs(r, "<H1>Index of ", title, "</H1>\n", NULL);
+	ap_brigade_putstrs(e, "<H1>Index of ", title, "</H1>\n", NULL);
     }
     if (rr != NULL) {
 	ap_destroy_sub_req(rr);
@@ -1044,7 +1044,8 @@
  * instead of a text document, meaning nothing will be displayed, but
  * oh well.
  */
-static void emit_tail(request_rec *r, char *readme_fname, int suppress_amble)
+static void emit_tail(request_rec *r, ap_bucket_brigade *e, 
+                      char *readme_fname, int suppress_amble)
 {
     apr_file_t *f = NULL;
     request_rec *rr = NULL;
@@ -1081,7 +1082,7 @@
 		 */
 		if (apr_open(&f, rr->filename, APR_READ,
                             APR_OS_DEFAULT, r->pool) == APR_SUCCESS) {
-		    do_emit_plain(r, f);
+		    do_emit_plain(e, f);
 		    apr_close(f);
 		    suppress_sig = 1;
 		}
@@ -1090,10 +1091,10 @@
     }
     
     if (!suppress_sig) {
-	ap_rputs(ap_psignature("", r), r);
+	ap_brigade_puts(e, ap_psignature("", r));
     }
     if (!suppress_post) {
-	ap_rputs("</BODY></HTML>\n", r);
+	ap_brigade_puts(e, "</BODY></HTML>\n");
     }
     if (rr != NULL) {
 	ap_destroy_sub_req(rr);
@@ -1274,8 +1275,8 @@
  * current request, the link changes its meaning to reverse the order when
  * selected again.  Non-active fields always start in ascending order.
  */
-static void emit_link(request_rec *r, char *anchor, char fname, char curkey,
-                      char curdirection, int nosort)
+static void emit_link(request_rec *r, ap_bucket_brigade *e, char *anchor, 
+                      char fname, char curkey, char curdirection, int nosort)
 {
     char qvalue[5];
     int reverse;
@@ -1287,16 +1288,17 @@
 	qvalue[4] = '\0';
 	reverse = ((curkey == fname) && (curdirection == D_ASCENDING));
 	qvalue[3] = reverse ? D_DESCENDING : D_ASCENDING;
-	ap_rvputs(r, "<A HREF=\"", qvalue, "\">", anchor, "</A>", NULL);
+	ap_brigade_putstrs(e, "<A HREF=\"", qvalue, "\">", anchor, "</A>", NULL);
     }
     else {
-        ap_rputs(anchor, r);
+        ap_brigade_puts(e, anchor);
     }
 }
 
 static void output_directories(struct ent **ar, int n,
 			       autoindex_config_rec *d, request_rec *r,
-			       int autoindex_opts, char keyid, char direction)
+			       int autoindex_opts, char keyid, char direction,
+                               ap_bucket_brigade *e)
 {
     int x;
     apr_size_t rv;
@@ -1328,44 +1330,44 @@
     pad_scratch[name_width] = '\0';
 
     if (autoindex_opts & FANCY_INDEXING) {
-	ap_rputs("<PRE>", r);
+	ap_brigade_puts(e, "<PRE>");
 	if ((tp = find_default_icon(d, "^^BLANKICON^^"))) {
-	    ap_rvputs(r, "<IMG SRC=\"", ap_escape_html(scratch, tp),
+	    ap_brigade_putstrs(e, "<IMG SRC=\"", ap_escape_html(scratch, tp),
 		   "\" ALT=\"     \"", NULL);
 	    if (d->icon_width && d->icon_height) {
-		ap_rprintf
+		ap_brigade_printf
 		    (
-			r,
+			e,
 			" HEIGHT=\"%d\" WIDTH=\"%d\"",
 			d->icon_height,
 			d->icon_width
 		    );
 	    }
-	    ap_rputs("> ", r);
+	    ap_brigade_puts(e, "> ");
 	}
-        emit_link(r, "Name", K_NAME, keyid, direction, static_columns);
-	ap_rputs(pad_scratch + 4, r);
+        emit_link(r, e, "Name", K_NAME, keyid, direction, static_columns);
+	ap_brigade_puts(e, pad_scratch + 4);
 	/*
 	 * Emit the guaranteed-at-least-one-space-between-columns byte.
 	 */
-	ap_rputs(" ", r);
+	ap_brigade_puts(e, " ");
 	if (!(autoindex_opts & SUPPRESS_LAST_MOD)) {
-            emit_link(r, "Last modified", K_LAST_MOD, keyid, direction,
+            emit_link(r, e, "Last modified", K_LAST_MOD, keyid, direction,
                       static_columns);
-	    ap_rputs("       ", r);
+	    ap_brigade_puts(e, "       ");
 	}
 	if (!(autoindex_opts & SUPPRESS_SIZE)) {
-            emit_link(r, "Size", K_SIZE, keyid, direction, static_columns);
-	    ap_rputs("  ", r);
+            emit_link(r, e, "Size", K_SIZE, keyid, direction, static_columns);
+	    ap_brigade_puts(e, "  ");
 	}
 	if (!(autoindex_opts & SUPPRESS_DESC)) {
-            emit_link(r, "Description", K_DESC, keyid, direction,
+            emit_link(r, e, "Description", K_DESC, keyid, direction,
                       static_columns);
 	}
-	ap_rputs("\n<HR>\n", r);
+	ap_brigade_puts(e, "\n<HR>\n");
     }
     else {
-	ap_rputs("<UL>", r);
+	ap_brigade_puts(e, "<UL>");
     }
 
     for (x = 0; x < n; x++) {
@@ -1391,23 +1393,23 @@
 
 	if (autoindex_opts & FANCY_INDEXING) {
 	    if (autoindex_opts & ICONS_ARE_LINKS) {
-		ap_rvputs(r, "<A HREF=\"", anchor, "\">", NULL);
+		ap_brigade_putstrs(e, "<A HREF=\"", anchor, "\">", NULL);
 	    }
 	    if ((ar[x]->icon) || d->default_icon) {
-		ap_rvputs(r, "<IMG SRC=\"",
+		ap_brigade_putstrs(e, "<IMG SRC=\"",
 			  ap_escape_html(scratch,
 					 ar[x]->icon ? ar[x]->icon
 					             : d->default_icon),
 			  "\" ALT=\"[", (ar[x]->alt ? ar[x]->alt : "   "),
 			  "]\"", NULL);
 		if (d->icon_width && d->icon_height) {
-		    ap_rprintf(r, " HEIGHT=\"%d\" WIDTH=\"%d\"",
+		    ap_brigade_printf(e, " HEIGHT=\"%d\" WIDTH=\"%d\"",
 			       d->icon_height, d->icon_width);
 		}
-		ap_rputs(">", r);
+		ap_brigade_puts(e, ">");
 	    }
 	    if (autoindex_opts & ICONS_ARE_LINKS) {
-		ap_rputs("</A>", r);
+		ap_brigade_puts(e, "</A>");
 	    }
 
 	    nwidth = strlen(t2);
@@ -1420,13 +1422,13 @@
 	      t2 = name_scratch;
 	      nwidth = name_width;
 	    }
-	    ap_rvputs(r, " <A HREF=\"", anchor, "\">",
+	    ap_brigade_putstrs(e, " <A HREF=\"", anchor, "\">",
 	      ap_escape_html(scratch, t2), "</A>", pad_scratch + nwidth,
 	      NULL);
 	    /*
 	     * The blank before the storm.. er, before the next field.
 	     */
-	    ap_rputs(" ", r);
+	    ap_brigade_puts(e, " ");
 	    if (!(autoindex_opts & SUPPRESS_LAST_MOD)) {
 		if (ar[x]->lm != -1) {
 		    char time_str[MAX_STRING_LEN];
@@ -1434,35 +1436,35 @@
                     apr_explode_localtime(&ts, ar[x]->lm);
 		    apr_strftime(time_str, &rv, MAX_STRING_LEN, 
                                 "%d-%b-%Y %H:%M  ", &ts);
-		    ap_rputs(time_str, r);
+		    ap_brigade_puts(e, time_str);
 		}
 		else {
 		    /*Length="22-Feb-1998 23:42  " (see 4 lines above) */
-		    ap_rputs("                   ", r);
+		    ap_brigade_puts(e, "                   ");
 		}
 	    }
 	    if (!(autoindex_opts & SUPPRESS_SIZE)) {
-		ap_send_size(ar[x]->size, r);
-		ap_rputs("  ", r);
+		ap_send_size(ar[x]->size, e);
+		ap_brigade_puts(e, "  ");
 	    }
 	    if (!(autoindex_opts & SUPPRESS_DESC)) {
 		if (ar[x]->desc) {
-		    ap_rputs(terminate_description(d, ar[x]->desc,
-						   autoindex_opts), r);
+		    ap_brigade_puts(e, terminate_description(d, ar[x]->desc,
+						   autoindex_opts));
 		}
 	    }
 	}
 	else {
-	    ap_rvputs(r, "<LI><A HREF=\"", anchor, "\"> ", t2,
+	    ap_brigade_putstrs(e, "<LI><A HREF=\"", anchor, "\"> ", t2,
 		      "</A>", NULL);
 	}
-	ap_rputc('\n', r);
+	ap_brigade_putc(e, '\n');
     }
     if (autoindex_opts & FANCY_INDEXING) {
-	ap_rputs("</PRE>", r);
+	ap_brigade_puts(e, "</PRE>");
     }
     else {
-	ap_rputs("</UL>", r);
+	ap_brigade_puts(e, "</UL>");
     }
 }
 
@@ -1534,7 +1536,7 @@
 }
 
 
-static int index_directory(request_rec *r,
+static int index_directory(request_rec *r, ap_bucket_brigade *e,
 			   autoindex_config_rec *autoindex_conf)
 {
     char *title_name = ap_escape_html(r->pool, r->uri);
@@ -1581,7 +1583,7 @@
 	*title_endp-- = '\0';
     }
 
-    emit_head(r, find_header(autoindex_conf, r),
+    emit_head(r, e, find_header(autoindex_conf, r),
 	      autoindex_opts & SUPPRESS_PREAMBLE, title_name);
 
     /*
@@ -1646,13 +1648,13 @@
 	      (int (*)(const void *, const void *)) dsortf);
     }
     output_directories(ar, num_ent, autoindex_conf, r, autoindex_opts, keyid,
-		       direction);
+		       direction, e);
     apr_closedir(d);
 
     if (autoindex_opts & FANCY_INDEXING) {
-	ap_rputs("<HR>\n", r);
+	ap_brigade_puts(e, "<HR>\n");
     }
-    emit_tail(r, find_readme(autoindex_conf, r),
+    emit_tail(r, e, find_readme(autoindex_conf, r),
 	      autoindex_opts & SUPPRESS_PREAMBLE);
 
     return 0;
@@ -1664,6 +1666,7 @@
 {
     autoindex_config_rec *d;
     int allow_opts;
+    ap_bucket_brigade *e = ap_brigade_create(r->pool);
 
     if(strcmp(r->handler,DIR_MAGIC_TYPE))
 	return DECLINED;
@@ -1681,6 +1684,7 @@
     /* OK, nothing easy.  Trot out the heavy artillery... */
 
     if (allow_opts & OPT_INDEXES) {
+        apr_status_t status;
 	/* KLUDGE --- make the sub_req lookups happen in the right directory.
 	 * Fixing this in the sub_req_lookup functions themselves is difficult,
 	 * and would probably break virtual includes...
@@ -1689,7 +1693,13 @@
 	if (r->filename[strlen(r->filename) - 1] != '/') {
 	    r->filename = apr_pstrcat(r->pool, r->filename, "/", NULL);
 	}
-	return index_directory(r, d);
+	status = index_directory(r, e, d);
+        if (status == 0) {
+            return ap_pass_brigade(r->output_filters, e);
+        }
+        else {
+            return status;
+        }
     }
     else {
 	ap_log_rerror(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, 0, r,
Index: server/util_filter.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/util_filter.c,v
retrieving revision 1.42
diff -u -d -b -w -u -r1.42 util_filter.c
--- server/util_filter.c	2000/12/29 13:56:30	1.42
+++ server/util_filter.c	2001/01/17 14:15:22
@@ -237,6 +237,7 @@
              */
             next->r->eos_sent = 1;
         }
+        ap_brigade_standardize(bb);
         return next->frec->filter_func.out_func(next, bb);
     }
     return AP_NOBODY_WROTE;
Index: server/util_script.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/util_script.c,v
retrieving revision 1.52
diff -u -d -b -w -u -r1.52 util_script.c
--- server/util_script.c	2000/12/20 16:43:49	1.52
+++ server/util_script.c	2001/01/17 14:15:25
@@ -629,26 +629,26 @@
     return res;
 }
 
-AP_DECLARE(void) ap_send_size(apr_ssize_t size, request_rec *r)
+AP_DECLARE(void) ap_send_size(apr_ssize_t size, ap_bucket_brigade *b)
 {
     /* XXX: this -1 thing is a gross hack */
     if (size == (apr_ssize_t)-1) {
-	ap_rputs("    -", r);
+	ap_brigade_puts(b, "    -");
     }
     else if (!size) {
-	ap_rputs("   0k", r);
+	ap_brigade_puts(b, "   0k");
     }
     else if (size < 1024) {
-	ap_rputs("   1k", r);
+	ap_brigade_puts(b, "   1k");
     }
     else if (size < 1048576) {
-	ap_rprintf(r, "%4" APR_SSIZE_T_FMT "k", (size + 512) / 1024);
+	ap_brigade_printf(b, "%4" APR_SSIZE_T_FMT "k", (size + 512) / 1024);
     }
     else if (size < 103809024) {
-	ap_rprintf(r, "%4.1fM", size / 1048576.0);
+	ap_brigade_printf(b, "%4.1fM", size / 1048576.0);
     }
     else {
-	ap_rprintf(r, "%4" APR_SSIZE_T_FMT "M", (size + 524288) / 1048576);
+	ap_brigade_printf(b, "%4" APR_SSIZE_T_FMT "M", (size + 524288) / 1048576);
     }
 }
 
Index: srclib/apr-util/include/ap_buckets.h
===================================================================
RCS file: /home/cvs/apr-util/include/ap_buckets.h,v
retrieving revision 1.62
diff -u -d -b -w -u -r1.62 ap_buckets.h
--- srclib/apr-util/include/ap_buckets.h	2000/12/31 12:12:56	1.62
+++ srclib/apr-util/include/ap_buckets.h	2001/01/17 14:15:30
@@ -74,6 +74,8 @@
  * @package Bucket Brigades
  */
 
+#define APR_BUCKET_BUFF_SIZE  4096
+
 typedef enum {AP_BLOCK_READ, AP_NONBLOCK_READ} ap_read_type;
 
 /*
@@ -233,6 +235,8 @@
      *  the destroying function is responsible for killing the cleanup.
      */
     apr_pool_t *p;
+    char *start;
+    char *end;
     /** The buckets in the brigade are on this list. */
     /*
      * XXX: the ap_bucket_list structure doesn't actually need a name tag
@@ -620,9 +624,10 @@
 APU_DECLARE(int) ap_brigade_to_iovec(ap_bucket_brigade *b, 
 				    struct iovec *vec, int nvec);
 
+APU_DECLARE(void) ap_brigade_standardize(ap_bucket_brigade *b);
+
 /**
- * This function writes a list of strings into a bucket brigade.  We just 
- * allocate a new heap bucket for each string.
+ * This function writes a list of strings into a bucket brigade.
  * @param b The bucket brigade to add to
  * @param va A list of strings to add
  * @return The number of bytes added to the brigade
@@ -631,8 +636,25 @@
 APU_DECLARE(int) ap_brigade_vputstrs(ap_bucket_brigade *b, va_list va);
 
 /**
+ * This function writes an string into a bucket brigade.
+ * @param b The bucket brigade to add to
+ * @param str The string to add
+ * @return The number of bytes added to the brigade
+ * @deffunc int ap_brigade_puts(ap_bucket_brigade *b, const char *str)
+ */
+APU_DECLARE_NONSTD(int) ap_brigade_puts(ap_bucket_brigade *b, const char *str);
+
+/**
+ * This function writes a character into a bucket brigade.
+ * @param b The bucket brigade to add to
+ * @param c The character to add
+ * @return The number of bytes added to the brigade
+ * @deffunc int ap_brigade_putc(ap_bucket_brigade *b, const char c)
+ */
+APU_DECLARE_NONSTD(int) ap_brigade_putc(ap_bucket_brigade *b, const char c);
+
+/**
  * This function writes an unspecified number of strings into a bucket brigade.
- * We just allocate a new heap bucket for each string.
  * @param b The bucket brigade to add to
  * @param ... The strings to add
  * @return The number of bytes added to the brigade
@@ -641,7 +663,7 @@
 APU_DECLARE_NONSTD(int) ap_brigade_putstrs(ap_bucket_brigade *b, ...);
 
 /**
- * Evaluate a printf and put the resulting string into a bucket at the end 
+ * Evaluate a printf and put the resulting string at the end 
  * of the bucket brigade.
  * @param b The brigade to write to
  * @param fmt The format of the string to write
@@ -652,7 +674,7 @@
 APU_DECLARE_NONSTD(int) ap_brigade_printf(ap_bucket_brigade *b, const char *fmt, ...);
 
 /**
- * Evaluate a printf and put the resulting string into a bucket at the end 
+ * Evaluate a printf and put the resulting string at the end 
  * of the bucket brigade.
  * @param b The brigade to write to
  * @param fmt The format of the string to write
Index: srclib/apr-util/src/buckets/ap_brigade.c
===================================================================
RCS file: /home/cvs/apr-util/src/buckets/ap_brigade.c,v
retrieving revision 1.2
diff -u -d -b -w -u -r1.2 ap_brigade.c
--- srclib/apr-util/src/buckets/ap_brigade.c	2001/01/17 04:29:26	1.2
+++ srclib/apr-util/src/buckets/ap_brigade.c	2001/01/17 14:15:31
@@ -99,6 +99,9 @@
 
     b = apr_palloc(p, sizeof(*b));
     b->p = p;
+
+    b->start = b->end = apr_palloc(p, APR_BUCKET_BUFF_SIZE);
+
     AP_RING_INIT(&b->list, ap_bucket, link);
 
     apr_register_cleanup(b->p, b, ap_brigade_cleanup, ap_brigade_cleanup);
@@ -185,12 +188,23 @@
     return vec - orig;
 }
 
+APU_DECLARE(void) ap_brigade_standardize(ap_bucket_brigade *b)
+{
+    ap_bucket *e;
+    apr_size_t i = b->end - b->start;
+
+    if (i) {
+        e = ap_bucket_create_pool(b->start, i, b->p);
+        AP_BRIGADE_INSERT_TAIL(b, e);
+
+        b->start = b->end = apr_palloc(b->p, APR_BUCKET_BUFF_SIZE);
+    }
+}
+
 APU_DECLARE(int) ap_brigade_vputstrs(ap_bucket_brigade *b, va_list va)
 {
-    ap_bucket *r;
     const char *x;
     int j, k;
-    apr_size_t i;
 
     for (k = 0;;) {
         x = va_arg(va, const char *);
@@ -198,20 +212,44 @@
             break;
         j = strlen(x);
        
-	/* XXX: copy or not? let the caller decide? */
-        r = ap_bucket_create_heap(x, j, 1, &i);
-        if (i != j) {
-            /* Do we need better error reporting?  */
-            return -1;
+        if (j > (APR_BUCKET_BUFF_SIZE - (b->end - b->start))) {
+            ap_brigade_standardize(b);
         }
-        k += i;
+        memcpy(b->end, x, j);
+        b->end += j;
 
-        AP_BRIGADE_INSERT_TAIL(b, r);
+        k += j;
     }
 
     return k;
 }
 
+APU_DECLARE(int) ap_brigade_putc(ap_bucket_brigade *b, const char c)
+{
+    if (1 > (APR_BUCKET_BUFF_SIZE - (b->end - b->start))) {
+        ap_brigade_standardize(b);
+    }
+    memcpy(b->end, &c, 1);
+    b->end++;
+
+    return 1;
+}
+
+APU_DECLARE(int) ap_brigade_puts(ap_bucket_brigade *b, const char *str)
+{
+    apr_size_t j;
+
+    j = strlen(str);
+   
+    if (j > (APR_BUCKET_BUFF_SIZE - (b->end - b->start))) {
+        ap_brigade_standardize(b);
+    }
+    memcpy(b->end, str, j);
+    b->end += j;
+
+    return j;
+}
+
 APU_DECLARE_NONSTD(int) ap_brigade_putstrs(ap_bucket_brigade *b, ...)
 {
     va_list va;
@@ -240,13 +278,8 @@
      * directly into a bucket.  I'm being lazy right now.  RBB
      */
     char buf[4096];
-    ap_bucket *r;
-    int res;
-
-    res = apr_vsnprintf(buf, 4096, fmt, va);
 
-    r = ap_bucket_create_heap(buf, strlen(buf), 1, NULL);
-    AP_BRIGADE_INSERT_TAIL(b, r);
+    apr_vsnprintf(buf, 4096, fmt, va);
 
-    return res;
+    return ap_brigade_puts(b, buf);
 }


_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------


Re: [PATCH] bucket problems.

Posted by rb...@covalent.net.
> Well, it is hard to distinguish what you are intended to be an *example*
> versus the *real* solution.
> 
> "Here's what I'm thinking" seems to imply just that -- where you think the
> code needs to go for the long term.
> 
> The changes to mod_autoindex are badness.

No, one change to mod_autoindex is badness.  We need to actually call
ap_pass_brigade when it makes sense.  The rest of the changes are fine.

> 1) overall performance?
> 2) speeding up ap_r*
> 3) reducing the number of traversals down the chain
> 4) fixing sub-optimal network packet behavior
> 5) too many chunks in the output
> 6) ...
> 
> What is the problem being solved by this patch? "here is another API; do you
> see how modules can use it to do output?"

The specific problem is that ap_r* functions suck right now.  They are
completely horrible because we have to create a single bucket for each
call to ap_r*.  Also, when we have so many small buckets, our
core_output_filter ends up creating a bunch of iovec's that have 16
elements, and are very small.  We need to actually buffer data into a
single buffer, and we should do this without creating the individual
buckets.

> I feel that if we have a better handle on the specific issue, then we can
> concentrate on that one better. While it may have been stated before, and I
> could have missed it, it really isn't clear right now.
> [ Dean had a number of issues with the performance, but I'm not sure which
>   one we're specifically solving ]

My patch solves the problems mentioned above.  I am not currently aware of
any other problems brought up by Dean.

Ryan

_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------


Re: [PATCH] bucket problems.

Posted by Greg Stein <gs...@lyra.org>.
On Wed, Jan 17, 2001 at 01:00:54PM -0800, rbb@covalent.net wrote:
> 
> > >      With your API, some weirdo will create a filesystem with
> > > infinite entries in a directory, 
> > 
> > yeah, like Mandrake 7.2.  watch out for their logrotate - update that
> > sucker if you put on 7.2.  It screws up in /var/log/messages/mail and
> > /var/log/messages/news, and starts creating tons of
> > blah.1.gz.3.gz.1.gz.5.gz.1.gz files.  If you ignore it and go sailing
> > (for example) and don't reboot, the daily cron job eventually takes more
> > than a day...  No, I didn't try mod_autoindex in those directories - ls
> > and rm blah* wouldn't even work.
> 
> Again, please do not take a quick hack that was made to provide a working
> example as a good long-term solution.  mod_autoindex as it was written
> last night would be a poor example of a long-term solution, but it made a
> very easy short-term example module.

Well, it is hard to distinguish what you are intended to be an *example*
versus the *real* solution.

"Here's what I'm thinking" seems to imply just that -- where you think the
code needs to go for the long term.

The changes to mod_autoindex are badness.


I'm unclear on the *specific* problem that is being solved:

1) overall performance?
2) speeding up ap_r*
3) reducing the number of traversals down the chain
4) fixing sub-optimal network packet behavior
5) too many chunks in the output
6) ...

What is the problem being solved by this patch? "here is another API; do you
see how modules can use it to do output?"

I feel that if we have a better handle on the specific issue, then we can
concentrate on that one better. While it may have been stated before, and I
could have missed it, it really isn't clear right now.
[ Dean had a number of issues with the performance, but I'm not sure which
  one we're specifically solving ]

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

Re: [PATCH] bucket problems.

Posted by rb...@covalent.net.
> >      With your API, some weirdo will create a filesystem with
> > infinite entries in a directory, 
> 
> yeah, like Mandrake 7.2.  watch out for their logrotate - update that
> sucker if you put on 7.2.  It screws up in /var/log/messages/mail and
> /var/log/messages/news, and starts creating tons of
> blah.1.gz.3.gz.1.gz.5.gz.1.gz files.  If you ignore it and go sailing
> (for example) and don't reboot, the daily cron job eventually takes more
> than a day...  No, I didn't try mod_autoindex in those directories - ls
> and rm blah* wouldn't even work.

Again, please do not take a quick hack that was made to provide a working
example as a good long-term solution.  mod_autoindex as it was written
last night would be a poor example of a long-term solution, but it made a
very easy short-term example module.

Ryan

_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------


Re: [PATCH] bucket problems.

Posted by Greg Ames <gr...@raleigh.ibm.com>.
Jeff Trawick wrote:
>      With your API, some weirdo will create a filesystem with
> infinite entries in a directory, 

yeah, like Mandrake 7.2.  watch out for their logrotate - update that
sucker if you put on 7.2.  It screws up in /var/log/messages/mail and
/var/log/messages/news, and starts creating tons of
blah.1.gz.3.gz.1.gz.5.gz.1.gz files.  If you ignore it and go sailing
(for example) and don't reboot, the daily cron job eventually takes more
than a day...  No, I didn't try mod_autoindex in those directories - ls
and rm blah* wouldn't even work.

Greg

Re: [PATCH] bucket problems.

Posted by rb...@covalent.net.
> > You are taking a part of the patch that was a quick hack and extrapolating
> > that it is the only way to do this.  I added the pass_brigade where I did
> > because I wanted a working example.  A real example will actually have to
> > make this call whenever it makes sense.
> 
> Do you mean that you would expect a module using this API to regularly
> decide whether or not to call ap_pass_brigade()?  It would be really
> nice to avoid that.  But avoiding it breaks your desire to avoid
> having an API which knows about both filters and buckets.

I was simply saying that the module will be responsible for passing the
brigade to the next filter.

> > Don't change the ap_r* API at all
> > Good performance for ap_r* functions
> > Ability to use both ap_bucket_create and ap_r* functions in the same
> > 	function
> > Ability to use network congestion to stop sending data
> > Don't allow the server to even try to use up all the memory
> > 
> > You can't have all five of these.  Pick a few and design for them.
> 
> As long as the app calls a flush routine (ap_rflush()), all five of
> these were met by Victor's patch if I recall correctly.

No, Victor's patch was just plain wrong.  Instead of associating a buffer
with a bucket brigade, it associated it with a request_rec, which means
that only Apache gets any benefit.

> > It doesn't matter.  The biggest problem with Victor's patch is that it
> > puts the buffer in the request_rec, which means that we aren't helping
> > anybody other than Apache itself.
> 
> As I mentioned before, macros could avoid having the core code
> know about the request_rec.
> 
>   #define ap_rputs(s,r) \
>   ap_brigade_puts(r->field1, r->field2, x, y)

This doesn't look at all like Victor's.  Forget it.  I'm putting a brigade
inside the request_rec, and just using it.  This is absolutely the wrong
approach, because it allows people to forget they are using buckets.  Oh
well, I'm sick and tired of arguing.

> But the key issue is whether some API knows buckets and filters.  To
> meet what I thought was the goal ("Allow apps which use the ap_r* API
> to perform reasonably"), I think we need such an API.

The ap_r* API will never perform reasonably.  It is the wrong
model.  Instead of advertising what you are doing to module writers, you
are forcing them to be one level removed, which is absolutely bogus.  Oh
well.

Ryan

_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------



Re: [PATCH] bucket problems.

Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
From: "Peter J. Cranstone" <cr...@remotecommunications.com>
Sent: Wednesday, January 17, 2001 4:16 PM


> Guys,
> 
> Feel free to flame me or plain ignore me but I have a suggestion. Why not
> leave filtering out of the equation until the beta is stable. Filtering is
> going to cause issues with 1.x modules and will take time to explain to your
> existing base.

Hmmm... as you are our Target Audience [Module Author] we have obviously
failed miserably here.  The point of the Alphas were to draw in the Module
Authors into the equation.  Almost every module author has asked at some
point, "why can't I just...?"  That's what Apache 2.0 is meant to be.

Apache 1.x was composed of handlers.  Nothing more (until authors tried to
hack on ssl, something that wasn't simply another message body.)

Apache 2.0 is composed of filters.  Yes, there are handler 'end-points',
but it's pointless to worry about a beta of 2.0 that does what 1.x accomplished
quite nicely.

What I'm saying is not to insult you, but if our module Authors haven't
'gotten it', then we haven't done our job.  And I'm not insulting Ryan either,
who preaches the gospel to every reporter who will listen (even when they
cross up their facts.)

> What I think you need IMHO is to get 2.0 stable with just one or two
> features. You have enough on the table to do that. Then introduce filtering
> at a later stage. Adding the whole thing now along with all the other
> features is going to result in 1.x users avoiding the upgrade.

Nak.  There are only two goals in 2.0 - introduce native threading, and
build the server in a filtered model.  Those are it.

> Backwards compatibility to older modules is KEY until you have the new
> filtering system stable and OUT performing anything else. Then users will go
> through the pain of adapting their modules for the newer more stable, faster
> Apache 2.0

Agreed we need to find ways to -help- module authors make the leap.  But I
personally hacked the 2.0 mod_isapi.  Compare the sources.  It wasn't that
much of a leap for an 'end-point' handler.  The work and pain is in writing
a truly usable filter.



Re: [PATCH] bucket problems.

Posted by Greg Ames <gr...@raleigh.ibm.com>.
rbb@covalent.net wrote:
> 
> There is one problem with this whole idea.  The filtering is in
> already.  The idea now is to perfect it, so that it works well for
> people.  We will hit a beta soon, because otherwise you are correct,
> people will leave this project.  Be patient, give us another couple of
> weeks to hammer this out finally.
> 
> Ryan
> 
> On Wed, 17 Jan 2001, Jeffrey A. Stuart wrote:
> 
> > You know what, I have to agree with Peter on this.  Get apache 2.0 out. ;)
> > The one thing that will KILL an open source project is "apparent" lack of
> > progress. 

Jeff, I understand where you & Peter are coming from.  I want to get a
beta out there that performs close to 1.3 so badly I can taste it.  

But I'm 100% with Ryan on this - filtering is a keeper.  I was _very_
skeptical about filtering at first, but am now sold.  We are doing
charset translation with it, for example, in a way that lets us do
things like Unicode without impacting people who aren't interested.  

Yes we have some very vocal disagreements going on at the moment.  But I
think that's because we are very passionate about our work, and want to
get the last little bit right.  I truly believe that's where we're at
now with the dynamic html "tinygram" issue.  We need to re-introduce
some positive aspects of the 1.3 design, in a general way, without
taking away from our new baby. 

Stay with us for another week or two.  I think you'll be glad you did!

Regards,
Greg

RE: [PATCH] bucket problems.

Posted by "Peter J. Cranstone" <cr...@remotecommunications.com>.
> The filtering is in already.  The idea now is to perfect it, so that it
works well for
> people.  We will hit a beta soon, because otherwise you are correct,
> people will leave this project. Be patient, give us another couple of
> weeks to hammer this out finally.


Ryan,

We'll be here. Kevin has already coded a version of mod_gzip that uses the
new 2.0 filtering 'scheme' to compress ALL Server output but we are holding
off until you have the filtering code 'stable'. We need you to finalize on
the API calls.

Having just released mod_gzip we know first hand the issues you are dealing
and we constantly faced with the same stability vs. feature issue.

You have two key issues - native threading, and build the server in a
filtered model. Which ONE do you want to make stable first. Once you have
that you then have Apache 2.0 beta released. This gives positive momentum to
the project.

What comes next is feature number two which will need to include outside
module - author support. We can then release mod_gzip for Apache 2.0....
more positive momentum for the Apache Group which continues to build on a
successful Apache 2.0 code base.

Regards,


Peter


-----Original Message-----
From: rbb@covalent.net [mailto:rbb@covalent.net]
Sent: Wednesday, January 17, 2001 5:00 PM
To: new-httpd@apache.org
Subject: RE: [PATCH] bucket problems.



There is one problem with this whole idea.  The filtering is in
already.  The idea now is to perfect it, so that it works well for
people.  We will hit a beta soon, because otherwise you are correct,
people will leave this project.  Be patient, give us another couple of
weeks to hammer this out finally.

Ryan

On Wed, 17 Jan 2001, Jeffrey A. Stuart wrote:

> You know what, I have to agree with Peter on this.  Get apache 2.0 out. ;)
> The one thing that will KILL an open source project is "apparent" lack of
> progress.  Please note I stress the word APPARENT... Perfect example.
> Proftpd 1.2.  It has had no apparent progress since July of last year.  In
> my mind and in a number of other people that I've talked to, Proftpd is
> DEAD... Which is a shame...  it was (maybe even is) a good program but...
>
> And a couple of times a month, people come to me and ask about apache 2.0
> and when will it be out.  They had heard it was coming out (in beta
format)
> Nov of last year, then, end of Dec, then ... Maybe it's time to step back
> for a second and decide what Apache 2.0 should REALLY have in it.  The KDE
> folk had to do that.  When they decided to come out with 2.0, it took them
a
> VERY long time.  Then after about 9 months or so, they finally had to
decide
> to drop some features from 2.0 and release what they had.
>
> Now yes, you can say that there is traffic on the mailing list and if you
> REALLY want to know what's going on, get on the mailing list.  Well, most
of
> the people I deal with are not "techies" and don't have the time OR desire
> to read this mailing list.  That's my job. :)  (And to keep their servers
up
> and running.. :))  So they see no activity while I on the other hand see a
> FLURRY of activity. :)
>
> This of course is just my opinion.
>
> --
> Jeff (FurBall)
> WebOverdrive Newbie Tech Board
> http://www.topniche.com/tech/
> furball@weboverdrive.com
>
> -----Original Message-----
> From: Peter J. Cranstone [mailto:cranstone@remotecommunications.com]
> Sent: Wednesday, January 17, 2001 5:17 PM
> To: new-httpd@apache.org
> Subject: RE: [PATCH] bucket problems.
>
> Guys,
>
> Feel free to flame me or plain ignore me but I have a suggestion. Why not
> leave filtering out of the equation until the beta is stable. Filtering is
> going to cause issues with 1.x modules and will take time to explain to
your
> existing base.
>
> What I think you need IMHO is to get 2.0 stable with just one or two
> features. You have enough on the table to do that. Then introduce
filtering
> at a later stage. Adding the whole thing now along with all the other
> features is going to result in 1.x users avoiding the upgrade.
>
> Backwards compatibility to older modules is KEY until you have the new
> filtering system stable and OUT performing anything else. Then users will
go
> through the pain of adapting their modules for the newer more stable,
faster
> Apache 2.0
>
> This is just a thought. Ignoring me will cost less effort than flaming me
> but you know where I am :)
>
> Regards,
>
>
> Peter
> RCI
>
> -----Original Message-----
> From: trawick@bellsouth.net [mailto:trawick@bellsouth.net]On Behalf Of
> Jeff Trawick
> Sent: Wednesday, January 17, 2001 2:51 PM
> To: new-httpd@apache.org
> Subject: Re: [PATCH] bucket problems.
>
>
> rbb@covalent.net writes:
>
> > > I think the very idea of an API which adds to a brigade is
> > > problematic.  We want an API which handles output, sending it down the
> > > filter chain when necessary (i.e., when our x-kilobyte coalesce buffer
> > > is full).  With your API, some weirdo will create a filesystem with
> > > infinite entries in a directory, and rather than having
mod_autoindex's
> > > processing of this directory limited by network I/O to the client,
> > > mod_autoindex will loop until all storage is exhausted.
> >
> > You are taking a part of the patch that was a quick hack and
extrapolating
> > that it is the only way to do this.  I added the pass_brigade where I
did
> > because I wanted a working example.  A real example will actually have
to
> > make this call whenever it makes sense.
>
> Do you mean that you would expect a module using this API to regularly
> decide whether or not to call ap_pass_brigade()?  It would be really
> nice to avoid that.  But avoiding it breaks your desire to avoid
> having an API which knows about both filters and buckets.
>
> > > I really think we want some changes with the coalesce property but
> > > also the flush-to-the-network-as-we-go property, rather than
> > > continually building up the brigade until the module sees fit to call
> > > ap_pass_brigade().
> >
> > If you can design/implement this, please feel free.  As I see it, we
have
> > multiple people with different priorities.  This is what I have
currently
> > heard as priorities, please add any if I miss some.
>
> As I recall, Victor already started down this road but was shot down.
>
> > Don't change the ap_r* API at all
> > Good performance for ap_r* functions
> > Ability to use both ap_bucket_create and ap_r* functions in the same
> >       function
> > Ability to use network congestion to stop sending data
> > Don't allow the server to even try to use up all the memory
> >
> > You can't have all five of these.  Pick a few and design for them.
>
> As long as the app calls a flush routine (ap_rflush()), all five of
> these were met by Victor's patch if I recall correctly.
>
> > > But to do that the API needs to know about filters too (enough to call
> > > ap_pass_brigade()), so there is another parameter to
> > > ap_brigade_something().
> >
> > No, that is tying the buckets to the filters, which ties buckets to
Apache
> > basically, which is not correct IMHO.
>
> The app has to do extra work if this type of API doesn't handle
> buckets and filters.
>
> > It doesn't matter.  The biggest problem with Victor's patch is that it
> > puts the buffer in the request_rec, which means that we aren't helping
> > anybody other than Apache itself.
>
> As I mentioned before, macros could avoid having the core code
> know about the request_rec.
>
>   #define ap_rputs(s,r) \
>   ap_brigade_puts(r->field1, r->field2, x, y)
>
>   (or some such)
>
> But the key issue is whether some API knows buckets and filters.  To
> meet what I thought was the goal ("Allow apps which use the ap_r* API
> to perform reasonably"), I think we need such an API.
>
> Thanks,
> --
> Jeff Trawick | trawickj@bellsouth.net | PGP public key at web site:
>        http://www.geocities.com/SiliconValley/Park/9289/
>              Born in Roswell... married an alien...
>
>
>


____________________________________________________________________________
___
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
----------------------------------------------------------------------------
---


RE: [PATCH] bucket problems.

Posted by "Jeffrey A. Stuart" <js...@neo.rr.com>.
Let me say this... Me personally, I don't mind waiting... the LAST thing
that I want to see is see apache 2.0 released live and suddenly see some
huge bug occur. That's the QUICKEST way to hurt a project; maybe even
fatally. :(  I do NOT want to see a fiasco with Apache like Microsoft tends
to have when they release new software. :)  So do it right...  A friend of
mine once gave me a little quote about software development.  He said,
"There are only three things involved in a project.  Quality, Money, and
Time.  Pick 2 of the three."  And that's been true ever since... So...
choose Quality and Money. :)

--
Jeff (FurBall)
WebOverdrive Newbie Tech Board
http://www.topniche.com/tech/
furball@weboverdrive.com

-----Original Message-----
From: rbb@covalent.net [mailto:rbb@covalent.net]
Sent: Wednesday, January 17, 2001 7:00 PM
To: new-httpd@apache.org
Subject: RE: [PATCH] bucket problems.


There is one problem with this whole idea.  The filtering is in
already.  The idea now is to perfect it, so that it works well for
people.  We will hit a beta soon, because otherwise you are correct,
people will leave this project.  Be patient, give us another couple of
weeks to hammer this out finally.

Ryan

On Wed, 17 Jan 2001, Jeffrey A. Stuart wrote:

> You know what, I have to agree with Peter on this.  Get apache 2.0 out. ;)
> The one thing that will KILL an open source project is "apparent" lack of
> progress.  Please note I stress the word APPARENT... Perfect example.
> Proftpd 1.2.  It has had no apparent progress since July of last year.  In
> my mind and in a number of other people that I've talked to, Proftpd is
> DEAD... Which is a shame...  it was (maybe even is) a good program but...
>
> And a couple of times a month, people come to me and ask about apache 2.0
> and when will it be out.  They had heard it was coming out (in beta
format)
> Nov of last year, then, end of Dec, then ... Maybe it's time to step back
> for a second and decide what Apache 2.0 should REALLY have in it.  The KDE
> folk had to do that.  When they decided to come out with 2.0, it took them
a
> VERY long time.  Then after about 9 months or so, they finally had to
decide
> to drop some features from 2.0 and release what they had.
>
> Now yes, you can say that there is traffic on the mailing list and if you
> REALLY want to know what's going on, get on the mailing list.  Well, most
of
> the people I deal with are not "techies" and don't have the time OR desire
> to read this mailing list.  That's my job. :)  (And to keep their servers
up
> and running.. :))  So they see no activity while I on the other hand see a
> FLURRY of activity. :)
>
> This of course is just my opinion.
>
> --
> Jeff (FurBall)
> WebOverdrive Newbie Tech Board
> http://www.topniche.com/tech/
> furball@weboverdrive.com
>

____________________________________________________________________________
___
Ryan Bloom                              rbb@apache.org
406 29th St.
San Francisco, CA 94131
----------------------------------------------------------------------------
---



RE: [PATCH] bucket problems.

Posted by rb...@covalent.net.
There is one problem with this whole idea.  The filtering is in
already.  The idea now is to perfect it, so that it works well for
people.  We will hit a beta soon, because otherwise you are correct,
people will leave this project.  Be patient, give us another couple of
weeks to hammer this out finally.

Ryan

On Wed, 17 Jan 2001, Jeffrey A. Stuart wrote:

> You know what, I have to agree with Peter on this.  Get apache 2.0 out. ;)
> The one thing that will KILL an open source project is "apparent" lack of
> progress.  Please note I stress the word APPARENT... Perfect example.
> Proftpd 1.2.  It has had no apparent progress since July of last year.  In
> my mind and in a number of other people that I've talked to, Proftpd is
> DEAD... Which is a shame...  it was (maybe even is) a good program but...
> 
> And a couple of times a month, people come to me and ask about apache 2.0
> and when will it be out.  They had heard it was coming out (in beta format)
> Nov of last year, then, end of Dec, then ... Maybe it's time to step back
> for a second and decide what Apache 2.0 should REALLY have in it.  The KDE
> folk had to do that.  When they decided to come out with 2.0, it took them a
> VERY long time.  Then after about 9 months or so, they finally had to decide
> to drop some features from 2.0 and release what they had.
> 
> Now yes, you can say that there is traffic on the mailing list and if you
> REALLY want to know what's going on, get on the mailing list.  Well, most of
> the people I deal with are not "techies" and don't have the time OR desire
> to read this mailing list.  That's my job. :)  (And to keep their servers up
> and running.. :))  So they see no activity while I on the other hand see a
> FLURRY of activity. :)
> 
> This of course is just my opinion.
> 
> --
> Jeff (FurBall)
> WebOverdrive Newbie Tech Board
> http://www.topniche.com/tech/
> furball@weboverdrive.com
> 
> -----Original Message-----
> From: Peter J. Cranstone [mailto:cranstone@remotecommunications.com]
> Sent: Wednesday, January 17, 2001 5:17 PM
> To: new-httpd@apache.org
> Subject: RE: [PATCH] bucket problems.
> 
> Guys,
> 
> Feel free to flame me or plain ignore me but I have a suggestion. Why not
> leave filtering out of the equation until the beta is stable. Filtering is
> going to cause issues with 1.x modules and will take time to explain to your
> existing base.
> 
> What I think you need IMHO is to get 2.0 stable with just one or two
> features. You have enough on the table to do that. Then introduce filtering
> at a later stage. Adding the whole thing now along with all the other
> features is going to result in 1.x users avoiding the upgrade.
> 
> Backwards compatibility to older modules is KEY until you have the new
> filtering system stable and OUT performing anything else. Then users will go
> through the pain of adapting their modules for the newer more stable, faster
> Apache 2.0
> 
> This is just a thought. Ignoring me will cost less effort than flaming me
> but you know where I am :)
> 
> Regards,
> 
> 
> Peter
> RCI
> 
> -----Original Message-----
> From: trawick@bellsouth.net [mailto:trawick@bellsouth.net]On Behalf Of
> Jeff Trawick
> Sent: Wednesday, January 17, 2001 2:51 PM
> To: new-httpd@apache.org
> Subject: Re: [PATCH] bucket problems.
> 
> 
> rbb@covalent.net writes:
> 
> > > I think the very idea of an API which adds to a brigade is
> > > problematic.  We want an API which handles output, sending it down the
> > > filter chain when necessary (i.e., when our x-kilobyte coalesce buffer
> > > is full).  With your API, some weirdo will create a filesystem with
> > > infinite entries in a directory, and rather than having mod_autoindex's
> > > processing of this directory limited by network I/O to the client,
> > > mod_autoindex will loop until all storage is exhausted.
> >
> > You are taking a part of the patch that was a quick hack and extrapolating
> > that it is the only way to do this.  I added the pass_brigade where I did
> > because I wanted a working example.  A real example will actually have to
> > make this call whenever it makes sense.
> 
> Do you mean that you would expect a module using this API to regularly
> decide whether or not to call ap_pass_brigade()?  It would be really
> nice to avoid that.  But avoiding it breaks your desire to avoid
> having an API which knows about both filters and buckets.
> 
> > > I really think we want some changes with the coalesce property but
> > > also the flush-to-the-network-as-we-go property, rather than
> > > continually building up the brigade until the module sees fit to call
> > > ap_pass_brigade().
> >
> > If you can design/implement this, please feel free.  As I see it, we have
> > multiple people with different priorities.  This is what I have currently
> > heard as priorities, please add any if I miss some.
> 
> As I recall, Victor already started down this road but was shot down.
> 
> > Don't change the ap_r* API at all
> > Good performance for ap_r* functions
> > Ability to use both ap_bucket_create and ap_r* functions in the same
> >       function
> > Ability to use network congestion to stop sending data
> > Don't allow the server to even try to use up all the memory
> >
> > You can't have all five of these.  Pick a few and design for them.
> 
> As long as the app calls a flush routine (ap_rflush()), all five of
> these were met by Victor's patch if I recall correctly.
> 
> > > But to do that the API needs to know about filters too (enough to call
> > > ap_pass_brigade()), so there is another parameter to
> > > ap_brigade_something().
> >
> > No, that is tying the buckets to the filters, which ties buckets to Apache
> > basically, which is not correct IMHO.
> 
> The app has to do extra work if this type of API doesn't handle
> buckets and filters.
> 
> > It doesn't matter.  The biggest problem with Victor's patch is that it
> > puts the buffer in the request_rec, which means that we aren't helping
> > anybody other than Apache itself.
> 
> As I mentioned before, macros could avoid having the core code
> know about the request_rec.
> 
>   #define ap_rputs(s,r) \
>   ap_brigade_puts(r->field1, r->field2, x, y)
> 
>   (or some such)
> 
> But the key issue is whether some API knows buckets and filters.  To
> meet what I thought was the goal ("Allow apps which use the ap_r* API
> to perform reasonably"), I think we need such an API.
> 
> Thanks,
> --
> Jeff Trawick | trawickj@bellsouth.net | PGP public key at web site:
>        http://www.geocities.com/SiliconValley/Park/9289/
>              Born in Roswell... married an alien...
> 
> 
> 


_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------


RE: [PATCH] bucket problems.

Posted by "Jeffrey A. Stuart" <js...@neo.rr.com>.
You know what, I have to agree with Peter on this.  Get apache 2.0 out. ;)
The one thing that will KILL an open source project is "apparent" lack of
progress.  Please note I stress the word APPARENT... Perfect example.
Proftpd 1.2.  It has had no apparent progress since July of last year.  In
my mind and in a number of other people that I've talked to, Proftpd is
DEAD... Which is a shame...  it was (maybe even is) a good program but...

And a couple of times a month, people come to me and ask about apache 2.0
and when will it be out.  They had heard it was coming out (in beta format)
Nov of last year, then, end of Dec, then ... Maybe it's time to step back
for a second and decide what Apache 2.0 should REALLY have in it.  The KDE
folk had to do that.  When they decided to come out with 2.0, it took them a
VERY long time.  Then after about 9 months or so, they finally had to decide
to drop some features from 2.0 and release what they had.

Now yes, you can say that there is traffic on the mailing list and if you
REALLY want to know what's going on, get on the mailing list.  Well, most of
the people I deal with are not "techies" and don't have the time OR desire
to read this mailing list.  That's my job. :)  (And to keep their servers up
and running.. :))  So they see no activity while I on the other hand see a
FLURRY of activity. :)

This of course is just my opinion.

--
Jeff (FurBall)
WebOverdrive Newbie Tech Board
http://www.topniche.com/tech/
furball@weboverdrive.com

-----Original Message-----
From: Peter J. Cranstone [mailto:cranstone@remotecommunications.com]
Sent: Wednesday, January 17, 2001 5:17 PM
To: new-httpd@apache.org
Subject: RE: [PATCH] bucket problems.

Guys,

Feel free to flame me or plain ignore me but I have a suggestion. Why not
leave filtering out of the equation until the beta is stable. Filtering is
going to cause issues with 1.x modules and will take time to explain to your
existing base.

What I think you need IMHO is to get 2.0 stable with just one or two
features. You have enough on the table to do that. Then introduce filtering
at a later stage. Adding the whole thing now along with all the other
features is going to result in 1.x users avoiding the upgrade.

Backwards compatibility to older modules is KEY until you have the new
filtering system stable and OUT performing anything else. Then users will go
through the pain of adapting their modules for the newer more stable, faster
Apache 2.0

This is just a thought. Ignoring me will cost less effort than flaming me
but you know where I am :)

Regards,


Peter
RCI

-----Original Message-----
From: trawick@bellsouth.net [mailto:trawick@bellsouth.net]On Behalf Of
Jeff Trawick
Sent: Wednesday, January 17, 2001 2:51 PM
To: new-httpd@apache.org
Subject: Re: [PATCH] bucket problems.


rbb@covalent.net writes:

> > I think the very idea of an API which adds to a brigade is
> > problematic.  We want an API which handles output, sending it down the
> > filter chain when necessary (i.e., when our x-kilobyte coalesce buffer
> > is full).  With your API, some weirdo will create a filesystem with
> > infinite entries in a directory, and rather than having mod_autoindex's
> > processing of this directory limited by network I/O to the client,
> > mod_autoindex will loop until all storage is exhausted.
>
> You are taking a part of the patch that was a quick hack and extrapolating
> that it is the only way to do this.  I added the pass_brigade where I did
> because I wanted a working example.  A real example will actually have to
> make this call whenever it makes sense.

Do you mean that you would expect a module using this API to regularly
decide whether or not to call ap_pass_brigade()?  It would be really
nice to avoid that.  But avoiding it breaks your desire to avoid
having an API which knows about both filters and buckets.

> > I really think we want some changes with the coalesce property but
> > also the flush-to-the-network-as-we-go property, rather than
> > continually building up the brigade until the module sees fit to call
> > ap_pass_brigade().
>
> If you can design/implement this, please feel free.  As I see it, we have
> multiple people with different priorities.  This is what I have currently
> heard as priorities, please add any if I miss some.

As I recall, Victor already started down this road but was shot down.

> Don't change the ap_r* API at all
> Good performance for ap_r* functions
> Ability to use both ap_bucket_create and ap_r* functions in the same
>       function
> Ability to use network congestion to stop sending data
> Don't allow the server to even try to use up all the memory
>
> You can't have all five of these.  Pick a few and design for them.

As long as the app calls a flush routine (ap_rflush()), all five of
these were met by Victor's patch if I recall correctly.

> > But to do that the API needs to know about filters too (enough to call
> > ap_pass_brigade()), so there is another parameter to
> > ap_brigade_something().
>
> No, that is tying the buckets to the filters, which ties buckets to Apache
> basically, which is not correct IMHO.

The app has to do extra work if this type of API doesn't handle
buckets and filters.

> It doesn't matter.  The biggest problem with Victor's patch is that it
> puts the buffer in the request_rec, which means that we aren't helping
> anybody other than Apache itself.

As I mentioned before, macros could avoid having the core code
know about the request_rec.

  #define ap_rputs(s,r) \
  ap_brigade_puts(r->field1, r->field2, x, y)

  (or some such)

But the key issue is whether some API knows buckets and filters.  To
meet what I thought was the goal ("Allow apps which use the ap_r* API
to perform reasonably"), I think we need such an API.

Thanks,
--
Jeff Trawick | trawickj@bellsouth.net | PGP public key at web site:
       http://www.geocities.com/SiliconValley/Park/9289/
             Born in Roswell... married an alien...



RE: [PATCH] bucket problems.

Posted by "Peter J. Cranstone" <cr...@remotecommunications.com>.
Guys,

Feel free to flame me or plain ignore me but I have a suggestion. Why not
leave filtering out of the equation until the beta is stable. Filtering is
going to cause issues with 1.x modules and will take time to explain to your
existing base.

What I think you need IMHO is to get 2.0 stable with just one or two
features. You have enough on the table to do that. Then introduce filtering
at a later stage. Adding the whole thing now along with all the other
features is going to result in 1.x users avoiding the upgrade.

Backwards compatibility to older modules is KEY until you have the new
filtering system stable and OUT performing anything else. Then users will go
through the pain of adapting their modules for the newer more stable, faster
Apache 2.0

This is just a thought. Ignoring me will cost less effort than flaming me
but you know where I am :)

Regards,


Peter
RCI

-----Original Message-----
From: trawick@bellsouth.net [mailto:trawick@bellsouth.net]On Behalf Of
Jeff Trawick
Sent: Wednesday, January 17, 2001 2:51 PM
To: new-httpd@apache.org
Subject: Re: [PATCH] bucket problems.


rbb@covalent.net writes:

> > I think the very idea of an API which adds to a brigade is
> > problematic.  We want an API which handles output, sending it down the
> > filter chain when necessary (i.e., when our x-kilobyte coalesce buffer
> > is full).  With your API, some weirdo will create a filesystem with
> > infinite entries in a directory, and rather than having mod_autoindex's
> > processing of this directory limited by network I/O to the client,
> > mod_autoindex will loop until all storage is exhausted.
>
> You are taking a part of the patch that was a quick hack and extrapolating
> that it is the only way to do this.  I added the pass_brigade where I did
> because I wanted a working example.  A real example will actually have to
> make this call whenever it makes sense.

Do you mean that you would expect a module using this API to regularly
decide whether or not to call ap_pass_brigade()?  It would be really
nice to avoid that.  But avoiding it breaks your desire to avoid
having an API which knows about both filters and buckets.

> > I really think we want some changes with the coalesce property but
> > also the flush-to-the-network-as-we-go property, rather than
> > continually building up the brigade until the module sees fit to call
> > ap_pass_brigade().
>
> If you can design/implement this, please feel free.  As I see it, we have
> multiple people with different priorities.  This is what I have currently
> heard as priorities, please add any if I miss some.

As I recall, Victor already started down this road but was shot down.

> Don't change the ap_r* API at all
> Good performance for ap_r* functions
> Ability to use both ap_bucket_create and ap_r* functions in the same
> 	function
> Ability to use network congestion to stop sending data
> Don't allow the server to even try to use up all the memory
>
> You can't have all five of these.  Pick a few and design for them.

As long as the app calls a flush routine (ap_rflush()), all five of
these were met by Victor's patch if I recall correctly.

> > But to do that the API needs to know about filters too (enough to call
> > ap_pass_brigade()), so there is another parameter to
> > ap_brigade_something().
>
> No, that is tying the buckets to the filters, which ties buckets to Apache
> basically, which is not correct IMHO.

The app has to do extra work if this type of API doesn't handle
buckets and filters.

> It doesn't matter.  The biggest problem with Victor's patch is that it
> puts the buffer in the request_rec, which means that we aren't helping
> anybody other than Apache itself.

As I mentioned before, macros could avoid having the core code
know about the request_rec.

  #define ap_rputs(s,r) \
  ap_brigade_puts(r->field1, r->field2, x, y)

  (or some such)

But the key issue is whether some API knows buckets and filters.  To
meet what I thought was the goal ("Allow apps which use the ap_r* API
to perform reasonably"), I think we need such an API.

Thanks,
--
Jeff Trawick | trawickj@bellsouth.net | PGP public key at web site:
       http://www.geocities.com/SiliconValley/Park/9289/
             Born in Roswell... married an alien...


Re: [PATCH] bucket problems.

Posted by Jeff Trawick <tr...@bellsouth.net>.
rbb@covalent.net writes:

> > I think the very idea of an API which adds to a brigade is
> > problematic.  We want an API which handles output, sending it down the
> > filter chain when necessary (i.e., when our x-kilobyte coalesce buffer
> > is full).  With your API, some weirdo will create a filesystem with
> > infinite entries in a directory, and rather than having mod_autoindex's
> > processing of this directory limited by network I/O to the client,
> > mod_autoindex will loop until all storage is exhausted.
> 
> You are taking a part of the patch that was a quick hack and extrapolating
> that it is the only way to do this.  I added the pass_brigade where I did
> because I wanted a working example.  A real example will actually have to
> make this call whenever it makes sense.

Do you mean that you would expect a module using this API to regularly
decide whether or not to call ap_pass_brigade()?  It would be really
nice to avoid that.  But avoiding it breaks your desire to avoid
having an API which knows about both filters and buckets.

> > I really think we want some changes with the coalesce property but
> > also the flush-to-the-network-as-we-go property, rather than
> > continually building up the brigade until the module sees fit to call
> > ap_pass_brigade().
> 
> If you can design/implement this, please feel free.  As I see it, we have
> multiple people with different priorities.  This is what I have currently
> heard as priorities, please add any if I miss some.

As I recall, Victor already started down this road but was shot down.

> Don't change the ap_r* API at all
> Good performance for ap_r* functions
> Ability to use both ap_bucket_create and ap_r* functions in the same
> 	function
> Ability to use network congestion to stop sending data
> Don't allow the server to even try to use up all the memory
> 
> You can't have all five of these.  Pick a few and design for them.

As long as the app calls a flush routine (ap_rflush()), all five of
these were met by Victor's patch if I recall correctly.

> > But to do that the API needs to know about filters too (enough to call
> > ap_pass_brigade()), so there is another parameter to
> > ap_brigade_something().
> 
> No, that is tying the buckets to the filters, which ties buckets to Apache
> basically, which is not correct IMHO.

The app has to do extra work if this type of API doesn't handle
buckets and filters.

> It doesn't matter.  The biggest problem with Victor's patch is that it
> puts the buffer in the request_rec, which means that we aren't helping
> anybody other than Apache itself.

As I mentioned before, macros could avoid having the core code
know about the request_rec.

  #define ap_rputs(s,r) \
  ap_brigade_puts(r->field1, r->field2, x, y)

  (or some such)

But the key issue is whether some API knows buckets and filters.  To
meet what I thought was the goal ("Allow apps which use the ap_r* API
to perform reasonably"), I think we need such an API.

Thanks,
-- 
Jeff Trawick | trawickj@bellsouth.net | PGP public key at web site:
       http://www.geocities.com/SiliconValley/Park/9289/
             Born in Roswell... married an alien...

Re: [PATCH] bucket problems.

Posted by rb...@covalent.net.
> I think the very idea of an API which adds to a brigade is
> problematic.  We want an API which handles output, sending it down the
> filter chain when necessary (i.e., when our x-kilobyte coalesce buffer
> is full).  With your API, some weirdo will create a filesystem with
> infinite entries in a directory, and rather than having mod_autoindex's
> processing of this directory limited by network I/O to the client,
> mod_autoindex will loop until all storage is exhausted.

You are taking a part of the patch that was a quick hack and extrapolating
that it is the only way to do this.  I added the pass_brigade where I did
because I wanted a working example.  A real example will actually have to
make this call whenever it makes sense.

> I really think we want some changes with the coalesce property but
> also the flush-to-the-network-as-we-go property, rather than
> continually building up the brigade until the module sees fit to call
> ap_pass_brigade().

If you can design/implement this, please feel free.  As I see it, we have
multiple people with different priorities.  This is what I have currently
heard as priorities, please add any if I miss some.

Don't change the ap_r* API at all
Good performance for ap_r* functions
Ability to use both ap_bucket_create and ap_r* functions in the same
	function
Ability to use network congestion to stop sending data
Don't allow the server to even try to use up all the memory

You can't have all five of these.  Pick a few and design for them.

> But to do that the API needs to know about filters too (enough to call
> ap_pass_brigade()), so there is another parameter to
> ap_brigade_something().

No, that is tying the buckets to the filters, which ties buckets to Apache
basically, which is not correct IMHO.  The other option is to pass a
bucket_brigade to ap_r*, and pass things down when the brigade gets too
large, this would be realtively easy to do, but it requires modifying the
ap_r* API, which we were told is a no-no.  This also gets very confusing,
because when you use ap_r* you don't have to call ap_pass_brigade, but if
you don't use ap_r* you do need to make that call.  I have to say that if
you really want to confuse people, then offer five different ways to
generate content.

> many people will mix calls to fprintf() with calls to write()?  (Okay,
> you can but you gotta get stdio to flush before bypassing stdio...  We
> can allow that with ap_rXXX() too.)

It doesn't matter.  The biggest problem with Victor's patch is that it
puts the buffer in the request_rec, which means that we aren't helping
anybody other than Apache itself.

Ryan

_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------








Re: [PATCH] bucket problems.

Posted by Jeff Trawick <tr...@bellsouth.net>.
rbb@covalent.net writes:

> This patch addresses Dean's problems with the bucket code.  This basically
> introduces a simple buffer into the brigade, which can be used to coalesce
> data before it is even put into a bucket.  This also adds a couple of
> functions to just append data to the end of the brigade, using this
> buffer.  There are going to be more of these functions, but this is what
> was required to make mod_autoindex work.  That is the other half of this
> patch, I have modified mod_autoindex to use those functions instead of
> ap_r* to actually write the data for the request.

I think the very idea of an API which adds to a brigade is
problematic.  We want an API which handles output, sending it down the
filter chain when necessary (i.e., when our x-kilobyte coalesce buffer
is full).  With your API, some weirdo will create a filesystem with
infinite entries in a directory, and rather than having mod_autoindex's
processing of this directory limited by network I/O to the client,
mod_autoindex will loop until all storage is exhausted.

With other types of modules you could envision more practical
situations where the module eats up infinite storage in the brigade
built by ap_brigade_XXX().  The general point is that we can't consume
arbitrary amounts of storage for the coalesce buffers.

I really think we want some changes with the coalesce property but
also the flush-to-the-network-as-we-go property, rather than
continually building up the brigade until the module sees fit to call
ap_pass_brigade().

But to do that the API needs to know about filters too (enough to call
ap_pass_brigade()), so there is another parameter to
ap_brigade_something().

It isn't so important to me that the reasonably-performing ap_r*
equivalent is independent of HTTP/request_rec.  In fact, it is useful
for it not to be (as fewer module code has to be different for 2.0).

If it is so important to others, consider also providing ap_rputs() et
al which are macros which access appropriate fields in the request_rec
and pass them to ap_brigade_something().

> Finally, module writers can use both ap_brigade_* functions and
> ap_bucket_create_* functions at the same time, but they must be
> diligent.  

Ummm... I didn't see that the Victor-type patch was so confusing.  How
many people will mix calls to fprintf() with calls to write()?  (Okay,
you can but you gotta get stdio to flush before bypassing stdio...  We
can allow that with ap_rXXX() too.)

Anyway, when you describe what has to be done to make it work with
your API, it doesn't seem significantly easier than what an app as to
do with fprintf()+write() or Victor's-ap_rputs()+ap_pass_brigade().

Have fun...
-- 
Jeff Trawick | trawickj@bellsouth.net | PGP public key at web site:
       http://www.geocities.com/SiliconValley/Park/9289/
             Born in Roswell... married an alien...

Re: [PATCH] bucket problems.

Posted by Bill Stoddard <bi...@wstoddard.com>.

> rbb@covalent.net wrote:
> >
> > > > was required to make mod_autoindex work.  That is the other half of
this
> > > > patch, I have modified mod_autoindex to use those functions instead of
> > > > ap_r* to actually write the data for the request.
> > >
> > > One obvious question: is there a _really_ good reason to change the name
> > > of ap_rputs?  This dumps unnecessary work onto the module authors IMO,
> > > at a time we are trying to stabilize the code base.  If this code is
> > > solid, why not just hijack the ap_rputs entry point?
> >
> > A couple of reasons.
> >
> > 1)  ap_r* functions are httpd specific,
>
> ... and that's what Apache modules are for the most part....
>
> >                           these need to be brigade
> > functions.
> >
>
> at the plumbing level, yes.  at the module API level, booooo!!
> hissss!!

The buffer is associated with a brigade. It makes sense to associate the
methods that operate on that buffer with the brigade as well.  Yes,
ap_brigade_blah is a bit long winded (for C code anyways :-) but big deal.  A
handler like mod_autoindex written to this new API will contain two
conceptually new things: a call to ap_brigade_create() and a call to
ap_pass_brigade()).  The other calls (in this handler) that operate on the
brigade are symantically identical to BUFF calls. A handler author doesn't
need to understand all the bucket manipulation API which is the goodness I was
looking for.  I am not quite to the point of wanting to totally eliminate the
ap_r* calls but I'm close.

Bill


Re: [PATCH] bucket problems.

Posted by rb...@covalent.net.
> How about turning ap_r* into httpd-specific wrappers that know how to
> call your new functions, while presenting the current API appearance to
> Apache modules?  The wrappers would have to know how to get a brigade
> from the r, of course.  Then everybody works on day 1 with no

Where are you going to put the brigade?  You can't put a brigade into the
request_rec.  It doesn't belong there.  You have no idea what later
modules are going to do with that brigade.  Plus, now each filter has two
brigades, one from the request_rec, and one from the argument list.  Which
one should be used?  How are you going to document that the brigade in the
request_rec is only valid in the handler phase?  Why are we adding more to
the request_rec?  Apache 2.0 supports the ability to add more protocols,
and many of those are going to be hampered by needing a request_rec, we
should be ripping stuff out of that structure, not adding more.

> effort/code churn in module land.  If we later find out that the cost of

It is absolutely impossible for modules to just work out of the box.  Get
that whole idea out of your head please.  The module writer is going to
need to control passing the brigade down to the next filter.  If the ap_r*
functions remain, modules will continue to compile, but they won't
work.  Try removing the call to ap_pass_brigade from mod_autoindex, and
run it.  You won't get any data.

> I see a lot of changed lines in mod_autoindex, and they're not doing
> anything worth bragging about at ApacheCon.

So what!  This has nothing to do with making changes that we can brag
about.  This has to do with what is the correct approach for the module
writer.  In Apache 1.3, module authors were writing data for the
request.  In Apache 2.0, module writers are writing data for the next
filter.  The ideas are different, and how they are treated are
different.  The big problem we had with keeping the ap_r* functions, is
that they don't have the correct information to work successfully.  This
solves that by giving the functions the correct fields.

Ryan
_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------



Re: [PATCH] bucket problems.

Posted by Greg Ames <gr...@raleigh.ibm.com>.
rbb@covalent.net wrote:
> 
> > > was required to make mod_autoindex work.  That is the other half of this
> > > patch, I have modified mod_autoindex to use those functions instead of
> > > ap_r* to actually write the data for the request.
> >
> > One obvious question: is there a _really_ good reason to change the name
> > of ap_rputs?  This dumps unnecessary work onto the module authors IMO,
> > at a time we are trying to stabilize the code base.  If this code is
> > solid, why not just hijack the ap_rputs entry point?
> 
> A couple of reasons.
> 
> 1)  ap_r* functions are httpd specific, 

... and that's what Apache modules are for the most part....

>                           these need to be brigade
> functions.
> 

at the plumbing level, yes.  at the module API level, booooo!! 
hissss!!  


> 2)  ap_r* functions work on a request_rec, these work on a bucket_brigade,
> so calling them ap_r* doesn't make any sense, because there is no r.
> 
> 3)  Even if we kept the function name the same, the API is different, and
> if somebody is going to go in and modify the ap_rputs call to pass a
> bucket_brigade instead of a request_rec, they can change the name too.  I
> also fully expect that the ap_r* functions will go away completely, but
> this allows us to have a working server for the day it will take to make
> that happen.
> 

How about turning ap_r* into httpd-specific wrappers that know how to
call your new functions, while presenting the current API appearance to
Apache modules?  The wrappers would have to know how to get a brigade
from the r, of course.  Then everybody works on day 1 with no
effort/code churn in module land.  If we later find out that the cost of
the wrappers actually matter in some cases, we can change individual
modules in a controlled manner.

I see a lot of changed lines in mod_autoindex, and they're not doing
anything worth bragging about at ApacheCon.

> 
> > One thing that I will be looking for is what will cost of the *average*
> > ap_rputs/ap_brigade_puts call be with the new scheme...how close will it
> > come to 1.3?  It needs to be not much more than one or two function
> > calls, a few tests, and a copy into the buffer.
> 
> The average ap_brigade_puts call is a simple memcpy, and that is it.  

Coool!  I was hoping that was the case.  

Greg

Re: [PATCH] bucket problems.

Posted by rb...@covalent.net.
> > was required to make mod_autoindex work.  That is the other half of this
> > patch, I have modified mod_autoindex to use those functions instead of
> > ap_r* to actually write the data for the request.
> 
> One obvious question: is there a _really_ good reason to change the name
> of ap_rputs?  This dumps unnecessary work onto the module authors IMO,
> at a time we are trying to stabilize the code base.  If this code is
> solid, why not just hijack the ap_rputs entry point?

A couple of reasons.

1)  ap_r* functions are httpd specific, these need to be brigade
functions.

2)  ap_r* functions work on a request_rec, these work on a bucket_brigade,
so calling them ap_r* doesn't make any sense, because there is no r.

3)  Even if we kept the function name the same, the API is different, and
if somebody is going to go in and modify the ap_rputs call to pass a
bucket_brigade instead of a request_rec, they can change the name too.  I
also fully expect that the ap_r* functions will go away completely, but
this allows us to have a working server for the day it will take to make
that happen.

> > Finally, module writers can use both ap_brigade_* functions and
> > ap_bucket_create_* functions at the same time, but they must be
> > diligent.  
> 
> I think you are saying you have a solution for Greg Stein's concern
> about mixing the APIs.  Cool!  
> 
> But, as stated above, I would like to maintain the ap_rputs API to make
> life easier for module writers and to minimize changes to innocent
> modules.  

Modules are going to change regardless if we implement this.  We are no
longer passing around a request_rec, we are passing around a
bucket_brigade, and that should be obvious just by looking at the function
name.

> One thing that I will be looking for is what will cost of the *average*
> ap_rputs/ap_brigade_puts call be with the new scheme...how close will it
> come to 1.3?  It needs to be not much more than one or two function
> calls, a few tests, and a copy into the buffer.

The average ap_brigade_puts call is a simple memcpy, and that is it.  At
either end of the scale (buffer non-existant [see below], buffer
full) this becomes a palloc and a memcpy, but even that is much less
expensive than what we have now.

I have found one problem with the patch, and a simple solution.  Right
now, allocating a brigade is cheap, with this patch, it becomes a 4K+
allocation, which is unacceptable to me.  The solution is easy, we just
delay creating the buffer until the first call to one of the ap_brigade_*
functions.  When I commit the real thing, this will be implemented as
well.

Ryan

_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------


Re: [PATCH] bucket problems.

Posted by Greg Ames <gr...@raleigh.ibm.com>.
rbb@covalent.net wrote:
> 

Ryan, I haven't given this a decent review yet, which I plan to do.  But
I'd like to throw out a few comments from the 20,000 foot level.

>                                                             this is what
> was required to make mod_autoindex work.  That is the other half of this
> patch, I have modified mod_autoindex to use those functions instead of
> ap_r* to actually write the data for the request.
> 

One obvious question: is there a _really_ good reason to change the name
of ap_rputs?  This dumps unnecessary work onto the module authors IMO,
at a time we are trying to stabilize the code base.  If this code is
solid, why not just hijack the ap_rputs entry point?

> 
> Finally, module writers can use both ap_brigade_* functions and
> ap_bucket_create_* functions at the same time, but they must be
> diligent.  

I think you are saying you have a solution for Greg Stein's concern
about mixing the APIs.  Cool!  

But, as stated above, I would like to maintain the ap_rputs API to make
life easier for module writers and to minimize changes to innocent
modules.  

> 
> Please review ASAP, 

yessir.

One thing that I will be looking for is what will cost of the *average*
ap_rputs/ap_brigade_puts call be with the new scheme...how close will it
come to 1.3?  It needs to be not much more than one or two function
calls, a few tests, and a copy into the buffer.

Greg