You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by dean gaudet <de...@arctic.org> on 2001/01/07 21:28:46 UTC
buckets and rputs/rprintf
below are messages i sent to new-httpd...
as near as i can tell the correct fix to this problem is to fix the
ap_brigade_putstr and _printf and related functions to do buffering a la
apache-1.3.
at the moment these functions result in essentially one-copy, so we're
being incredibly inefficient. not only allocating reams of memory, but
also passing itty bitty writev()s to the kernel.
if you study bprintf in apache-1.3 you'll see that it writes directly into
the 4k buffer on the BUFF, and flushes from there as needed.
my suggestion is to allocate 4k buffers, and put them into the brigade
only when they're full, and use a 1.3-style bprintf/bputs which write
direct into these buffers.
the next optimisation that will be needed is something that i called
"pagemill" in libsplim (just 'cause i drive by pagemill rd. every day).
this essentially just keeps pages (buffers) on a LIFO list rather than
freeing them. this is a general optimisation useful for all fixed-sized
allocations which tend to be re-used. (i.e. the buffers, the bucket
structures, ...)
i've not studied how this relates to i/o layering yet (or filters or
whatever they're called :)
-dean
---------- Forwarded message ----------
Reply-To: new-httpd@apache.org
Date: Sun, 7 Jan 2001 11:51:47 -0800 (PST)
From: dean gaudet <de...@arctic.org>
To: new-httpd@apache.org
Subject: woah, "GET /" with autoindex
X-comment: visit http://arctic.org/~dean/legal for information regarding
copyright and disclaimer.
woah. ok there's some serious problems here.
this is from a "GET / HTTP/1.0" which results in an autoindex (dunno why
it's not grabbing one of the index.htmls, but i'm not worrying about that,
the autoindex is probably far more interesting).
ryan, here's the memory allocation you were talking about i think. this
does do the allocation every time through. aiee. (and it looks like
linux returns memory to the system each request, note the final brk()
which truncates the heap.)
also, this is a perfect example of why zero-copy can be totally wrong.
passing 100 bytes to the kernel in each writev() in 8 elements is way
slower than passing the kernel 4096 byte buffers. i haven't written up
the test to prove it to you, but this is one of the exact problems i had
to deal with on the last zero-copy library i worked with.
this one is showstopper material.
in general you want to buffer when modules are doing dynamic generation
and you want to zero-copy when modules are sending bulk data.
-dean
accept(5, {sin_family=AF_INET, sin_port=htons(2511), sin_addr=inet_addr("204.107.140.52")}}, [16]) = 8
rt_sigaction(SIGUSR1, {SIG_IGN}, {0x807f774, [], SA_INTERRUPT|0x4000000}, 8) = 0
setsockopt(8, IPPROTO_TCP1, [1], 4) = 0
getsockname(8, {sin_family=AF_INET, sin_port=htons(8888), sin_addr=inet_addr("204.107.140.52")}}, [16]) = 0
fcntl(8, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(8, F_SETFL, O_RDWR|O_NONBLOCK) = 0
read(8, "GET / HTTP/1.0\nHost: arctic.org:"..., 8192) = 38
gettimeofday({978896760, 331262}, NULL) = 0
stat("/home/dean/ap2/htdocs", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/home/dean/ap2/htdocs/", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 9
fstat(9, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fcntl(9, F_SETFD, FD_CLOEXEC) = 0
brk(0x8152000) = 0x8152000
stat("/home/dean/ap2/htdocs/HEADER", 0xbffff6e0) = -1 ENOENT (No such file or directory)
stat("/home/dean/ap2/htdocs", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
brk(0x8154000) = 0x8154000
brk(0x8156000) = 0x8156000
brk(0x8158000) = 0x8158000
getdents(9, /* 38 entries */, 3933) = 1016
stat("/home/dean/ap2/htdocs/..", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
stat("/home/dean/ap2/htdocs/apache_pb.gif", {st_mode=S_IFREG|0664, st_size=2326, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ca", {st_mode=S_IFREG|0664, st_size=1864, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.cz", {st_mode=S_IFREG|0664, st_size=1623, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.de", {st_mode=S_IFREG|0664, st_size=2250, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.dk", {st_mode=S_IFREG|0664, st_size=1547, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ee", {st_mode=S_IFREG|0664, st_size=1867, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.el", {st_mode=S_IFREG|0664, st_size=1657, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.en", {st_mode=S_IFREG|0664, st_size=1349, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.es", {st_mode=S_IFREG|0664, st_size=1772, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.et", {st_mode=S_IFREG|0664, st_size=1867, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.fr", {st_mode=S_IFREG|0664, st_size=1559, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.he.iso8859-8", {st_mode=S_IFREG|0664, st_size=4115, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.it", {st_mode=S_IFREG|0664, st_size=1818, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ja.iso2022-jp", {st_mode=S_IFREG|0664, st_size=1610, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ja.jis", {st_mode=S_IFREG|0664, st_size=1610, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.kr.iso-kr", {st_mode=S_IFREG|0664, st_size=1303, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.kr.iso2022-kr", {st_mode=S_IFREG|0664, st_size=1303, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ltz", {st_mode=S_IFREG|0664, st_size=1876, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.lu", {st_mode=S_IFREG|0664, st_size=1876, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.nl", {st_mode=S_IFREG|0664, st_size=2007, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.no", {st_mode=S_IFREG|0664, st_size=1504, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.po.iso-pl", {st_mode=S_IFREG|0664, st_size=1480, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.pt", {st_mode=S_IFREG|0664, st_size=1812, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.pt-br", {st_mode=S_IFREG|0664, st_size=1917, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ru.cp-1251", {st_mode=S_IFREG|0664, st_size=1561, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ru.cp866", {st_mode=S_IFREG|0664, st_size=1555, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ru.iso-ru", {st_mode=S_IFREG|0664, st_size=1559, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ru.koi8-r", {st_mode=S_IFREG|0664, st_size=1555, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ru.ucs2", {st_mode=S_IFREG|0664, st_size=3134, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ru.ucs4", {st_mode=S_IFREG|0664, st_size=6268, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.ru.utf8", {st_mode=S_IFREG|0664, st_size=2288, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.se", {st_mode=S_IFREG|0664, st_size=1670, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.tw", {st_mode=S_IFREG|0664, st_size=1032, ...}) = 0
stat("/home/dean/ap2/htdocs/index.html.tw.Big5", {st_mode=S_IFREG|0664, st_size=1032, ...}) = 0
stat("/home/dean/ap2/htdocs/manual", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
getdents(9, /* 0 entries */, 3933) = 0
brk(0x815a000) = 0x815a000
brk(0x815c000) = 0x815c000
brk(0x815e000) = 0x815e000
brk(0x8160000) = 0x8160000
brk(0x8162000) = 0x8162000
brk(0x8164000) = 0x8164000
brk(0x8166000) = 0x8166000
brk(0x8168000) = 0x8168000
brk(0x816a000) = 0x816a000
brk(0x816c000) = 0x816c000
brk(0x816e000) = 0x816e000
brk(0x8170000) = 0x8170000
brk(0x8172000) = 0x8172000
brk(0x8174000) = 0x8174000
brk(0x8176000) = 0x8176000
brk(0x8178000) = 0x8178000
brk(0x817a000) = 0x817a000
brk(0x817c000) = 0x817c000
brk(0x817e000) = 0x817e000
brk(0x8180000) = 0x8180000
brk(0x8182000) = 0x8182000
brk(0x8184000) = 0x8184000
brk(0x8186000) = 0x8186000
brk(0x8188000) = 0x8188000
brk(0x818a000) = 0x818a000
brk(0x818c000) = 0x818c000
brk(0x818e000) = 0x818e000
brk(0x8190000) = 0x8190000
brk(0x8191000) = 0x8191000
brk(0x8193000) = 0x8193000
brk(0x8195000) = 0x8195000
brk(0x8197000) = 0x8197000
brk(0x8199000) = 0x8199000
brk(0x819b000) = 0x819b000
brk(0x819d000) = 0x819d000
brk(0x819f000) = 0x819f000
brk(0x81a1000) = 0x81a1000
brk(0x81a3000) = 0x81a3000
brk(0x81a5000) = 0x81a5000
brk(0x81a7000) = 0x81a7000
brk(0x81a9000) = 0x81a9000
brk(0x81ab000) = 0x81ab000
brk(0x81ad000) = 0x81ad000
brk(0x81af000) = 0x81af000
brk(0x81b1000) = 0x81b1000
brk(0x81b3000) = 0x81b3000
brk(0x81b5000) = 0x81b5000
brk(0x81b7000) = 0x81b7000
brk(0x81b9000) = 0x81b9000
brk(0x81bb000) = 0x81bb000
brk(0x81bd000) = 0x81bd000
brk(0x81bf000) = 0x81bf000
brk(0x81c1000) = 0x81c1000
brk(0x81c3000) = 0x81c3000
brk(0x81c5000) = 0x81c5000
brk(0x81c7000) = 0x81c7000
brk(0x81c9000) = 0x81c9000
brk(0x81cb000) = 0x81cb000
brk(0x81cd000) = 0x81cd000
brk(0x81cf000) = 0x81cf000
brk(0x81d1000) = 0x81d1000
brk(0x81d2000) = 0x81d2000
brk(0x81d4000) = 0x81d4000
brk(0x81d6000) = 0x81d6000
brk(0x81d8000) = 0x81d8000
brk(0x81da000) = 0x81da000
brk(0x81dc000) = 0x81dc000
brk(0x81de000) = 0x81de000
brk(0x81e0000) = 0x81e0000
brk(0x81e2000) = 0x81e2000
brk(0x81e4000) = 0x81e4000
brk(0x81e6000) = 0x81e6000
brk(0x81e8000) = 0x81e8000
brk(0x81ea000) = 0x81ea000
brk(0x81ec000) = 0x81ec000
brk(0x81ee000) = 0x81ee000
brk(0x81f0000) = 0x81f0000
brk(0x81f2000) = 0x81f2000
brk(0x81f4000) = 0x81f4000
brk(0x81f6000) = 0x81f6000
brk(0x81f8000) = 0x81f8000
brk(0x81fa000) = 0x81fa000
brk(0x81fc000) = 0x81fc000
brk(0x81fe000) = 0x81fe000
brk(0x8200000) = 0x8200000
brk(0x8202000) = 0x8202000
brk(0x8204000) = 0x8204000
brk(0x8206000) = 0x8206000
brk(0x8208000) = 0x8208000
brk(0x820a000) = 0x820a000
brk(0x820c000) = 0x820c000
brk(0x820e000) = 0x820e000
brk(0x8210000) = 0x8210000
brk(0x8212000) = 0x8212000
brk(0x8213000) = 0x8213000
brk(0x8215000) = 0x8215000
brk(0x8217000) = 0x8217000
brk(0x8219000) = 0x8219000
brk(0x821b000) = 0x821b000
brk(0x821d000) = 0x821d000
brk(0x821f000) = 0x821f000
brk(0x8221000) = 0x8221000
brk(0x8223000) = 0x8223000
brk(0x8225000) = 0x8225000
brk(0x8227000) = 0x8227000
brk(0x8229000) = 0x8229000
brk(0x822b000) = 0x822b000
brk(0x822d000) = 0x822d000
brk(0x822f000) = 0x822f000
brk(0x8231000) = 0x8231000
brk(0x8233000) = 0x8233000
brk(0x8235000) = 0x8235000
brk(0x8237000) = 0x8237000
brk(0x8239000) = 0x8239000
brk(0x823b000) = 0x823b000
brk(0x823d000) = 0x823d000
brk(0x823f000) = 0x823f000
brk(0x8241000) = 0x8241000
brk(0x8243000) = 0x8243000
brk(0x8245000) = 0x8245000
brk(0x8247000) = 0x8247000
brk(0x8249000) = 0x8249000
brk(0x824b000) = 0x824b000
brk(0x824d000) = 0x824d000
brk(0x824f000) = 0x824f000
brk(0x8251000) = 0x8251000
brk(0x8253000) = 0x8253000
brk(0x8254000) = 0x8254000
brk(0x8256000) = 0x8256000
brk(0x8258000) = 0x8258000
brk(0x825a000) = 0x825a000
brk(0x825c000) = 0x825c000
brk(0x825e000) = 0x825e000
brk(0x8260000) = 0x8260000
brk(0x8262000) = 0x8262000
brk(0x8264000) = 0x8264000
brk(0x8266000) = 0x8266000
brk(0x8268000) = 0x8268000
brk(0x826a000) = 0x826a000
brk(0x826c000) = 0x826c000
brk(0x826e000) = 0x826e000
brk(0x8270000) = 0x8270000
brk(0x8272000) = 0x8272000
brk(0x8274000) = 0x8274000
brk(0x8276000) = 0x8276000
brk(0x8278000) = 0x8278000
brk(0x827a000) = 0x827a000
brk(0x827c000) = 0x827c000
brk(0x827e000) = 0x827e000
brk(0x8280000) = 0x8280000
brk(0x8282000) = 0x8282000
brk(0x8284000) = 0x8284000
brk(0x8286000) = 0x8286000
brk(0x8288000) = 0x8288000
brk(0x828a000) = 0x828a000
brk(0x828c000) = 0x828c000
brk(0x828e000) = 0x828e000
brk(0x8290000) = 0x8290000
brk(0x8292000) = 0x8292000
brk(0x8294000) = 0x8294000
brk(0x8295000) = 0x8295000
brk(0x8297000) = 0x8297000
brk(0x8299000) = 0x8299000
brk(0x829b000) = 0x829b000
brk(0x829d000) = 0x829d000
brk(0x829f000) = 0x829f000
brk(0x82a1000) = 0x82a1000
brk(0x82a3000) = 0x82a3000
brk(0x82a5000) = 0x82a5000
brk(0x82a7000) = 0x82a7000
brk(0x82a9000) = 0x82a9000
brk(0x82ab000) = 0x82ab000
brk(0x82ad000) = 0x82ad000
brk(0x82af000) = 0x82af000
brk(0x82b1000) = 0x82b1000
brk(0x82b3000) = 0x82b3000
brk(0x82b5000) = 0x82b5000
brk(0x82b7000) = 0x82b7000
brk(0x82b9000) = 0x82b9000
brk(0x82bb000) = 0x82bb000
brk(0x82bd000) = 0x82bd000
brk(0x82bf000) = 0x82bf000
brk(0x82c1000) = 0x82c1000
brk(0x82c3000) = 0x82c3000
brk(0x82c5000) = 0x82c5000
brk(0x82c7000) = 0x82c7000
brk(0x82c9000) = 0x82c9000
brk(0x82cb000) = 0x82cb000
brk(0x82cd000) = 0x82cd000
brk(0x82cf000) = 0x82cf000
brk(0x82d1000) = 0x82d1000
brk(0x82d3000) = 0x82d3000
brk(0x82d5000) = 0x82d5000
brk(0x82d6000) = 0x82d6000
brk(0x82d8000) = 0x82d8000
brk(0x82da000) = 0x82da000
brk(0x82dc000) = 0x82dc000
brk(0x82de000) = 0x82de000
brk(0x82e0000) = 0x82e0000
brk(0x82e2000) = 0x82e2000
brk(0x82e4000) = 0x82e4000
brk(0x82e6000) = 0x82e6000
brk(0x82e8000) = 0x82e8000
brk(0x82ea000) = 0x82ea000
brk(0x82ec000) = 0x82ec000
brk(0x82ee000) = 0x82ee000
brk(0x82f0000) = 0x82f0000
brk(0x82f2000) = 0x82f2000
brk(0x82f4000) = 0x82f4000
brk(0x82f6000) = 0x82f6000
brk(0x82f8000) = 0x82f8000
brk(0x82fa000) = 0x82fa000
brk(0x82fc000) = 0x82fc000
brk(0x82fe000) = 0x82fe000
brk(0x8300000) = 0x8300000
brk(0x8302000) = 0x8302000
brk(0x8304000) = 0x8304000
brk(0x8306000) = 0x8306000
brk(0x8308000) = 0x8308000
brk(0x830a000) = 0x830a000
brk(0x830c000) = 0x830c000
brk(0x830e000) = 0x830e000
brk(0x8310000) = 0x8310000
brk(0x8312000) = 0x8312000
brk(0x8314000) = 0x8314000
brk(0x8316000) = 0x8316000
brk(0x8317000) = 0x8317000
brk(0x8319000) = 0x8319000
brk(0x831b000) = 0x831b000
brk(0x831d000) = 0x831d000
brk(0x831f000) = 0x831f000
brk(0x8321000) = 0x8321000
brk(0x8323000) = 0x8323000
brk(0x8325000) = 0x8325000
brk(0x8327000) = 0x8327000
brk(0x8329000) = 0x8329000
brk(0x832b000) = 0x832b000
brk(0x832d000) = 0x832d000
brk(0x832f000) = 0x832f000
brk(0x8331000) = 0x8331000
brk(0x8333000) = 0x8333000
brk(0x8335000) = 0x8335000
brk(0x8337000) = 0x8337000
brk(0x8339000) = 0x8339000
brk(0x833b000) = 0x833b000
brk(0x833d000) = 0x833d000
brk(0x833f000) = 0x833f000
brk(0x8341000) = 0x8341000
brk(0x8343000) = 0x8343000
brk(0x8345000) = 0x8345000
brk(0x8347000) = 0x8347000
brk(0x8349000) = 0x8349000
brk(0x834b000) = 0x834b000
brk(0x834d000) = 0x834d000
brk(0x834f000) = 0x834f000
brk(0x8351000) = 0x8351000
brk(0x8353000) = 0x8353000
brk(0x8355000) = 0x8355000
brk(0x8357000) = 0x8357000
brk(0x8358000) = 0x8358000
brk(0x835a000) = 0x835a000
brk(0x835c000) = 0x835c000
brk(0x835e000) = 0x835e000
brk(0x8360000) = 0x8360000
brk(0x8362000) = 0x8362000
brk(0x8364000) = 0x8364000
brk(0x8366000) = 0x8366000
brk(0x8368000) = 0x8368000
brk(0x836a000) = 0x836a000
brk(0x836c000) = 0x836c000
brk(0x836e000) = 0x836e000
brk(0x8370000) = 0x8370000
brk(0x8372000) = 0x8372000
brk(0x8374000) = 0x8374000
brk(0x8376000) = 0x8376000
brk(0x8378000) = 0x8378000
brk(0x837a000) = 0x837a000
brk(0x837c000) = 0x837c000
brk(0x837e000) = 0x837e000
brk(0x8380000) = 0x8380000
brk(0x8382000) = 0x8382000
brk(0x8384000) = 0x8384000
brk(0x8386000) = 0x8386000
brk(0x8388000) = 0x8388000
brk(0x838a000) = 0x838a000
brk(0x838c000) = 0x838c000
brk(0x838e000) = 0x838e000
brk(0x8390000) = 0x8390000
brk(0x8392000) = 0x8392000
brk(0x8394000) = 0x8394000
brk(0x8396000) = 0x8396000
brk(0x8398000) = 0x8398000
brk(0x8399000) = 0x8399000
brk(0x839b000) = 0x839b000
brk(0x839d000) = 0x839d000
brk(0x839f000) = 0x839f000
brk(0x83a1000) = 0x83a1000
brk(0x83a3000) = 0x83a3000
brk(0x83a5000) = 0x83a5000
brk(0x83a7000) = 0x83a7000
brk(0x83a9000) = 0x83a9000
brk(0x83ab000) = 0x83ab000
brk(0x83ad000) = 0x83ad000
brk(0x83af000) = 0x83af000
brk(0x83b1000) = 0x83b1000
brk(0x83b3000) = 0x83b3000
brk(0x83b5000) = 0x83b5000
brk(0x83b7000) = 0x83b7000
brk(0x83b9000) = 0x83b9000
brk(0x83bb000) = 0x83bb000
brk(0x83bd000) = 0x83bd000
brk(0x83bf000) = 0x83bf000
brk(0x83c1000) = 0x83c1000
brk(0x83c3000) = 0x83c3000
brk(0x83c5000) = 0x83c5000
brk(0x83c7000) = 0x83c7000
brk(0x83c9000) = 0x83c9000
brk(0x83cb000) = 0x83cb000
brk(0x83cd000) = 0x83cd000
brk(0x83cf000) = 0x83cf000
brk(0x83d1000) = 0x83d1000
brk(0x83d3000) = 0x83d3000
brk(0x83d5000) = 0x83d5000
brk(0x83d7000) = 0x83d7000
brk(0x83d9000) = 0x83d9000
brk(0x83da000) = 0x83da000
brk(0x83dc000) = 0x83dc000
brk(0x83de000) = 0x83de000
brk(0x83e0000) = 0x83e0000
brk(0x83e2000) = 0x83e2000
brk(0x83e4000) = 0x83e4000
brk(0x83e6000) = 0x83e6000
close(9) = 0
stat("/home/dean/ap2/htdocs/README", 0xbffff6e4) = -1 ENOENT (No such file or directory)
stat("/home/dean/ap2/htdocs", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
brk(0x83e8000) = 0x83e8000
writev(8, [{"HTTP/1.1 200 OK\r\nDate: Sun, 07 J"..., 251}, {"<!DOCTYPE HTML PUBLIC \"-//W3C//D"..., 56}, {"<HTML>\n <HEAD>\n <TITLE>Index of"..., 33}, {"/", 1}, {"</TITLE>\n </HEAD>\n <BODY>\n", 26}, {"<H1>Index of ", 13}, {"/", 1}, {"</H1>\n", 6}, {"<PRE>", 5}, {"<IMG SRC=\"", 10}, {"/icons/blank.gif", 16}, {"\" ALT=\" \"", 13}, {"> ", 2}, {"<A HREF=\"", 9}, {"?N=D", 4}, {"\">", 2}], 16) = 448
writev(8, [{"Name", 4}, {"</A>", 4}, {" ", 19}, {" ", 1}, {"<A HREF=\"", 9}, {"?M=A", 4}, {"\">", 2}, {"Last modified", 13}, {"</A>", 4}, {" ", 7}, {"<A HREF=\"", 9}, {"?S=A", 4}, {"\">", 2}, {"Size", 4}, {"</A>", 4}, {" ", 2}], 16) = 92
writev(8, [{"<A HREF=\"", 9}, {"?D=A", 4}, {"\">", 2}, {"Description", 11}, {"</A>", 4}, {"\n<HR>\n", 6}, {"<IMG SRC=\"", 10}, {"/icons/back.gif", 15}, {"\" ALT=\"[", 8}, {"DIR", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"/", 1}, {"\">", 2}, {"Parent Directory", 16}], 16) = 104
writev(8, [{"</A>", 4}, {" ", 7}, {" ", 1}, {"07-Jan-2001 11:24 ", 19}, {" -", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/image2.gif", 17}, {"\" ALT=\"[", 8}, {"IMG", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"apache_pb.gif", 13}, {"\">", 2}], 16) = 105
writev(8, [{"apache_pb.gif", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"02-Jul-1996 23:18 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ca", 13}], 16) = 117
writev(8, [{"\">", 2}, {"index.html.ca", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"03-Dec-2000 08:44 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}], 16) = 106
writev(8, [{"index.html.cz", 13}, {"\">", 2}, {"index.html.cz", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"14-Nov-2000 17:11 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}], 16) = 109
writev(8, [{" <A HREF=\"", 10}, {"index.html.de", 13}, {"\">", 2}, {"index.html.de", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"03-Dec-2000 08:44 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}], 16) = 118
writev(8, [{">", 1}, {" <A HREF=\"", 10}, {"index.html.dk", 13}, {"\">", 2}, {"index.html.dk", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"14-Nov-2000 17:11 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/unknown.gif", 18}, {"\" ALT=\"[", 8}, {" ", 3}], 16) = 120
writev(8, [{"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ee", 13}, {"\">", 2}, {"index.html.ee", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"14-Nov-2000 17:11 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}], 16) = 116
writev(8, [{"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.el", 13}, {"\">", 2}, {"index.html.el", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"14-Dec-2000 19:46 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}], 16) = 111
writev(8, [{"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.en", 13}, {"\">", 2}, {"index.html.en", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"03-Dec-2000 08:44 ", 19}, {" 1k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}], 16) = 104
writev(8, [{"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.es", 13}, {"\">", 2}, {"index.html.es", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"14-Nov-2000 17:11 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}], 16) = 109
writev(8, [{"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.et", 13}, {"\">", 2}, {"index.html.et", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"14-Nov-2000 17:28 ", 19}, {" 2k", 5}, {" ", 2}], 16) = 118
writev(8, [{"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.fr", 13}, {"\">", 2}, {"index.html.fr", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"04-Jan-2001 04:02 ", 19}, {" 2k", 5}], 16) = 117
writev(8, [{" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/unknown.gif", 18}, {"\" ALT=\"[", 8}, {" ", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.he.iso8859-8", 23}, {"\">", 2}, {"index.html.he.iso8859-8", 23}, {"</A>", 4}, {" ", 1}, {"03-Dec-2000 08:48 ", 19}, {" 4k", 5}], 16) = 132
writev(8, [{" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.it", 13}, {"\">", 2}, {"index.html.it", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"03-Dec-2000 08:44 ", 19}], 16) = 114
writev(8, [{" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ja.iso2022-jp", 24}, {"\">", 2}, {"index.html.ja.iso202..>", 26}, {"</A>", 4}, {" ", 1}, {"03-Dec-2000 08:48 ", 19}], 16) = 133
writev(8, [{" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ja.jis", 17}, {"\">", 2}, {"index.html.ja.jis", 17}, {"</A>", 4}, {" ", 6}, {" ", 1}], 16) = 104
writev(8, [{"03-Dec-2000 08:48 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/unknown.gif", 18}, {"\" ALT=\"[", 8}, {" ", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.kr.iso-kr", 20}, {"\">", 2}, {"index.html.kr.iso-kr", 20}, {"</A>", 4}, {" ", 3}], 16) = 128
writev(8, [{" ", 1}, {"03-Dec-2000 08:48 ", 19}, {" 1k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.kr.iso2022-kr", 24}, {"\">", 2}, {"index.html.kr.iso202..>", 26}, {"</A>", 4}], 16) = 133
writev(8, [{" ", 1}, {"03-Dec-2000 08:48 ", 19}, {" 1k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ltz", 14}, {"\">", 2}, {"index.html.ltz", 14}, {"</A>", 4}], 16) = 111
writev(8, [{" ", 9}, {" ", 1}, {"03-Dec-2000 08:44 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/unknown.gif", 18}, {"\" ALT=\"[", 8}, {" ", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.lu", 13}, {"\">", 2}, {"index.html.lu", 13}], 16) = 117
writev(8, [{"</A>", 4}, {" ", 10}, {" ", 1}, {"03-Dec-2000 08:44 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.nl", 13}, {"\">", 2}], 16) = 106
writev(8, [{"index.html.nl", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"14-Nov-2000 17:11 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.no", 13}], 16) = 117
writev(8, [{"\">", 2}, {"index.html.no", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"19-Oct-2000 12:18 ", 19}, {" 1k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/unknown.gif", 18}, {"\" ALT=\"[", 8}, {" ", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}], 16) = 109
writev(8, [{"index.html.po.iso-pl", 20}, {"\">", 2}, {"index.html.po.iso-pl", 20}, {"</A>", 4}, {" ", 3}, {" ", 1}, {"19-Dec-2000 13:30 ", 19}, {" 1k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}], 16) = 116
writev(8, [{" <A HREF=\"", 10}, {"index.html.pt", 13}, {"\">", 2}, {"index.html.pt", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}, {"03-Dec-2000 08:44 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}], 16) = 118
writev(8, [{">", 1}, {" <A HREF=\"", 10}, {"index.html.pt-br", 16}, {"\">", 2}, {"index.html.pt-br", 16}, {"</A>", 4}, {" ", 7}, {" ", 1}, {"14-Nov-2000 17:11 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}], 16) = 120
writev(8, [{"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ru.cp-1251", 21}, {"\">", 2}, {"index.html.ru.cp-1251", 21}, {"</A>", 4}, {" ", 2}, {" ", 1}, {"03-Dec-2000 08:48 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}], 16) = 124
writev(8, [{"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ru.cp866", 19}, {"\">", 2}, {"index.html.ru.cp866", 19}, {"</A>", 4}, {" ", 4}, {" ", 1}, {"03-Dec-2000 08:48 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}], 16) = 117
writev(8, [{"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ru.iso-ru", 20}, {"\">", 2}, {"index.html.ru.iso-ru", 20}, {"</A>", 4}, {" ", 3}, {" ", 1}, {"03-Dec-2000 08:48 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}], 16) = 111
writev(8, [{"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ru.koi8-r", 20}, {"\">", 2}, {"index.html.ru.koi8-r", 20}, {"</A>", 4}, {" ", 3}, {" ", 1}, {"03-Dec-2000 08:48 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}], 16) = 116
writev(8, [{"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ru.ucs2", 18}, {"\">", 2}, {"index.html.ru.ucs2", 18}, {"</A>", 4}, {" ", 5}, {" ", 1}, {"04-Apr-2000 08:41 ", 19}, {" 3k", 5}, {" ", 2}], 16) = 123
writev(8, [{"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ru.ucs4", 18}, {"\">", 2}, {"index.html.ru.ucs4", 18}, {"</A>", 4}, {" ", 5}, {" ", 1}, {"04-Apr-2000 08:41 ", 19}, {" 6k", 5}], 16) = 122
writev(8, [{" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.ru.utf8", 18}, {"\">", 2}, {"index.html.ru.utf8", 18}, {"</A>", 4}, {" ", 5}, {" ", 1}, {"03-Dec-2000 08:48 ", 19}], 16) = 119
writev(8, [{" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.se", 13}, {"\">", 2}, {"index.html.se", 13}, {"</A>", 4}, {" ", 10}, {" ", 1}], 16) = 100
writev(8, [{"03-Dec-2000 08:44 ", 19}, {" 2k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.tw", 13}, {"\">", 2}, {"index.html.tw", 13}, {"</A>", 4}, {" ", 10}], 16) = 118
writev(8, [{" ", 1}, {"03-Dec-2000 08:44 ", 19}, {" 1k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/text.gif", 15}, {"\" ALT=\"[", 8}, {"TXT", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"index.html.tw.Big5", 18}, {"\">", 2}, {"index.html.tw.Big5", 18}, {"</A>", 4}], 16) = 119
writev(8, [{" ", 5}, {" ", 1}, {"03-Dec-2000 08:48 ", 19}, {" 1k", 5}, {" ", 2}, {"\n", 1}, {"<IMG SRC=\"", 10}, {"/icons/folder.gif", 17}, {"\" ALT=\"[", 8}, {"DIR", 3}, {"]\"", 2}, {">", 1}, {" <A HREF=\"", 10}, {"manual/", 7}, {"\">", 2}, {"manual/", 7}], 16) = 100
read(8, 0x814cfb0, 8192) = -1 EAGAIN (Resource temporarily unavailable)
writev(8, [{"</A>", 4}, {" ", 16}, {" ", 1}, {"07-Jan-2001 10:36 ", 19}, {" -", 5}, {" ", 2}, {"\n", 1}, {"</PRE>", 6}, {"<HR>\n", 5}, {"<ADDRESS>Apache/2.0b1-dev Server"..., 76}, {"</BODY></HTML>\n", 15}], 11) = 150
brk(0x8150000) = 0x8150000
write(4, "204.107.140.52 - - [07/Jan/2001:"..., 74) = 74
shutdown(8, 1 /* send */) = 0
gettimeofday({978896760, 422093}, NULL) = 0
read(8, 0xbffff760, 512) = -1 EAGAIN (Resource temporarily unavailable)
select(9, [8], NULL, NULL, {30, 0}) = 1 (in [8], left {29, 980000})
read(8, "", 512) = 0
close(8) = 0
rt_sigaction(SIGUSR1, {0x807f774, [], SA_INTERRUPT|0x4000000}, {SIG_IGN}, 8) = 0
accept(5, <unfinished ...>
---------- Forwarded message ----------
Reply-To: new-httpd@apache.org
Date: Sun, 7 Jan 2001 11:57:41 -0800 (PST)
From: dean gaudet <dg...@arctic.org>
To: new-httpd@apache.org
Subject: Re: woah, "GET /" with autoindex
X-comment: visit http://arctic.org/~dean/legal for information regarding
copyright and disclaimer.
On Sun, 7 Jan 2001, dean gaudet wrote:
> woah. ok there's some serious problems here.
>
> this is from a "GET / HTTP/1.0" which results in an autoindex (dunno why
> it's not grabbing one of the index.htmls, but i'm not worrying about that,
> the autoindex is probably far more interesting).
btw, the tcpdump for this shows suboptimal packets as well. this is
because without nagle, we've told the kernel that each of our writev()s is
to be put on the network.
11:53:12.839439 eth0 < client.1131 > server.8888: S 1366931848:1366931848(0) win 32120 <mss 1460,sackOK,timestamp 983960148 0,nop,wscale 0> (DF)
11:53:12.839531 eth0 > server.8888 > client.1131: S 1369163235:1369163235(0) ack 1366931849 win 32120 <mss 1460,sackOK,timestamp 769294182 983960148,nop,wscale 0> (DF)
11:53:12.856197 eth0 < client.1131 > server.8888: . 1:1(0) ack 1 win 32120 <nop,nop,timestamp 983960150 769294182> (DF)
11:53:14.969574 eth0 < client.1131 > server.8888: P 1:17(16) ack 1 win 32120 <nop,nop,timestamp 983960361 769294182> (DF)
11:53:14.969626 eth0 > server.8888 > client.1131: . 1:1(0) ack 17 win 32120 <nop,nop,timestamp 769294395 983960361> (DF)
11:53:15.109544 eth0 < client.1131 > server.8888: P 17:19(2) ack 1 win 32120 <nop,nop,timestamp 983960375 769294395> (DF)
11:53:15.119937 eth0 > server.8888 > client.1131: . 1:1(0) ack 19 win 32120 <nop,nop,timestamp 769294411 983960375> (DF)
11:53:15.139046 eth0 > server.8888 > client.1131: P 1:449(448) ack 19 win 32120 <nop,nop,timestamp 769294412 983960375> (DF)
11:53:15.139164 eth0 > server.8888 > client.1131: P 449:541(92) ack 19 win 32120 <nop,nop,timestamp 769294412 983960375> (DF)
11:53:15.167745 eth0 < client.1131 > server.8888: . 19:19(0) ack 449 win 32120 <nop,nop,timestamp 983960381 769294412> (DF)
11:53:15.167806 eth0 > server.8888 > client.1131: P 541:1989(1448) ack 19 win 32120 <nop,nop,timestamp 769294415 983960381> (DF)
11:53:15.167821 eth0 > server.8888 > client.1131: P 1989:3437(1448) ack 19 win 32120 <nop,nop,timestamp 769294415 983960381> (DF)
11:53:15.174817 eth0 < client.1131 > server.8888: . 19:19(0) ack 541 win 32120 <nop,nop,timestamp 983960382 769294412> (DF)
11:53:15.174882 eth0 > server.8888 > client.1131: P 3437:4885(1448) ack 19 win 32120 <nop,nop,timestamp 769294416 983960382> (DF)
11:53:15.174904 eth0 > server.8888 > client.1131: FP 4885:5072(187) ack 19 win 32120 <nop,nop,timestamp 769294416 983960382> (DF)
11:53:15.207674 eth0 < client.1131 > server.8888: . 19:19(0) ack 3437 win 31856 <nop,nop,timestamp 983960385 769294415> (DF)
11:53:15.227760 eth0 < client.1131 > server.8888: . 19:19(0) ack 5073 win 30408 <nop,nop,timestamp 983960387 769294416> (DF)
11:53:15.238239 eth0 < client.1131 > server.8888: F 19:19(0) ack 5073 win 31856 <nop,nop,timestamp 983960388 769294416> (DF)
11:53:15.238296 eth0 > server.8888 > client.1131: . 5073:5073(0) ack 20 win 32120 <nop,nop,timestamp 769294422 983960388> (DF)
1.3 would generate three 1448 byte packets plus one straggler.
on linux you could CORK in 2.0 and unCORK at the end and it'd generate the
right packets... but that's not portable. so the fix needs to live
somewhere in the i/o layering... which i'm not familiar enough with yet.
-dean
Re: buckets and rputs/rprintf
Posted by rb...@covalent.net.
> > The second question is where would the buffer live? In the
> > old BUFF code, it was easy, because there was only one BUFF per
> > request. Now, we have multiple buckets per request. We could put the
> > buffer in the brigade itself, but that makes the brigade much more
> > heavy-weight than it is right now. One of the very nice things about the
> > current brigades, is that they are so light-weight. We can drop them on
> > the ground, and let the pool worry about them. If we are allocating a 4k
> > buffer inside of them, we really have to take the brigade outside of the
> > pool and use malloc/free on it, which takes away some security.
>
> dude, buckets aren't "light-weight"... look at the malloc()s and the
No, brigades are light-weight buckets really aren't. We have tried to
make the buckets more light-weight, but obviously we haven't gotten it
right yet. But, you can't just put the buffer into a bucket, because
different buckets have different properties.
> writev() crud. that's not light weight. as far as you've explained to me
> so far you don't have a good solution for either of these problems.
>
> look at 2.0 ap_rputs:
>
> bb = ap_brigade_create(r->pool);
> b = ap_bucket_create_transient(str, len);
> AP_BRIGADE_INSERT_TAIL(bb, b);
> ap_pass_brigade(r->output_filters, bb);
>
>
> look at 1.3 ap_rputs:
>
> rcode = ap_bputs(str, r->connection->client);
>
> and 1.3 ap_bputs:
>
> int i, j = strlen(x);
> i = ap_bwrite(fb, x, j);
>
> tada. 2.0 allocates a brigade, a bucket, a memory region, copies into the
> memory region, appends it to a list.
Which is completely wrong. We should be buffering the data, and putting
the buffer into a heap bucket, so that everything is one-copy. The
buffering goes into the ap_r* functions though, so it is do-able.
> > When I originally wrote some of the bucket code, I had heap buckets
> > always allocate a 4k buffer, and if we were adding to the end of a
> > brigade that has a heap at the end, we just appended the new data into
> > the last bucket. This ends up being a very difficult thing to do
> > well.
>
> it's difficult, so? look at the 1.3 BUFF and tell me it wasn't difficult
> to get right :)
But it's more than that. Most filters aren't going to just create heap
buckets, they are going to create a bunch of different bucket types. Take
a look at mod_include. In that case, the 4k buffer just doesn't help
much.
> > I personally don't mind stupid generators performing badly. I would much
> > rather just re-write them to work well, than to try to force the old API
> > to perform as well as the new API.
>
> i don't consider mod_autoindex "stupid". are you aware that PHP uses
> rputs/etc. as well? i don't consider PHP stupid either.
>
> the old API was convenient for the module programmer. the new API is
> convenient for you. that's the wrong solution.
The thing is, the new API is the only API for filter writers, so PHP is
strictly a generator right now. The old API just doesn't interact
correctly with the new model. That is a lack of time/nudging, not a lack
of ability for it to. There have been many people talking about making
the old API work much better with the underlying buckets, but nobody has
done it yet. The buffer is the correct solution, but it belongs in
Apache's API, not the bucket API. The bucket API just doesn't have a
place to put it that makes sense.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: buckets and rputs/rprintf
Posted by dean gaudet <de...@arctic.org>.
On Sun, 7 Jan 2001 rbb@covalent.net wrote:
>
> > as near as i can tell the correct fix to this problem is to fix the
> > ap_brigade_putstr and _printf and related functions to do buffering a la
> > apache-1.3.
>
> Those functions aren't used anyplace, so modifying them won't help this
> problem. The solution is to modify ap_r* functions in Apache to buffer
> correctly. The problem is that Apache's mod_autoindex is using the old
> ap_r* functions instead of using the buckets directly in a sane way.
ok this is silly talking on both lists. i suppose i shouldn't have even
bothered posting anything here since this is clearly a cross httpd/apr
issue.
as i just posted to new-httpd, what i said is essentially true.
ap_rprintf results in the bucket printf... ap_rputs does essentially the
same (one-copy) thing as bucket_putstr.
> > my suggestion is to allocate 4k buffers, and put them into the brigade
> > only when they're full, and use a 1.3-style bprintf/bputs which write
> > direct into these buffers.
>
> Yes, but the buffer doesn't belong in the bucket code IMO, it belongs in
> the Apache code.
oh so buffers aren't needed in any of the other APR users? ha.
> The problem there, is that once we start to buffer, you
> can't use a combination of the bucket functions and the ap_r*
> functions.
sounds like an API problem then.
> The second question is where would the buffer live? In the
> old BUFF code, it was easy, because there was only one BUFF per
> request. Now, we have multiple buckets per request. We could put the
> buffer in the brigade itself, but that makes the brigade much more
> heavy-weight than it is right now. One of the very nice things about the
> current brigades, is that they are so light-weight. We can drop them on
> the ground, and let the pool worry about them. If we are allocating a 4k
> buffer inside of them, we really have to take the brigade outside of the
> pool and use malloc/free on it, which takes away some security.
dude, buckets aren't "light-weight"... look at the malloc()s and the
writev() crud. that's not light weight. as far as you've explained to me
so far you don't have a good solution for either of these problems.
look at 2.0 ap_rputs:
bb = ap_brigade_create(r->pool);
b = ap_bucket_create_transient(str, len);
AP_BRIGADE_INSERT_TAIL(bb, b);
ap_pass_brigade(r->output_filters, bb);
look at 1.3 ap_rputs:
rcode = ap_bputs(str, r->connection->client);
and 1.3 ap_bputs:
int i, j = strlen(x);
i = ap_bwrite(fb, x, j);
tada. 2.0 allocates a brigade, a bucket, a memory region, copies into the
memory region, appends it to a list.
1.3 allocates nothing, and copies to the final output buffer.
1.3 is light weight.
> When I originally wrote some of the bucket code, I had heap buckets
> always allocate a 4k buffer, and if we were adding to the end of a
> brigade that has a heap at the end, we just appended the new data into
> the last bucket. This ends up being a very difficult thing to do
> well.
it's difficult, so? look at the 1.3 BUFF and tell me it wasn't difficult
to get right :)
> We have a coalesce filter in Apache, that coalesces multiple buckets in
> the same brigade into a single bucket. This is one solution to not write
> such small blocks of data.
like i said, coalesce is the wrong answer. coalesce is pretty in the
abstract sense, but makes zero practical sense. it turns a one-copy
implementation into a two-copy implementation (with even more memory
allocation).
> I personally don't mind stupid generators performing badly. I would much
> rather just re-write them to work well, than to try to force the old API
> to perform as well as the new API.
i don't consider mod_autoindex "stupid". are you aware that PHP uses
rputs/etc. as well? i don't consider PHP stupid either.
the old API was convenient for the module programmer. the new API is
convenient for you. that's the wrong solution.
-dean
Re: buckets and rputs/rprintf
Posted by rb...@covalent.net.
> as near as i can tell the correct fix to this problem is to fix the
> ap_brigade_putstr and _printf and related functions to do buffering a la
> apache-1.3.
Those functions aren't used anyplace, so modifying them won't help this
problem. The solution is to modify ap_r* functions in Apache to buffer
correctly. The problem is that Apache's mod_autoindex is using the old
ap_r* functions instead of using the buckets directly in a sane way.
> my suggestion is to allocate 4k buffers, and put them into the brigade
> only when they're full, and use a 1.3-style bprintf/bputs which write
> direct into these buffers.
Yes, but the buffer doesn't belong in the bucket code IMO, it belongs in
the Apache code. The problem there, is that once we start to buffer, you
can't use a combination of the bucket functions and the ap_r*
functions. The second question is where would the buffer live? In the
old BUFF code, it was easy, because there was only one BUFF per
request. Now, we have multiple buckets per request. We could put the
buffer in the brigade itself, but that makes the brigade much more
heavy-weight than it is right now. One of the very nice things about the
current brigades, is that they are so light-weight. We can drop them on
the ground, and let the pool worry about them. If we are allocating a 4k
buffer inside of them, we really have to take the brigade outside of the
pool and use malloc/free on it, which takes away some security.
When I originally wrote some of the bucket code, I had heap buckets always
allocate a 4k buffer, and if we were adding to the end of a brigade that
has a heap at the end, we just appended the new data into the last
bucket. This ends up being a very difficult thing to do well.
We have a coalesce filter in Apache, that coalesces multiple buckets in
the same brigade into a single bucket. This is one solution to not write
such small blocks of data.
I personally don't mind stupid generators performing badly. I would much
rather just re-write them to work well, than to try to force the old API
to perform as well as the new API.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------