mod_logio patch testing Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002

[Brian Pane <br...@cnet.com>]

On Thu, 2002-10-24 at 19:45, David Burry wrote:
> At 08:45 PM 10/24/2002 -0500, William A. Rowe, Jr. wrote:
> >At 08:40 PM 10/24/2002, Bojan Smojver wrote:
> >>Quoting David Burry <db...@tagnet.org>:
> >>
> >>> Excellent!  I will perform some tests with that when I get a chance!  You
> >>> managed to get it working without breaking pipelining even?  That's awesome!
> >>
> >>That's what I *think*, which has been known to deviate from the truth, from time
> >>to time. However, I appreciate all input, especially results of the actual tests.
> >
> >  I recall you had tested a ton of 'little files' pipelined.
> >
> >  What might be more interesting is a 100MB download (over a fast pipe)
> >which is entirely 'sendfile'd out.  Apache would consider itself done with
> >the request long before it was finished with the connection.
> 
> 
> In case someone else wants to independently verify it...
> 
> The exact test I was doing was with a 70+ meg .tar.gz file both over a 100mbit ethernet and a 1.5mbit DSL, starting and canceling it multiple times in Windoze Internet Explorer 5 or 6 (which appears to effectively use byte range requests for subsequent tries, by the way) and monitoring what was logged each time.  This test isn't super precise on the byte count (I did not bother to go comb my IE cache) but it sure is obvious when it consistently logs the whole file size and I only received a small fraction according to the IE progress bar...  Also looking at the byte range requests with %{Range}i makes it obvious how much IE received previously on each subsequent try (IE appears to only request the part of the file it hasn't cached yet).
> 
> I was thinking of writing a script that did this in a more automated fashion... perhaps contributing a test to the apache test suite when I figure that thing out... ;o)

I just tried a similar test, using Bojan's latest patch.  Using
IE 5.5 on Windows 2k, I repeatedly requested a 32MB file from Apache
w/mod_logio (on Linux with sendfile) and stopped the transfer before
the browser had received the whole response.  Each time, the bytes-sent
value in the httpd access log was smaller than the file size.

I got similar results when using ab over the loopback as a test
client.

The only test case in which mod_logio didn't report as small a
byte count as expected was when I used telnet as the client and
stopped it (with ctrl-C) while the server was still sending the
response.  A syscall trace on the server showed that, after I
interrupted telnet, the httpd did the remainder of its sendfile
calls (each one of which managed to send the next 48KB of the
requestd file; that appears to be a kernel limit) very, very
quickly. I think that's just because telnet did a half-close
and continued reading and discarding data until the server
closed its end of the connection before actually exiting.

Based on the test results, the patch does appear to be an
improvement over the current code.  I'm planning to commit
it tonight.

Brian

Re: mod_logio patch testing Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002

[Bojan Smojver <bo...@rexursive.com>]

On Fri, 2002-10-25 at 15:31, Brian Pane wrote:

> The only test case in which mod_logio didn't report as small a
> byte count as expected was when I used telnet as the client and
> stopped it (with ctrl-C) while the server was still sending the
> response.  A syscall trace on the server showed that, after I
> interrupted telnet, the httpd did the remainder of its sendfile
> calls (each one of which managed to send the next 48KB of the
> requestd file; that appears to be a kernel limit) very, very
> quickly. I think that's just because telnet did a half-close
> and continued reading and discarding data until the server
> closed its end of the connection before actually exiting.

I have observed the same. However, if I closed xterm in which telnet was
running, instead of pressing Ctrl-C, the behaviour would change and
sizes were different. I agree with you that telnet probably continued
reading, although not displaying it. There is no way an 80 MB file (in
my test) could fly by my eyes that quickly :-)

Bojan