You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by rb...@covalent.net on 2001/01/19 17:42:19 UTC
Both ap_r* patches.
I am posting both ap_r* performance patches after they have been updated
for the new symbol names. I am attaching them, because I want to keep
them very separate. I can include them inline if anybody wants/needs me
to later.
I did make two very small changes to my patch.
1) Rename ap_brigade_standardize to apr_brigade_flush.
This just makes more sense, and makes the function name match what
it is actually doing.
2) Fix the problem where persistant connections were broken. This was a
minor fix, it was just removing a call to ap_rflush and adding a call to
ap_brigade_flush in end_output_stream.
I have tested both of these patches, but not very well. I believe that
they are exact ports of the old patches, but with the correct symbol
names. If I missed something, I apologize.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by James Sutherland <ja...@cam.ac.uk>.
On Mon, 22 Jan 2001 rbb@covalent.net wrote:
>
> > > > Not much better. :-( It would be really cool to see where the
> > > > bottlenecks are.
> > >
> > > Wait for a few days, the changes we've kicked around will go in today, and
> > > once I tie those into the core, we get down to a single stat of index.html,
> > > and then we can open the file by cloning that stat handle. Should be the
> > > smoothest dang thing you ever did see :-)
> >
> > Sounds good, but how can it help autoindex? autoindex kicks in when
> > there is no index.html (or no mod_dir possibly, or the server is broken
> > :-)
>
> The problem with the stat, is that on Windows, those are incredibly
> expensive. By changing the stat API to only get what is required, we will
> vastly speed up the requests on Windows.
Under NTFS, the timestamps, filename(s) and sizes (the stuff you see when
you "dir") are stored in the directory record, as well as the directory
record. A full-blown stat() will retrieve stuff stored in the individual
file records, meaning lots of extra MFT (Master File Table) accesses; just
retrieving the stuff "cached" in the parent directory's record will avoid
this.
(Anyone benchmarked FAT[32] vs NTFS for this kind of thing, BTW?)
James.
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
> > > Not much better. :-( It would be really cool to see where the
> > > bottlenecks are.
> >
> > Wait for a few days, the changes we've kicked around will go in today, and
> > once I tie those into the core, we get down to a single stat of index.html,
> > and then we can open the file by cloning that stat handle. Should be the
> > smoothest dang thing you ever did see :-)
>
> Sounds good, but how can it help autoindex? autoindex kicks in when
> there is no index.html (or no mod_dir possibly, or the server is broken
> :-)
The problem with the stat, is that on Windows, those are incredibly
expensive. By changing the stat API to only get what is required, we will
vastly speed up the requests on Windows.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
RE: Both ap_r* patches.
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
> From: gregames@raleigh.ibm.com [mailto:gregames@raleigh.ibm.com]
> Sent: Monday, January 22, 2001 5:26 PM
>
> > > > Apache 1.3 serves the page at 4.12 requests per second.
> > >
> > > Not much better. :-( It would be really cool to see where the
> > > bottlenecks are.
> >
> > Wait for a few days, the changes we've kicked around will go in today, and
> > once I tie those into the core, we get down to a single stat of index.html,
> > and then we can open the file by cloning that stat handle. Should be the
> > smoothest dang thing you ever did see :-)
>
> Sounds good, but how can it help autoindex? autoindex kicks in when
> there is no index.html (or no mod_dir possibly, or the server is broken
> :-)
First, it helps everywhere, because a single call to apr_stat() opens the file
handle, gets most stats (including inode), and having just cached that folder
in the filesystem, we strike again with a dir against the file to determine it's
whole name. Now apr_stat returns, and if it's a servable page, we open the
file from the apr_finfo_t structure which cached the 'no-read/no-write' file
handle, dup it with read permissions, and bang. If not, and it's a directory,
I'm envisioning some major overhauls of the semantic to leverage the apr_finfo_t
we get out of apr_dir_read() ... on win32 we obliviate the need to stat at all!
But that's rework that may not happen before beta 2.
Bill
answers our questions
about the true case of the filename on win32
Re: Both ap_r* patches.
Posted by Greg Ames <gr...@raleigh.ibm.com>.
> > > Apache 1.3 serves the page at 4.12 requests per second.
> >
> > Not much better. :-( It would be really cool to see where the
> > bottlenecks are.
>
> Wait for a few days, the changes we've kicked around will go in today, and
> once I tie those into the core, we get down to a single stat of index.html,
> and then we can open the file by cloning that stat handle. Should be the
> smoothest dang thing you ever did see :-)
Sounds good, but how can it help autoindex? autoindex kicks in when
there is no index.html (or no mod_dir possibly, or the server is broken
:-)
My guess is that all the stats for everything in the directory are the
big remaining bottleneck for autoindex. Actually, it would be nice if
we had another module around that generates a lot of small ap_r* calls
without the stat calls. This could help us scrub whichever patch is
committed, since the differences in ap_r* path length will become more
significant.
Greg
Re: Both ap_r* patches.
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
From: <rb...@covalent.net>
Sent: Monday, January 22, 2001 3:29 PM
> > > Requesting http://machine/htdocs/httpd-docs-2.0/apidoc via ab...
> > >
> > > Apache 2.0 w/o patch: 0.12 request per second !!!!
> > > Apache 2.0 with rbb.new patch: 3.64 requests per second. Wow, we are
> > > screaming now :-)
> > > Haven't tried out Greg's patch.
> > >
> >
> > Apache 1.3 serves the page at 4.12 requests per second.
>
> Not much better. :-( It would be really cool to see where the
> bottlenecks are.
Wait for a few days, the changes we've kicked around will go in today, and
once I tie those into the core, we get down to a single stat of index.html,
and then we can open the file by cloning that stat handle. Should be the
smoothest dang thing you ever did see :-)
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
> > Requesting http://machine/htdocs/httpd-docs-2.0/apidoc via ab...
> >
> > Apache 2.0 w/o patch: 0.12 request per second !!!!
> > Apache 2.0 with rbb.new patch: 3.64 requests per second. Wow, we are
> > screaming now :-)
> > Haven't tried out Greg's patch.
> >
>
> Apache 1.3 serves the page at 4.12 requests per second.
Not much better. :-( It would be really cool to see where the
bottlenecks are.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by Bill Stoddard <bi...@wstoddard.com>.
> Requesting http://machine/htdocs/httpd-docs-2.0/apidoc via ab...
>
> Apache 2.0 w/o patch: 0.12 request per second !!!!
> Apache 2.0 with rbb.new patch: 3.64 requests per second. Wow, we are
> screaming now :-)
> Haven't tried out Greg's patch.
>
Apache 1.3 serves the page at 4.12 requests per second.
Bill
RE: Both ap_r* patches.
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
> From: Bill Stoddard [mailto:bill@wstoddard.com]
> Sent: Friday, January 19, 2001 11:43 AM
>
> I was being facetious. The performance still sucks, just much MUCH less so.
> Has to do with the Windows code more than anything else now.
Ack. You are probably hurting both ways, in ab, and in httpd. I'm working
on the file system as we speak, specifically apr_stat, and then returning
to the core file processing. No canonical today, I'm working from the
objective of keeping the code flow where it is now, and extending apr_stat
to tell us what we need with the fewest cycles and smallest number of calls,
and sharing the code path with Unix.
Re: Both ap_r* patches.
Posted by Bill Stoddard <bi...@wstoddard.com>.
>
> On linux, I was actually seeing better performance on 2.0 than
> 1.3. Granted, I also saw performance that was slightly worse than 1.3.
>
> What does 1.3 look like on Windows?
>
Out of the office now so I cannot check the 1.3 performance. I'll let you know on Monday. OBTW, I
was running ab on Linux.
Bill
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
On linux, I was actually seeing better performance on 2.0 than
1.3. Granted, I also saw performance that was slightly worse than 1.3.
What does 1.3 look like on Windows?
Ryan
On Fri, 19 Jan 2001, Bill Stoddard wrote:
> I was being facetious. The performance still sucks, just much MUCH less so.
> Has to do with the Windows code more than anything else now.
>
> Bill
>
> >
> > Did you see the results I posted last night? Both of these patches have
> > about equal performance. :-) At this point, we are really no longer
> > asking about performance, we are really looking at preference about how
> > the patch is written.
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by Bill Stoddard <bi...@wstoddard.com>.
I was being facetious. The performance still sucks, just much MUCH less so.
Has to do with the Windows code more than anything else now.
Bill
>
> Did you see the results I posted last night? Both of these patches have
> about equal performance. :-) At this point, we are really no longer
> asking about performance, we are really looking at preference about how
> the patch is written.
>
> Ryan
>
> On Fri, 19 Jan 2001, Bill Stoddard wrote:
>
> > Requesting http://machine/htdocs/httpd-docs-2.0/apidoc via ab...
> >
> > Apache 2.0 w/o patch: 0.12 request per second !!!!
> > Apache 2.0 with rbb.new patch: 3.64 requests per second. Wow, we are
> > screaming now :-)
> > Haven't tried out Greg's patch.
> >
> > Bill
> > >
> > > I am posting both ap_r* performance patches after they have been updated
> > > for the new symbol names. I am attaching them, because I want to keep
> > > them very separate. I can include them inline if anybody wants/needs me
> > > to later.
> > >
> > > I did make two very small changes to my patch.
> > >
> > > 1) Rename ap_brigade_standardize to apr_brigade_flush.
> > > This just makes more sense, and makes the function name match what
> > > it is actually doing.
> > >
> > > 2) Fix the problem where persistant connections were broken. This was
a
> > > minor fix, it was just removing a call to ap_rflush and adding a call to
> > > ap_brigade_flush in end_output_stream.
> > >
> > > I have tested both of these patches, but not very well. I believe that
> > > they are exact ports of the old patches, but with the correct symbol
> > > names. If I missed something, I apologize.
> > >
> > > Ryan
> > >
> > >
> > >
> >
______________________________________________________________________________
> > _
> > > Ryan Bloom rbb@apache.org
> > > 406 29th St.
> > > San Francisco, CA 94131
> >
> ----------------------------------------------------------------------------
> > ---
> > >
> >
> >
>
>
>
______________________________________________________________________________
_
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> ----------------------------------------------------------------------------
---
>
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
Did you see the results I posted last night? Both of these patches have
about equal performance. :-) At this point, we are really no longer
asking about performance, we are really looking at preference about how
the patch is written.
Ryan
On Fri, 19 Jan 2001, Bill Stoddard wrote:
> Requesting http://machine/htdocs/httpd-docs-2.0/apidoc via ab...
>
> Apache 2.0 w/o patch: 0.12 request per second !!!!
> Apache 2.0 with rbb.new patch: 3.64 requests per second. Wow, we are
> screaming now :-)
> Haven't tried out Greg's patch.
>
> Bill
> >
> > I am posting both ap_r* performance patches after they have been updated
> > for the new symbol names. I am attaching them, because I want to keep
> > them very separate. I can include them inline if anybody wants/needs me
> > to later.
> >
> > I did make two very small changes to my patch.
> >
> > 1) Rename ap_brigade_standardize to apr_brigade_flush.
> > This just makes more sense, and makes the function name match what
> > it is actually doing.
> >
> > 2) Fix the problem where persistant connections were broken. This was a
> > minor fix, it was just removing a call to ap_rflush and adding a call to
> > ap_brigade_flush in end_output_stream.
> >
> > I have tested both of these patches, but not very well. I believe that
> > they are exact ports of the old patches, but with the correct symbol
> > names. If I missed something, I apologize.
> >
> > Ryan
> >
> >
> >
> ______________________________________________________________________________
> _
> > Ryan Bloom rbb@apache.org
> > 406 29th St.
> > San Francisco, CA 94131
> > ----------------------------------------------------------------------------
> ---
> >
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by Bill Stoddard <bi...@wstoddard.com>.
Requesting http://machine/htdocs/httpd-docs-2.0/apidoc via ab...
Apache 2.0 w/o patch: 0.12 request per second !!!!
Apache 2.0 with rbb.new patch: 3.64 requests per second. Wow, we are
screaming now :-)
Haven't tried out Greg's patch.
Bill
>
> I am posting both ap_r* performance patches after they have been updated
> for the new symbol names. I am attaching them, because I want to keep
> them very separate. I can include them inline if anybody wants/needs me
> to later.
>
> I did make two very small changes to my patch.
>
> 1) Rename ap_brigade_standardize to apr_brigade_flush.
> This just makes more sense, and makes the function name match what
> it is actually doing.
>
> 2) Fix the problem where persistant connections were broken. This was a
> minor fix, it was just removing a call to ap_rflush and adding a call to
> ap_brigade_flush in end_output_stream.
>
> I have tested both of these patches, but not very well. I believe that
> they are exact ports of the old patches, but with the correct symbol
> names. If I missed something, I apologize.
>
> Ryan
>
>
>
______________________________________________________________________________
_
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> ----------------------------------------------------------------------------
---
>
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
here's 2 more variants of the Apache::rwrite test:
no patches using ap_rwrite:
Server Software: Apache/2.0b1-dev
Server Hostname: localhost
Server Port: 8082
Document Path: /rwrite
Document Length: 429858 bytes
Concurrency Level: 5
Time taken for tests: 50.388 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 431235125 bytes
HTML transferred: 431047377 bytes
Requests per second: 19.85
Transfer rate: 8558.29 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 5 107
Processing: 187 245 1423
Total: 187 250 1530
no patches using modperl_wbucket_write:
Server Software: Apache/2.0b1-dev
Server Hostname: localhost
Server Port: 8082
Document Path: /rwrite
Document Length: 429858 bytes
Concurrency Level: 5
Time taken for tests: 25.968 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 430593864 bytes
HTML transferred: 430406490 bytes
Requests per second: 38.51
Transfer rate: 16581.71 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 6 98
Processing: 71 122 333
Total: 71 128 431
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
here's a better test, that ends up spitting out:
[2bytes] aa
[4bytes] aaaa
[8bytes] aaaaaaaa
[16bytes] aaaaaaaaaaaaaaaa
[32bytes] aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
and so on up to 8192 for each character a..Z :
package Apache::rwrite;
use strict;
sub handler {
my $r = shift;
$r->send_http_header;
for my $c ('a'..'Z') {
my $n = 2;
while ($n <= 8192) {
$r->write("[", $n, "bytes", "] ", $c x $n);
$r->write("\n");
$n *= 2;
}
}
return Apache::OK;
}
1;
"[", $n, "bytes", "] " could of course be written as "[${n}bytes] ", but i
wanted to sprinkle in some smaller pieces of data.
$r->write calls ap_rwrite() underneath for each argument, so 5 times for
the first call, once for the second.
the results show greg's faster, however, that is due to a bug, missing
lots of data (notice document length and transfered bytes). the output
with rbb.new patch is exactly the same if i run the output loop(s) outside
of apache.
i also just realized i'm not calling ap_rflush() at all, but i don't think
that is required for this test case, is it?
% ab -c 5 -n 1000 http://localhost:8082/rwrite
with rbb.new patch:
Server Software: Apache/2.0b1-dev
Server Hostname: localhost
Server Port: 8082
Document Path: /rwrite
Document Length: 429858 bytes
Concurrency Level: 5
Time taken for tests: 28.132 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 430585672 bytes
HTML transferred: 430398298 bytes
Requests per second: 35.55
Transfer rate: 15305.90 kb/
with greg.new patch:
Server Software: Apache/2.0b1-dev
Server Hostname: localhost
Server Port: 8082
Document Path: /rwrite
Document Length: 302418 bytes
Concurrency Level: 5
Time taken for tests: 24.402 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 303423701 bytes
HTML transferred: 303236140 bytes
Requests per second: 40.98
Transfer rate: 12434.38 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 7 93
Processing: 37 113 1246
Total: 37 120 1339
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
On Sun, 21 Jan 2001 rbb@covalent.net wrote:
> This is pretty consistent with all of the other tests we have run. I am a
> bit interested in your mod_perl specific patch, because it is slightly
> faster than my patch. How mod_perl specific is it?
its a super simple buffer api, no patch to core is needed. not perfect,
but heaps better than ap_rwrite() without either patch:
typedef struct {
int outcnt;
char outbuf[IOBUFSIZE];
apr_pool_t *pool;
ap_filter_t *filters;
} modperl_wbucket_t;
(note: pool and filters are only used with $filter->write, not $r->write)
MP_INLINE apr_status_t modperl_wbucket_pass(modperl_wbucket_t *wb,
const char *buf, apr_ssize_t
len)
{
apr_bucket_brigade *bb = apr_brigade_create(wb->pool);
apr_bucket *bucket = apr_bucket_create_transient(buf, len);
APR_BRIGADE_INSERT_TAIL(bb, bucket);
return ap_pass_brigade(wb->filters, bb);
}
MP_INLINE apr_status_t modperl_wbucket_flush(modperl_wbucket_t *wb)
{
apr_status_t rv = APR_SUCCESS;
if (wb->outcnt) {
rv = modperl_wbucket_pass(wb, wb->outbuf, wb->outcnt);
wb->outcnt = 0;
}
return rv;
}
MP_INLINE apr_status_t modperl_wbucket_write(modperl_wbucket_t *wb,
const char *buf,
apr_ssize_t *wlen)
{
apr_ssize_t len = *wlen;
*wlen = 0;
if ((len + wb->outcnt) > sizeof(wb->outbuf)) {
apr_status_t rv;
if ((rv = modperl_wbucket_flush(wb)) != APR_SUCCESS) {
return rv;
}
}
if (len >= sizeof(wb->outbuf)) {
*wlen = len;
return modperl_wbucket_pass(wb, buf, len);
}
else {
memcpy(&wb->outbuf[wb->outcnt], buf, len);
wb->outcnt += len;
*wlen = len;
return APR_SUCCESS;
}
}
the functions are not inlined during my tests (debug flags turned on)
the actual handler looks like this:
modperl_response_init(r);
retval = modperl_per_dir_callback(MP_RESPONSE_HANDLER, r);
modperl_response_finish(r);
where response_init sets up modperl's per-request config
(inside r->per_request_config), which contains a wbucket.
each call to $r->write calls wbucket_write underneath (instead of
ap_rwrite)
after the Perl callbacks are made, response_finish calls wbucket_flush.
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
> 'no patch' uses mod_perl's own buffer thingy. the other 2 use ap_rwrite
> with the patches. looks pretty close, but a small amount of data is
> written and ap_write is call just a few times. i'm working on another
> test to really hammer on it.
This is pretty consistent with all of the other tests we have run. I am a
bit interested in your mod_perl specific patch, because it is slightly
faster than my patch. How mod_perl specific is it?
Ryan
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
On Sun, 21 Jan 2001, Greg Stein wrote:
> And I've already found the problem in my patch. Unfortunately, I
> accidentally grabbed just a tidbit of OtherBill's apr_stat change. Now I'm
> grabbing the whole mother and rebuilding to do a final test.
ok. i put c version here anyhow:
http://www.apache.org/~dougm/rwrite_test.tar.gz
Re: Both ap_r* patches.
Posted by Greg Stein <gs...@lyra.org>.
On Sun, Jan 21, 2001 at 04:57:19PM -0800, Doug MacEachern wrote:
> On Sun, 21 Jan 2001 rbb@covalent.net wrote:
>
> > I'm not too conerned with the performance differences. The two patches
> > seem to have about the same performance when the code is working. Bugs we
> > can fix after the commit. The question is really the design of the two
> > patches. Either we use a filter, or we use a brigade API.
>
> all i can really offer at the moment is feedback on performance and
> functionality. i know just enough about filters to use the api, not
> enough to comment on the design of each patch. i'm too clueless to debug
> greg's patch, but i could write a C version of the Perl Apache::rwrite
> exercise if that helps at all.
And I've already found the problem in my patch. Unfortunately, I
accidentally grabbed just a tidbit of OtherBill's apr_stat change. Now I'm
grabbing the whole mother and rebuilding to do a final test.
I'll also reissue my patch in a moment (after OtherBill's change and with a
small suggestion from Greg Marr).
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
> > I'm not too conerned with the performance differences. The two patches
> > seem to have about the same performance when the code is working. Bugs we
> > can fix after the commit. The question is really the design of the two
> > patches. Either we use a filter, or we use a brigade API.
>
> all i can really offer at the moment is feedback on performance and
> functionality. i know just enough about filters to use the api, not
> enough to comment on the design of each patch. i'm too clueless to debug
> greg's patch, but i could write a C version of the Perl Apache::rwrite
> exercise if that helps at all.
BTW, I just fixed a performance problem in my patch. You were calling
ap_rwrite, which was calling ap_brigade_puts under the covers. That means
that although you were passing in a length, I was calling strlen. That's
fixed on my machine now. I have always realized that there were some
performance problems with my patch, but I just wanted to put the idea out
there, before I spend time making it better.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
On Sun, 21 Jan 2001 rbb@covalent.net wrote:
> I'm not too conerned with the performance differences. The two patches
> seem to have about the same performance when the code is working. Bugs we
> can fix after the commit. The question is really the design of the two
> patches. Either we use a filter, or we use a brigade API.
all i can really offer at the moment is feedback on performance and
functionality. i know just enough about filters to use the api, not
enough to comment on the design of each patch. i'm too clueless to debug
greg's patch, but i could write a C version of the Perl Apache::rwrite
exercise if that helps at all.
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
> the results show greg's faster, however, that is due to a bug, missing
> lots of data (notice document length and transfered bytes). the output
> with rbb.new patch is exactly the same if i run the output loop(s) outside
> of apache.
I'm not too conerned with the performance differences. The two patches
seem to have about the same performance when the code is working. Bugs we
can fix after the commit. The question is really the design of the two
patches. Either we use a filter, or we use a brigade API.
> i also just realized i'm not calling ap_rflush() at all, but i don't think
> that is required for this test case, is it?
ap_rflush shouldn't be required at all, in any case.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
here's a better test, that ends up spitting out:
[2bytes] aa
[4bytes] aaaa
[8bytes] aaaaaaaa
[16bytes] aaaaaaaaaaaaaaaa
[32bytes] aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
and so on up to 8192 for each character a..Z :
package Apache::rwrite;
use strict;
sub handler {
my $r = shift;
$r->send_http_header;
for my $c ('a'..'Z') {
my $n = 2;
while ($n <= 8192) {
$r->write("[", $n, "bytes", "] ", $c x $n);
$r->write("\n");
$n *= 2;
}
}
return Apache::OK;
}
1;
"[", $n, "bytes", "] " could of course be written as "[${n}bytes] ", but i
wanted to sprinkle in some smaller pieces of data.
$r->write calls ap_rwrite() underneath for each argument, so 5 times for
the first call, once for the second.
the results show greg's faster, however, that is due to a bug, missing
lots of data (notice document length and transfered bytes). the output
with rbb.new patch is exactly the same if i run the output loop(s) outside
of apache.
i also just realized i'm not calling ap_rflush() at all, but i don't think
that is required for this test case, is it?
% ab -c 5 -n 1000 http://localhost:8082/rwrite
with rbb.new patch:
Server Software: Apache/2.0b1-dev
Server Hostname: localhost
Server Port: 8082
Document Path: /rwrite
Document Length: 429858 bytes
Concurrency Level: 5
Time taken for tests: 28.132 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 430585672 bytes
HTML transferred: 430398298 bytes
Requests per second: 35.55
Transfer rate: 15305.90 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 5 89
Processing: 80 134 853
Total: 80 139 942
with greg.new patch:
Server Software: Apache/2.0b1-dev
Server Hostname: localhost
Server Port: 8082
Document Path: /rwrite
Document Length: 302418 bytes
Concurrency Level: 5
Time taken for tests: 24.402 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 303423701 bytes
HTML transferred: 303236140 bytes
Requests per second: 40.98
Transfer rate: 12434.38 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 7 93
Processing: 37 113 1246
Total: 37 120 1339
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
On Sun, 21 Jan 2001 rbb@covalent.net wrote:
>
> Dependancies are horribly broken, because they aren't in CVS. You must
> run make depend before doing any work if you want dependancies to
> work. :-)
ah ha, ok.
i did some testing with:
% ab -c 10 -n 1000 http://localhost:8082/osc/partIII/index.html
that is a slide (from apachecon) generated by a modperl handler.
'no patch' uses mod_perl's own buffer thingy. the other 2 use ap_rwrite
with the patches. looks pretty close, but a small amount of data is
written and ap_write is call just a few times. i'm working on another
test to really hammer on it.
no patch:
Server Software: Apache/2.0b1-dev
Server Hostname: localhost
Server Port: 8082
Document Path: /osc/partIII/index.html
Document Length: 1604 bytes
Concurrency Level: 10
Time taken for tests: 28.882 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 1812000 bytes
HTML transferred: 1604000 bytes
Requests per second: 34.62
Transfer rate: 62.74 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 0 4
Processing: 5 287 315
Total: 5 287 319
rbb.new patch:
Server Software: Apache/2.0b1-dev
Server Hostname: localhost
Server Port: 8082
Document Path: /osc/partIII/index.html
Document Length: 1604 bytes
Concurrency Level: 10
Time taken for tests: 29.123 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 1812000 bytes
HTML transferred: 1604000 bytes
Requests per second: 34.34
Transfer rate: 62.22 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 0 4
Processing: 6 289 382
Total: 6 289 386
greg.new patch:
Server Software: Apache/2.0b1-dev
Server Hostname: localhost
Server Port: 8082
Document Path: /osc/partIII/index.html
Document Length: 1604 bytes
Concurrency Level: 10
Time taken for tests: 30.066 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 1812000 bytes
HTML transferred: 1604000 bytes
Requests per second: 33.26
Transfer rate: 60.27 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 0 4
Processing: 6 298 594
Total: 6 298 598
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
Dependancies are horribly broken, because they aren't in CVS. You must
run make depend before doing any work if you want dependancies to
work. :-)
Ryan
On Sun, 21 Jan 2001, Doug MacEachern wrote:
> On Sun, 21 Jan 2001 rbb@covalent.net wrote:
>
> > Uh, Head has been running on locus since late last night without any
> > problems at all. Could you try doing a make clean;make
>
> hmm, looks ok now. either dependicies are broken or i'm 'smoking grass'
> greg's patch is working now too.
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
On Sun, 21 Jan 2001 rbb@covalent.net wrote:
> Uh, Head has been running on locus since late last night without any
> problems at all. Could you try doing a make clean;make
hmm, looks ok now. either dependicies are broken or i'm 'smoking grass'
greg's patch is working now too.
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
> > Euh... that isn't my patch. I don't do anything with check_pipeline_flush().
> >
> > Ryan checked in a change to check_pipeline_flush() a day or two ago, but
> > that shouldn't cause the behavior you're seeing with or without that
> > checkin.
>
> you're right, this happens with no patches applied. i did a cvs up this
> morning and applied rbb's patch without testing HEAD first. strange thing
> is, i re-apply rbb's patch and get a different core dump.
> in any case, looks like HEAD is hosed.
Uh, Head has been running on locus since late last night without any
problems at all. Could you try doing a make clean;make
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
On Sun, 21 Jan 2001, Greg Stein wrote:
> Euh... that isn't my patch. I don't do anything with check_pipeline_flush().
>
> Ryan checked in a change to check_pipeline_flush() a day or two ago, but
> that shouldn't cause the behavior you're seeing with or without that
> checkin.
you're right, this happens with no patches applied. i did a cvs up this
morning and applied rbb's patch without testing HEAD first. strange thing
is, i re-apply rbb's patch and get a different core dump.
in any case, looks like HEAD is hosed.
Re: Both ap_r* patches.
Posted by Greg Stein <gs...@lyra.org>.
Euh... that isn't my patch. I don't do anything with check_pipeline_flush().
Ryan checked in a change to check_pipeline_flush() a day or two ago, but
that shouldn't cause the behavior you're seeing with or without that
checkin.
Do a cvs diff on http_request.c -- you should have no changes from my patch
or from Ryan's patch.
[ I'll post my "canonical" patch in a moment ]
Cheers,
-g
On Sun, Jan 21, 2001 at 11:50:33AM -0800, Doug MacEachern wrote:
> using greg's patch, all requests with or without calling ap_r* result in:
>
> 0x806f4b6 in check_pipeline_flush (r=0x84d6d24) at http_request.c:1371
> 1371 APR_BRIGADE_INSERT_HEAD(bb, e);
> (gdb) where
> #0 0x806f4b6 in check_pipeline_flush (r=0x84d6d24) at http_request.c:1371
> #1 0x806f4ff in ap_process_request (r=0x84d6d24) at http_request.c:1387
> #2 0x808ee2e in ap_process_http_connection (c=0x812d62c) at
> connection.c:249
> #3 0x808eb2b in ap_run_process_connection (c=0x812d62c) at
> connection.c:85
> #4 0x808ed65 in ap_process_connection (c=0x812d62c) at connection.c:221
> #5 0x808001b in process_socket (p=0x812d54c, sock=0x812d57c,
> my_child_num=0,
> my_thread_num=0) at mpmt_pthread.c:417
> #6 0x8080400 in worker_thread (dummy=0x84aa6b0) at mpmt_pthread.c:549
> #7 0x4008ddd0 in pthread_start_thread_event (arg=0xbf7ffc00) at
> manager.c:274
>
> (gdb) p *bb
> $2 = {p = 0x84d6cf4, start = 0x84db298 "\230²M\b\230²M\b",
> end = 0x84db298 "\230²M\b\230²M\b", list = {next = 0x0, prev =
> 0x84db294}}
> (gdb) p *e
> $3 = {link = {next = 0x0, prev = 0x84db2a0}, type = 0x80d6b88, length = 0,
> data = 0x0}
> (gdb) p *e->type
> $4 = {name = 0x80d6ba4 "FLUSH", num_func = 5,
> destroy = 0x8099b14 <apr_bucket_destroy_notimpl>,
> read = 0x809913c <flush_read>,
> setaside = 0x8099af0 <apr_bucket_setaside_notimpl>,
> split = 0x8099afc <apr_bucket_split_notimpl>, copy = 0x8099158
> <flush_copy>}
>
--
Greg Stein, http://www.lyra.org/
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
On Sun, 21 Jan 2001 rbb@covalent.net wrote:
> On Sun, 21 Jan 2001 rbb@covalent.net wrote:
>
> >
> > > patch below fixes for me, is this the right thing todo?
> >
> > Yep.
> >
> > Ryan
Funny thing is, I went to add this to my tree, and this patch is already
in my tree. Must have added it after I created the patches. Sorry about
that.
Ryan
> > > --- modules/http/http_protocol.c~ Sun Jan 21 10:39:42 2001
> > > +++ modules/http/http_protocol.c Sun Jan 21 11:16:03 2001
> > > @@ -1621,7 +1621,13 @@
> > > {
> > > apr_bucket *b;
> > >
> > > - apr_brigade_flush(r->bb);
> > > + if (r->bb) {
> > > + apr_brigade_flush(r->bb);
> > > + }
> > > + else {
> > > + r->bb = apr_brigade_create(r->pool);
> > > + }
> > > +
> > > b = apr_bucket_create_eos();
> > > APR_BRIGADE_INSERT_TAIL(r->bb, b);
> > > ap_pass_brigade(r->output_filters, r->bb);
> > >
> > >
> >
> >
> > _______________________________________________________________________________
> > Ryan Bloom rbb@apache.org
> > 406 29th St.
> > San Francisco, CA 94131
> > -------------------------------------------------------------------------------
> >
> >
>
>
> _______________________________________________________________________________
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> -------------------------------------------------------------------------------
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
On Sun, 21 Jan 2001 rbb@covalent.net wrote:
>
> > patch below fixes for me, is this the right thing todo?
>
> Yep.
>
> Ryan
BTW, the other option is to change all handlers to use r->bb for their
bucket brigade, whether they use ap_r* or not. Either is perfectly valid,
this patch is MUCH safer however.
Ryan
>
> >
> > --- modules/http/http_protocol.c~ Sun Jan 21 10:39:42 2001
> > +++ modules/http/http_protocol.c Sun Jan 21 11:16:03 2001
> > @@ -1621,7 +1621,13 @@
> > {
> > apr_bucket *b;
> >
> > - apr_brigade_flush(r->bb);
> > + if (r->bb) {
> > + apr_brigade_flush(r->bb);
> > + }
> > + else {
> > + r->bb = apr_brigade_create(r->pool);
> > + }
> > +
> > b = apr_bucket_create_eos();
> > APR_BRIGADE_INSERT_TAIL(r->bb, b);
> > ap_pass_brigade(r->output_filters, r->bb);
> >
> >
>
>
> _______________________________________________________________________________
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> -------------------------------------------------------------------------------
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by rb...@covalent.net.
> patch below fixes for me, is this the right thing todo?
Yep.
Ryan
>
> --- modules/http/http_protocol.c~ Sun Jan 21 10:39:42 2001
> +++ modules/http/http_protocol.c Sun Jan 21 11:16:03 2001
> @@ -1621,7 +1621,13 @@
> {
> apr_bucket *b;
>
> - apr_brigade_flush(r->bb);
> + if (r->bb) {
> + apr_brigade_flush(r->bb);
> + }
> + else {
> + r->bb = apr_brigade_create(r->pool);
> + }
> +
> b = apr_bucket_create_eos();
> APR_BRIGADE_INSERT_TAIL(r->bb, b);
> ap_pass_brigade(r->output_filters, r->bb);
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
rbb.new patch results in a core dump for any request that does not use
ap_r* :
0x809a318 in apr_brigade_flush (b=0x0) at apr_brigade.c:194
194 apr_size_t i = b->end - b->start;
(gdb) where
#0 0x809a318 in apr_brigade_flush (b=0x0) at apr_brigade.c:194
#1 0x8068eb7 in end_output_stream (r=0x84d6cc4) at http_protocol.c:1624
#2 0x806ac77 in ap_finalize_request_protocol (r=0x84d6cc4)
at http_protocol.c:2682
#3 0x806f267 in process_request_internal (r=0x84d6cc4) at
http_request.c:1356
#4 0x806f321 in ap_process_request (r=0x84d6cc4) at http_request.c:1378
#5 0x808ec5e in ap_process_http_connection (c=0x812d5cc) at
connection.c:249
#6 0x808e95b in ap_run_process_connection (c=0x812d5cc) at
connection.c:85
#7 0x808eb95 in ap_process_connection (c=0x812d5cc) at connection.c:221
#8 0x807fe4b in process_socket (p=0x812d4ec, sock=0x812d51c,
my_child_num=0,
my_thread_num=0) at mpmt_pthread.c:417
#9 0x8080230 in worker_thread (dummy=0x84aa650) at mpmt_pthread.c:549
#10 0x4008ddd0 in pthread_start_thread_event (arg=0xbf7ffc00) at
manager.c:274
patch below fixes for me, is this the right thing todo?
--- modules/http/http_protocol.c~ Sun Jan 21 10:39:42 2001
+++ modules/http/http_protocol.c Sun Jan 21 11:16:03 2001
@@ -1621,7 +1621,13 @@
{
apr_bucket *b;
- apr_brigade_flush(r->bb);
+ if (r->bb) {
+ apr_brigade_flush(r->bb);
+ }
+ else {
+ r->bb = apr_brigade_create(r->pool);
+ }
+
b = apr_bucket_create_eos();
APR_BRIGADE_INSERT_TAIL(r->bb, b);
ap_pass_brigade(r->output_filters, r->bb);
Re: Both ap_r* patches.
Posted by Doug MacEachern <do...@covalent.net>.
using greg's patch, all requests with or without calling ap_r* result in:
0x806f4b6 in check_pipeline_flush (r=0x84d6d24) at http_request.c:1371
1371 APR_BRIGADE_INSERT_HEAD(bb, e);
(gdb) where
#0 0x806f4b6 in check_pipeline_flush (r=0x84d6d24) at http_request.c:1371
#1 0x806f4ff in ap_process_request (r=0x84d6d24) at http_request.c:1387
#2 0x808ee2e in ap_process_http_connection (c=0x812d62c) at
connection.c:249
#3 0x808eb2b in ap_run_process_connection (c=0x812d62c) at
connection.c:85
#4 0x808ed65 in ap_process_connection (c=0x812d62c) at connection.c:221
#5 0x808001b in process_socket (p=0x812d54c, sock=0x812d57c,
my_child_num=0,
my_thread_num=0) at mpmt_pthread.c:417
#6 0x8080400 in worker_thread (dummy=0x84aa6b0) at mpmt_pthread.c:549
#7 0x4008ddd0 in pthread_start_thread_event (arg=0xbf7ffc00) at
manager.c:274
(gdb) p *bb
$2 = {p = 0x84d6cf4, start = 0x84db298 "\230²M\b\230²M\b",
end = 0x84db298 "\230²M\b\230²M\b", list = {next = 0x0, prev =
0x84db294}}
(gdb) p *e
$3 = {link = {next = 0x0, prev = 0x84db2a0}, type = 0x80d6b88, length = 0,
data = 0x0}
(gdb) p *e->type
$4 = {name = 0x80d6ba4 "FLUSH", num_func = 5,
destroy = 0x8099b14 <apr_bucket_destroy_notimpl>,
read = 0x809913c <flush_read>,
setaside = 0x8099af0 <apr_bucket_setaside_notimpl>,
split = 0x8099afc <apr_bucket_split_notimpl>, copy = 0x8099158
<flush_copy>}