You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by rb...@covalent.net on 2001/03/04 18:49:48 UTC
Re: apache 2.0.11 - tag 2.0.12?
On Sat, 24 Feb 2001, Brian Havard wrote:
> On Fri, 23 Feb 2001 14:21:22 -0600, William A. Rowe, Jr. wrote:
>
> >The split of mod_proxy effectively broke the 2_0_11 release on Win32. The
> >fix came very soon thereafter.
> >
> >I'd like to see us tag 2_0_12 just as soon as Brian indicates that HEAD
> >builds on OS2 after all the changes this week. I'm seeing no issues building
> >on Win32, and presume we are putting to bed the Unix headaches from all the
> >posts on that subject.
>
> Yeah, HEAD builds on OS/2 ok though the mod_include >64k bug still exists.
> Ryan, were you able to reproduce this?
Where are we with this? I can't reproduce it with any size file. I even
went so far as to create a 10Meg file to try testing it. Can you test
again with HEAD? I believe that Paul's patch to send after 9k os data is
processed may fix this problem.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: apache 2.0.11 - tag 2.0.12?
Posted by Cliff Woolley <cl...@yahoo.com>.
On Sun, 4 Mar 2001, Greg Stein wrote:
> > Where are we with this? I can't reproduce it with any size file. I even
> > went so far as to create a 10Meg file to try testing it. Can you test
> > again with HEAD? I believe that Paul's patch to send after 9k os data is
> > processed may fix this problem.
>
> Note that it may have just covered it up. The bug could still be lurking in
> there somewhere.
I had the same reaction and mentioned as much sideband to Ryan. He and I
both agree that we're fairly confident it's a mod_include-specific issue,
though we have no way to prove it. I spent quite some time today going
through the mod_include code trying to grok it (still not 100% there
:-)... at any rate, if it works now, it works... but for my own peace of
mind I'd still like to know what cause(s|d) the problem.
--Cliff
--------------------------------------------------------------
Cliff Woolley
cliffwoolley@yahoo.com
Charlottesville, VA
Re: apache 2.0.11 - tag 2.0.12?
Posted by Greg Stein <gs...@lyra.org>.
On Sun, Mar 04, 2001 at 09:49:48AM -0800, rbb@covalent.net wrote:
> On Sat, 24 Feb 2001, Brian Havard wrote:
>
> > On Fri, 23 Feb 2001 14:21:22 -0600, William A. Rowe, Jr. wrote:
> >
> > >The split of mod_proxy effectively broke the 2_0_11 release on Win32. The
> > >fix came very soon thereafter.
> > >
> > >I'd like to see us tag 2_0_12 just as soon as Brian indicates that HEAD
> > >builds on OS2 after all the changes this week. I'm seeing no issues building
> > >on Win32, and presume we are putting to bed the Unix headaches from all the
> > >posts on that subject.
> >
> > Yeah, HEAD builds on OS/2 ok though the mod_include >64k bug still exists.
> > Ryan, were you able to reproduce this?
>
> Where are we with this? I can't reproduce it with any size file. I even
> went so far as to create a 10Meg file to try testing it. Can you test
> again with HEAD? I believe that Paul's patch to send after 9k os data is
> processed may fix this problem.
Note that it may have just covered it up. The bug could still be lurking in
there somewhere.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: apache 2.0.11 - tag 2.0.12?
Posted by rb...@covalent.net.
On Mon, 5 Mar 2001, Brian Havard wrote:
> On Sun, 4 Mar 2001 09:49:48 -0800 (PST), rbb@covalent.net wrote:
>
> >On Sat, 24 Feb 2001, Brian Havard wrote:
> >
> >> On Fri, 23 Feb 2001 14:21:22 -0600, William A. Rowe, Jr. wrote:
> >>
> >> >The split of mod_proxy effectively broke the 2_0_11 release on Win32. The
> >> >fix came very soon thereafter.
> >> >
> >> >I'd like to see us tag 2_0_12 just as soon as Brian indicates that HEAD
> >> >builds on OS2 after all the changes this week. I'm seeing no issues building
> >> >on Win32, and presume we are putting to bed the Unix headaches from all the
> >> >posts on that subject.
> >>
> >> Yeah, HEAD builds on OS/2 ok though the mod_include >64k bug still exists.
> >> Ryan, were you able to reproduce this?
> >
> >Where are we with this? I can't reproduce it with any size file. I even
> >went so far as to create a 10Meg file to try testing it. Can you test
> >again with HEAD? I believe that Paul's patch to send after 9k os data is
> >processed may fix this problem.
>
> I thought so too but no, it's still there. However I just noticed that it
> works ok if the request is for HTTP/1.0. It only breaks on HTTP/1.1 so I'm
> thinking perhaps the chunk filter or something it uses (like
> apr_brigade_split) is broken. I'll try to find some time to step through
> the code & figure it out.
Ahhhhh..... I only tried with 1.0 requests. Could this be a part of the
pipelined request stuff? I may try to reproduce this on my computer
later, but I'm trying to finish moving code out of http.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: apache 2.0.11 - tag 2.0.12?
Posted by Brian Havard <br...@kheldar.apana.org.au>.
On Mon, 5 Mar 2001 07:49:13 -0800 (PST), rbb@covalent.net wrote:
>On Tue, 6 Mar 2001, Brian Havard wrote:
>
>> On Mon, 5 Mar 2001 06:56:37 -0800 (PST), rbb@covalent.net wrote:
>>
>> >> No need, I've think I've got to the bottom of it now. It appears that the
>> >> OS/2 implementation of writev() chokes if the total bytes > 64k. It returns
>> >> EINVAL but server/core.c:3189 says /* XXX: log the error */ rather than
>> >> actually logging the error :-(
>> >
>> >Cool.
>> >
>> >> So what's a reliable way to ensure no more than 64k gets writev()'d? I
>> >> guess I should break it up appropriately in apr_sendv() as it's most likely
>> >> an OS/2 only problem.
>> >
>> >Could you do it in core_output_filter? We should NEVER be writing that
>> >much one time anyway, so putting it in the core should help us see it,
>> >along with a debug message about writing too much?
>>
>> Well I noticed that the content length filter set aside ALL of
>> mod_include's output. I guess that's why core_output_filter got hit with it
>> all at once.
>
>That's bogus. The content length filter should almost NEVER set aside
>that much data. There is a bug there.
Well yeah, it should be chunking, it was an HTTP/1.1 request. That's what
1.3 does.
--
______________________________________________________________________________
| Brian Havard | "He is not the messiah! |
| brianh@kheldar.apana.org.au | He's a very naughty boy!" - Life of Brian |
------------------------------------------------------------------------------
Re: apache 2.0.11 - tag 2.0.12?
Posted by rb...@covalent.net.
On Tue, 6 Mar 2001, Brian Havard wrote:
> On Mon, 5 Mar 2001 06:56:37 -0800 (PST), rbb@covalent.net wrote:
>
> >> No need, I've think I've got to the bottom of it now. It appears that the
> >> OS/2 implementation of writev() chokes if the total bytes > 64k. It returns
> >> EINVAL but server/core.c:3189 says /* XXX: log the error */ rather than
> >> actually logging the error :-(
> >
> >Cool.
> >
> >> So what's a reliable way to ensure no more than 64k gets writev()'d? I
> >> guess I should break it up appropriately in apr_sendv() as it's most likely
> >> an OS/2 only problem.
> >
> >Could you do it in core_output_filter? We should NEVER be writing that
> >much one time anyway, so putting it in the core should help us see it,
> >along with a debug message about writing too much?
>
> Well I noticed that the content length filter set aside ALL of
> mod_include's output. I guess that's why core_output_filter got hit with it
> all at once.
That's bogus. The content length filter should almost NEVER set aside
that much data. There is a bug there.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: apache 2.0.11 - tag 2.0.12?
Posted by Brian Havard <br...@kheldar.apana.org.au>.
On Mon, 5 Mar 2001 06:56:37 -0800 (PST), rbb@covalent.net wrote:
>> No need, I've think I've got to the bottom of it now. It appears that the
>> OS/2 implementation of writev() chokes if the total bytes > 64k. It returns
>> EINVAL but server/core.c:3189 says /* XXX: log the error */ rather than
>> actually logging the error :-(
>
>Cool.
>
>> So what's a reliable way to ensure no more than 64k gets writev()'d? I
>> guess I should break it up appropriately in apr_sendv() as it's most likely
>> an OS/2 only problem.
>
>Could you do it in core_output_filter? We should NEVER be writing that
>much one time anyway, so putting it in the core should help us see it,
>along with a debug message about writing too much?
Well I noticed that the content length filter set aside ALL of
mod_include's output. I guess that's why core_output_filter got hit with it
all at once.
--
______________________________________________________________________________
| Brian Havard | "He is not the messiah! |
| brianh@kheldar.apana.org.au | He's a very naughty boy!" - Life of Brian |
------------------------------------------------------------------------------
Re: apache 2.0.11 - tag 2.0.12?
Posted by Cliff Woolley <cl...@yahoo.com>.
On Mon, 5 Mar 2001 rbb@covalent.net wrote:
> We need to standardize on either 9000 or 8192. I prefer 8192, but I know
> others prefer 9000. Let's just pick one and change everything.
I already changed the brigade buffering code to use 8192. AFAIK, all of
the actual buckets code is standardized on 8192. There could be other
places in Apache that are still using 9000, of course.
+1 for standardizing everything else on 8192 as well.
--Cliff
--------------------------------------------------------------
Cliff Woolley
cliffwoolley@yahoo.com
Charlottesville, VA
Re: apache 2.0.11 - tag 2.0.12?
Posted by Cliff Woolley <cl...@yahoo.com>.
On Mon, 5 Mar 2001 rbb@covalent.net wrote:
> We need to standardize on either 9000 or 8192. I prefer 8192, but I know
> others prefer 9000. Let's just pick one and change everything.
I already changed the brigade buffering code to use 8192. AFAIK, all of
the actual buckets code is standardized on 8192. There could be other
places in Apache that are still using 9000, of course.
+1 for standardizing everything else on 8192 as well.
--Cliff
--------------------------------------------------------------
Cliff Woolley
cliffwoolley@yahoo.com
Charlottesville, VA
Re: apache 2.0.11 - tag 2.0.12?
Posted by rb...@covalent.net.
> No need, I've think I've got to the bottom of it now. It appears that the
> OS/2 implementation of writev() chokes if the total bytes > 64k. It returns
> EINVAL but server/core.c:3189 says /* XXX: log the error */ rather than
> actually logging the error :-(
Cool.
> So what's a reliable way to ensure no more than 64k gets writev()'d? I
> guess I should break it up appropriately in apr_sendv() as it's most likely
> an OS/2 only problem.
Could you do it in core_output_filter? We should NEVER be writing that
much one time anyway, so putting it in the core should help us see it,
along with a debug message about writing too much?
> I also noticed some odd behaviour, the iovec looks like:
>
> $65 = {{iov_base = 0x16c340 "", iov_len = 284}, {iov_base = 0x152c20 "",
> iov_len = 8192}, {iov_base = 0x154c80 "8", ' ' <repeats 199 times>...,
> iov_len = 808}, {iov_base = 0x154fa8 ' ' <repeats 200 times>...,
> iov_len = 7384}, {iov_base = 0x156d20 "16", ' ' <repeats 198 times>...,
> iov_len = 1616}, {iov_base = 0x157370 ' ' <repeats 200 times>...,
> iov_len = 6576}, {iov_base = 0x158dc0 "24", ' ' <repeats 198 times>...,
> iov_len = 2424}, {iov_base = 0x159738 ' ' <repeats 200 times>...,
> iov_len = 5768}, {iov_base = 0x15ae60 "32", ' ' <repeats 198 times>...,
> iov_len = 3232}, {iov_base = 0x15bb00 ' ' <repeats 200 times>...,
> iov_len = 4960}, {iov_base = 0x15cf00 "40", ' ' <repeats 198 times>...,
> iov_len = 4040}, {
> iov_base = 0x15dec8 ' ' <repeats 55 times>, "\n44", ' ' <repeats 142
> times>..., iov_len = 4152}, {iov_base = 0x15efa0 "48", ' ' <repeats 198
> times>...,
> iov_len = 4848}, {iov_base = 0x160290 ' ' <repeats 200 times>...,
> iov_len = 3344}, {iov_base = 0x161040 "56", ' ' <repeats 198 times>...,
> iov_len = 5656}, {iov_base = 0x162658 ' ' <repeats 200 times>...,
> iov_len = 2536}}
>
> It looks like an interaction going on between a 9000 byte limit & an 8192
> byte limit. Looks far from optimal to me.....
We need to standardize on either 9000 or 8192. I prefer 8192, but I know
others prefer 9000. Let's just pick one and change everything.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: apache 2.0.11 - tag 2.0.12?
Posted by Brian Havard <br...@kheldar.apana.org.au>.
On Sun, 4 Mar 2001 22:23:27 -0500 (EST), Cliff Woolley wrote:
>On Mon, 5 Mar 2001, Brian Havard wrote:
>
>> it dies when the total output exceeds 64k, header+body, not when the
>> requested file exceeds 64k. Subtle difference but may point to a
>> completely different cause.
>
>Yes, I'd missed that. And yes, you're probably right that it points in a
>different direction. Actually, that makes me feel better in a way,
>because it somewhat validates my theory that file_read() is not
>responsible.
>
>Anyway, I'm getting set up to try to duplicate the problem. Will keep you
>all posted.
No need, I've think I've got to the bottom of it now. It appears that the
OS/2 implementation of writev() chokes if the total bytes > 64k. It returns
EINVAL but server/core.c:3189 says /* XXX: log the error */ rather than
actually logging the error :-(
So what's a reliable way to ensure no more than 64k gets writev()'d? I
guess I should break it up appropriately in apr_sendv() as it's most likely
an OS/2 only problem.
I also noticed some odd behaviour, the iovec looks like:
$65 = {{iov_base = 0x16c340 "", iov_len = 284}, {iov_base = 0x152c20 "",
iov_len = 8192}, {iov_base = 0x154c80 "8", ' ' <repeats 199 times>...,
iov_len = 808}, {iov_base = 0x154fa8 ' ' <repeats 200 times>...,
iov_len = 7384}, {iov_base = 0x156d20 "16", ' ' <repeats 198 times>...,
iov_len = 1616}, {iov_base = 0x157370 ' ' <repeats 200 times>...,
iov_len = 6576}, {iov_base = 0x158dc0 "24", ' ' <repeats 198 times>...,
iov_len = 2424}, {iov_base = 0x159738 ' ' <repeats 200 times>...,
iov_len = 5768}, {iov_base = 0x15ae60 "32", ' ' <repeats 198 times>...,
iov_len = 3232}, {iov_base = 0x15bb00 ' ' <repeats 200 times>...,
iov_len = 4960}, {iov_base = 0x15cf00 "40", ' ' <repeats 198 times>...,
iov_len = 4040}, {
iov_base = 0x15dec8 ' ' <repeats 55 times>, "\n44", ' ' <repeats 142
times>..., iov_len = 4152}, {iov_base = 0x15efa0 "48", ' ' <repeats 198
times>...,
iov_len = 4848}, {iov_base = 0x160290 ' ' <repeats 200 times>...,
iov_len = 3344}, {iov_base = 0x161040 "56", ' ' <repeats 198 times>...,
iov_len = 5656}, {iov_base = 0x162658 ' ' <repeats 200 times>...,
iov_len = 2536}}
It looks like an interaction going on between a 9000 byte limit & an 8192
byte limit. Looks far from optimal to me.....
--
______________________________________________________________________________
| Brian Havard | "He is not the messiah! |
| brianh@kheldar.apana.org.au | He's a very naughty boy!" - Life of Brian |
------------------------------------------------------------------------------
Re: apache 2.0.11 - tag 2.0.12?
Posted by Cliff Woolley <cl...@yahoo.com>.
On Mon, 5 Mar 2001, Brian Havard wrote:
> it dies when the total output exceeds 64k, header+body, not when the
> requested file exceeds 64k. Subtle difference but may point to a
> completely different cause.
Yes, I'd missed that. And yes, you're probably right that it points in a
different direction. Actually, that makes me feel better in a way,
because it somewhat validates my theory that file_read() is not
responsible.
Anyway, I'm getting set up to try to duplicate the problem. Will keep you
all posted.
--Cliff
--------------------------------------------------------------
Cliff Woolley
cliffwoolley@yahoo.com
Charlottesville, VA
Re: apache 2.0.11 - tag 2.0.12?
Posted by David Reid <dr...@jetnet.co.uk>.
FWIW, peter Moore has just sent me more detail on the problem that BeOS
seems to have with large files, and it looks like 1.1 has a problem whereas
1.0 is fine. Could be related to Brian's issue. I've asked him to follow
up with more detail, but his email is peter@cdws.com.au if you want to talk
directly.
The file that's barfing is the large image on
http://bp6.cdws.com.au/bp6cpu.html
(http://bp6.cdws.com.au/images/screen1.tga)
Of course, given that both peter and Brian are in Australia, maybe it's just
a southern hemisphere thing :)
david
>
> Can you give me a sample set of request headers to try to duplicate this?
>
> I've tried it with a 128KB file with MMAP and SENDFILE both turned off
> using HTTP/1.1 with and without chunking, and it still works on Linux.
>
Re: apache 2.0.11 - tag 2.0.12?
Posted by Cliff Woolley <cl...@yahoo.com>.
Can you give me a sample set of request headers to try to duplicate this?
I've tried it with a 128KB file with MMAP and SENDFILE both turned off
using HTTP/1.1 with and without chunking, and it still works on Linux.
Thanks,
Cliff
--------------------------------------------------------------
Cliff Woolley
cliffwoolley@yahoo.com
Charlottesville, VA
Re: apache 2.0.11 - tag 2.0.12?
Posted by Brian Havard <br...@kheldar.apana.org.au>.
On Sun, 4 Mar 2001 20:30:56 -0500 (EST), Cliff Woolley wrote:
>On Mon, 5 Mar 2001, Brian Havard wrote:
>
>> I thought so too but no, it's still there. However I just noticed that it
>> works ok if the request is for HTTP/1.0. It only breaks on HTTP/1.1 so I'm
>> thinking perhaps the chunk filter or something it uses (like
>> apr_brigade_split) is broken. I'll try to find some time to step through
>> the code & figure it out.
>
>apr_brigade_split() is pretty simple... but I wouldn't be surprised if the
>chunk filter has an issue hiding somewhere.
>
>I worked a week or so ago on cleaning up the logic in the chunk filter so
>that I could understand it myself (and to get rid of some unnecessary
>layers of logic), but I didn't get a chance to finish and check it in
>before other things got in the way. I'll go fish around and try to find
>that copy of my tree and get back to it. =-)
>
>Anyway, at least we now have something to go on... the case where it fails
>is non-MMAP systems on HTTP/1.1. The key part of the non-MMAP case is
>that file_read() ends up reading in the file into many 8KB heap buckets.
>If the thing fails on files >64KB, that's >8 calls to file_read(),
>obviously.
>
>We're getting close, I can smell it. ;-]
You may be on the wrong track, it dies when the total output exceeds 64k,
header+body, not when the requested file exceeds 64k. Subtle difference but
may point to a completely different cause.
--
______________________________________________________________________________
| Brian Havard | "He is not the messiah! |
| brianh@kheldar.apana.org.au | He's a very naughty boy!" - Life of Brian |
------------------------------------------------------------------------------
Re: apache 2.0.11 - tag 2.0.12?
Posted by Cliff Woolley <cl...@yahoo.com>.
On Sun, 4 Mar 2001, Cliff Woolley wrote:
> I worked a week or so ago on cleaning up the logic in the chunk filter so
Oh feh, so much for my memory. =-) It wasn't the chunk filter I was
thinking of at all. It was ap_http_filter() that I was messing with.
Back to the drawing board.
I'll try to duplicate this problem tonight and go from there.
--Cliff
--------------------------------------------------------------
Cliff Woolley
cliffwoolley@yahoo.com
Charlottesville, VA
Re: apache 2.0.11 - tag 2.0.12?
Posted by Cliff Woolley <cl...@yahoo.com>.
On Mon, 5 Mar 2001, Brian Havard wrote:
> I thought so too but no, it's still there. However I just noticed that it
> works ok if the request is for HTTP/1.0. It only breaks on HTTP/1.1 so I'm
> thinking perhaps the chunk filter or something it uses (like
> apr_brigade_split) is broken. I'll try to find some time to step through
> the code & figure it out.
apr_brigade_split() is pretty simple... but I wouldn't be surprised if the
chunk filter has an issue hiding somewhere.
I worked a week or so ago on cleaning up the logic in the chunk filter so
that I could understand it myself (and to get rid of some unnecessary
layers of logic), but I didn't get a chance to finish and check it in
before other things got in the way. I'll go fish around and try to find
that copy of my tree and get back to it. =-)
Anyway, at least we now have something to go on... the case where it fails
is non-MMAP systems on HTTP/1.1. The key part of the non-MMAP case is
that file_read() ends up reading in the file into many 8KB heap buckets.
If the thing fails on files >64KB, that's >8 calls to file_read(),
obviously.
We're getting close, I can smell it. ;-]
--Cliff
--------------------------------------------------------------
Cliff Woolley
cliffwoolley@yahoo.com
Charlottesville, VA
Re: apache 2.0.11 - tag 2.0.12?
Posted by Brian Havard <br...@kheldar.apana.org.au>.
On Sun, 4 Mar 2001 09:49:48 -0800 (PST), rbb@covalent.net wrote:
>On Sat, 24 Feb 2001, Brian Havard wrote:
>
>> On Fri, 23 Feb 2001 14:21:22 -0600, William A. Rowe, Jr. wrote:
>>
>> >The split of mod_proxy effectively broke the 2_0_11 release on Win32. The
>> >fix came very soon thereafter.
>> >
>> >I'd like to see us tag 2_0_12 just as soon as Brian indicates that HEAD
>> >builds on OS2 after all the changes this week. I'm seeing no issues building
>> >on Win32, and presume we are putting to bed the Unix headaches from all the
>> >posts on that subject.
>>
>> Yeah, HEAD builds on OS/2 ok though the mod_include >64k bug still exists.
>> Ryan, were you able to reproduce this?
>
>Where are we with this? I can't reproduce it with any size file. I even
>went so far as to create a 10Meg file to try testing it. Can you test
>again with HEAD? I believe that Paul's patch to send after 9k os data is
>processed may fix this problem.
I thought so too but no, it's still there. However I just noticed that it
works ok if the request is for HTTP/1.0. It only breaks on HTTP/1.1 so I'm
thinking perhaps the chunk filter or something it uses (like
apr_brigade_split) is broken. I'll try to find some time to step through
the code & figure it out.
--
______________________________________________________________________________
| Brian Havard | "He is not the messiah! |
| brianh@kheldar.apana.org.au | He's a very naughty boy!" - Life of Brian |
------------------------------------------------------------------------------