CGI single-byte reads Re: 2.0.30-dev load spiking [was: upgrade to FreeBSD 4.5-PRERELEASE]

[Brian Pane <br...@cnet.com>]

Aaron Bannert wrote:

>On Thu, Jan 03, 2002 at 11:56:26AM -0500, Greg Ames wrote:
>
>>I do see some weirdness in 2.0.30 with www.apache.org/dyn/closer.cgi -
>>it looks like we're doing one byte reads from the pipe to the cgi.  I
>>don't know yet if 2_0_28 does the same.  That's all I've spotted so far,
>>except for a couple of annoying extra syscalls that also exist in
>>2_0_28.
>>
>
>I don't think that would spike the run queue more than 1, no?
>
>Of course, one-byte reads from a cgi pipe are just plain bogus
>to begin with, but on a different scale methinks.
>

The one-byte reads are for the CGI's response header.  The problem is that
mod_cgi hands off the pipe (non-buffered) to ap_scan_script_header_err(),
which does single-byte reads until it finds the end of the header.  After
that, mod_cgi finally builds a pipe-bucket, which is buffered, so the 
reading
of the content happens in much larger chunks.

The easiest solution I can think of is to change ap_scan_script_header_err()
to accept a brigade rather than an apr_file_t, so that we can take advantage
of the pipe bucket's buffering when reading the headers.

Brian

Re: CGI single-byte reads Re: 2.0.30-dev load spiking [was: upgrade to FreeBSD 4.5-PRERELEASE]

[Bill Stoddard <bi...@wstoddard.com>]

> > On Fri, Jan 04, 2002 at 11:39:27PM -0800, Brian Pane wrote:
> > > The one-byte reads are for the CGI's response header.  The problem is that
> > > mod_cgi hands off the pipe (non-buffered) to ap_scan_script_header_err(),
> > > which does single-byte reads until it finds the end of the header.  After
> > > that, mod_cgi finally builds a pipe-bucket, which is buffered, so the
> > > reading
> > > of the content happens in much larger chunks.
> > >
> > > The easiest solution I can think of is to change ap_scan_script_header_err()
> > > to accept a brigade rather than an apr_file_t, so that we can take advantage
> > > of the pipe bucket's buffering when reading the headers.
> >
> > I've just committed a note about this in STATUS. I think converting
> > it to take a brigade is the way to go.
> >
> > -aaron
> >
> 
> ap_scan_script_header_err calls apr_file_gets(buf, len, (apr_file_t *) f) where len = 8191 bytes.
> 
> apr_file_gets reads one byte at a time, even if it has a large buffer available to it.  What's up
> with that? Looks like apr_file_gets is wonky.
> 
> Bill
> 

Ahh... I grok it now.  apr_file_gets is behaving like gets, as it should.  Brigades it is...

Bill

Re: CGI single-byte reads Re: 2.0.30-dev load spiking [was: upgrade to FreeBSD 4.5-PRERELEASE]

[Bill Stoddard <bi...@wstoddard.com>]

> On Fri, Jan 04, 2002 at 11:39:27PM -0800, Brian Pane wrote:
> > The one-byte reads are for the CGI's response header.  The problem is that
> > mod_cgi hands off the pipe (non-buffered) to ap_scan_script_header_err(),
> > which does single-byte reads until it finds the end of the header.  After
> > that, mod_cgi finally builds a pipe-bucket, which is buffered, so the
> > reading
> > of the content happens in much larger chunks.
> >
> > The easiest solution I can think of is to change ap_scan_script_header_err()
> > to accept a brigade rather than an apr_file_t, so that we can take advantage
> > of the pipe bucket's buffering when reading the headers.
>
> I've just committed a note about this in STATUS. I think converting
> it to take a brigade is the way to go.
>
> -aaron
>

ap_scan_script_header_err calls apr_file_gets(buf, len, (apr_file_t *) f) where len = 8191 bytes.

apr_file_gets reads one byte at a time, even if it has a large buffer available to it.  What's up
with that? Looks like apr_file_gets is wonky.

Bill

Re: CGI single-byte reads Re: 2.0.30-dev load spiking [was: upgrade to FreeBSD 4.5-PRERELEASE]

[Aaron Bannert <aa...@clove.org>]

On Fri, Jan 04, 2002 at 11:39:27PM -0800, Brian Pane wrote:
> The one-byte reads are for the CGI's response header.  The problem is that
> mod_cgi hands off the pipe (non-buffered) to ap_scan_script_header_err(),
> which does single-byte reads until it finds the end of the header.  After
> that, mod_cgi finally builds a pipe-bucket, which is buffered, so the 
> reading
> of the content happens in much larger chunks.
> 
> The easiest solution I can think of is to change ap_scan_script_header_err()
> to accept a brigade rather than an apr_file_t, so that we can take advantage
> of the pipe bucket's buffering when reading the headers.

I've just committed a note about this in STATUS. I think converting
it to take a brigade is the way to go.

-aaron