input filter which modifies body, Content-Length

[Jeff Trawick <tr...@gmail.com>]

(Apologies if there is juicy past discussion that I didn't find with
my search keywords.)

It isn't clear to me what an input filter should do about
Content-Length when it modifies the length of the body  (assuming that
this isn't chunked encoding).

It causes problems for some handlers to modify the length of the
request body but leave Content-Length alone.  Handlers which care
about Content-Length have probably already looked at it before the
input filter has been called anyway.

A handler that uses brigades to read the body and has no need to look
at Content-Length should be okay either way.

mod_cgi uses brigades to read the body but needs to look at
Content-Length before spawning the CGI script, so that's problematic. 
And there is an unexpected ordering requirement so that the input
filter can signal to this handler that the content-length can't be
trusted, before mod_cgi spawns the child.

ap_setup_client_block() looks at the initial Content-Length to know
how much to read, so that's problematic but perhaps fixable for these
types of handlers.

A filter which spools up to a configured amount of request body in
order to calculate content-length could be of some practical benefit,
since many request bodies are relatively small and this could
potentially allow mod_cgi[d] to properly handle chunked request
bodies, regardless of input filtering.  With such a filter installed,
and no need to spool beyond a configured limit, getting a brigade
would return bucket(s) with known length and an EOS at the end. 
Unknown lengths or EOS?  Better punt if you're mod_cgi[d].

Thoughts?

Re: input filter which modifies body, Content-Length

[Jeff Trawick <tr...@gmail.com>]

On 2/24/06, Greg Ames <gr...@apache.org> wrote:
> Jeff Trawick wrote:
>
> > It isn't clear to me what an input filter should do about
> > Content-Length when it modifies the length of the body  (assuming that
> > this isn't chunked encoding).
>
> > mod_cgi uses brigades to read the body but needs to look at
> > Content-Length before spawning the CGI script, so that's problematic.
> > And there is an unexpected ordering requirement so that the input
> > filter can signal to this handler that the content-length can't be
> > trusted, before mod_cgi spawns the child.
>
> so is this the current ordering?
>
> 1. mod_cgi[d] handler is dispatched
> 2. C-L environment variable is set for the script from the initial C-L header
> 3. CGI child is spawned.
> 4. mod_cgi[d] reads the body from input filters
> 5. foo_input_filter changes the body, invalidating the C-L env var + whatever local
> variables the script is using to track the length
> 6. CGI reads from stdin into buffer of length ?

looks right to me

>
> > A filter which spools up to a configured amount of request body in
> > order to calculate content-length could be of some practical benefit,
> > since many request bodies are relatively small and this could
> > potentially allow mod_cgi[d] to properly handle chunked request
> > bodies, regardless of input filtering.  With such a filter installed,
> > and no need to spool beyond a configured limit, getting a brigade
> > would return bucket(s) with known length and an EOS at the end.
> > Unknown lengths or EOS?  Better punt if you're mod_cgi[d].
>
> per offline discussions,
>
> * the CGI spec (fwiw) is oblivious to chunking
> * 1.3 and pre-filtering 2.0 used to fail CGI requests with chunked bodies.  getting them
> to work properly in common cases (i.e. < 8K bodies ) would be a step forward.
> * this is analogous to proxy trying to avoid chunking to the origin server, except the
> spec is weaker for CGIs
>
> a decent solution for CGI request body chunking would also solve some cases of input
> filters modifying the length.

y

I suspect a number of third-party modules would need to change (not to
mention our own).

The handler would need to get a brigade then, if there is EOS bucket
at the end and the length of each bucket is known, then use that for
content-length.  (Presumably a magic filter with configurable timeouts
and spooling capability would allow this to happen.)  Otherwise,
either fail the request (e.g., CGI) or don't handle the body in a way
that needs to know the length in advance.

Re: input filter which modifies body, Content-Length

[Greg Ames <gr...@apache.org>]

Jeff Trawick wrote:

> It isn't clear to me what an input filter should do about
> Content-Length when it modifies the length of the body  (assuming that
> this isn't chunked encoding).

> mod_cgi uses brigades to read the body but needs to look at
> Content-Length before spawning the CGI script, so that's problematic. 
> And there is an unexpected ordering requirement so that the input
> filter can signal to this handler that the content-length can't be
> trusted, before mod_cgi spawns the child.

so is this the current ordering?

1. mod_cgi[d] handler is dispatched
2. C-L environment variable is set for the script from the initial C-L header
3. CGI child is spawned.
4. mod_cgi[d] reads the body from input filters
5. foo_input_filter changes the body, invalidating the C-L env var + whatever local 
variables the script is using to track the length
6. CGI reads from stdin into buffer of length ?

> A filter which spools up to a configured amount of request body in
> order to calculate content-length could be of some practical benefit,
> since many request bodies are relatively small and this could
> potentially allow mod_cgi[d] to properly handle chunked request
> bodies, regardless of input filtering.  With such a filter installed,
> and no need to spool beyond a configured limit, getting a brigade
> would return bucket(s) with known length and an EOS at the end. 
> Unknown lengths or EOS?  Better punt if you're mod_cgi[d].

per offline discussions,

* the CGI spec (fwiw) is oblivious to chunking
* 1.3 and pre-filtering 2.0 used to fail CGI requests with chunked bodies.  getting them 
to work properly in common cases (i.e. < 8K bodies ) would be a step forward.
* this is analogous to proxy trying to avoid chunking to the origin server, except the 
spec is weaker for CGIs

a decent solution for CGI request body chunking would also solve some cases of input 
filters modifying the length.

Greg