You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Torsten Foertsch <to...@gmx.net> on 2009/03/20 20:17:08 UTC
mod_include supporting POST subrequests
Hi,
I need the include virtual directive to be able to issue POST requests.
It should pass the request body to the subrequest. So I came up with
the attached patch.
It allows to write
<!--#include method="post" virtual="..." --> or
<!--#include method="inherit" virtual="..." -->
I think the patch is right so far but I wouldn't mind if some more
experienced eyes had a look.
One problem remains. What if the HTML author uses that method=post thing
twice? The first call has consumed the post body. The second will try
to read something but the client has already sent all it can.
Hence, I'd like to
if (!eof_on_input()) {
rr->headers_in = r->headers_in;
}
How can I determine if the request body for the current request has
already been consumed?
Is there a chance that a patch like this will be accepted for inclusion
in apache?
Torsten
--
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net
Re: mod_include supporting POST subrequests
Posted by Torsten Foertsch <to...@gmx.net>.
On Wed 01 Apr 2009, Graham Leggett wrote:
> > Is there a chance for the patch to make it into 2.3++? If yes I'll
> > merge it with the KEPT_BODY stuff.
>
> Having two separate mechanisms to solve the same problem is not
> ideal. In addition, creating a solution that only works in one place
> (mod_include), is less ideal still.
>
> It should be relatively straightforward to amend the KEEP_BODY and
> KEPT_BODY filters so that, by default, the first attempt to read the
> body is passed through, and the second and subsequent attempts to
> read the body return an empty brigade.
>
> This will give you the behaviour you are looking for, and it will
> work anywhere within the server, not just in mod_include.
That is what I thought to do. If KeptBodySize is 0 the body is passed to
the first (sub)request that is reading it. All subsequent subrequests
will see an empty stream. If KeptBodySize is >0 the first (sub)request
reads the whole body and the KEEP_BODY_FILTER saves as much as is
configured. Subsequent subrequests are passed this kept body.
I also thought of writing the body to temporary files if configured. So
it's possible to preserve larger body without much headache about
memory consumption. Plus, I think it would be a nice feature to be able
to
<!--#include method="post" body="a=b;c=d" virtual="..." -->
and perhaps also to include an encoding="multipart/form-data", would it
not?
BTW, the current patch is not only for mod_include. It should work
(although not tested) for other filters/handlers as well as long as the
main request sets the "subreq-pass-request-body" note to prevent the
header table to be overwritten for the subrequest. This more or less
resembles what r->kept_body does in 2.3. Alternatively the caller could
overwrite the header table after creating the subreq but before running
it and set the original CL and TE headers.
But to restate my question, can I take your reply as "yes, go ahead, it
would be nice to have that feature in apache httpd"?
Torsten
--
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net
Re: mod_include supporting POST subrequests
Posted by Graham Leggett <mi...@sharp.fm>.
Torsten Foertsch wrote:
> I did and, frankly, it is not the solution I was looking for. One has to
> define a max. body size to be kept. The body is kept in RAM which can
> be a problem unless KeptBodySize is rather small. So I developed my
> patch further.
>
> It defers now the ap_discard_request_body call as much as possible. This
> gives output filters the chance to read the req body. If the client is
> expecting a "100 Continue" message it is sent just before the first
> line of output.
>
> Is there a chance for the patch to make it into 2.3++? If yes I'll merge
> it with the KEPT_BODY stuff.
Having two separate mechanisms to solve the same problem is not ideal.
In addition, creating a solution that only works in one place
(mod_include), is less ideal still.
It should be relatively straightforward to amend the KEEP_BODY and
KEPT_BODY filters so that, by default, the first attempt to read the
body is passed through, and the second and subsequent attempts to read
the body return an empty brigade.
This will give you the behaviour you are looking for, and it will work
anywhere within the server, not just in mod_include.
Regards,
Graham
--
Re: mod_include supporting POST subrequests
Posted by Torsten Foertsch <to...@gmx.net>.
On Fri 20 Mar 2009, Graham Leggett wrote:
> Torsten Foertsch wrote:
> > I need the include virtual directive to be able to issue POST
> > requests. It should pass the request body to the subrequest. So I
> > came up with the attached patch.
> >
> > It allows to write
> >
> > <!--#include method="post" virtual="..." --> or
> > <!--#include method="inherit" virtual="..." -->
> >
[...]
> Something like this has already been added to trunk, take a look at
> the KEEP_BODY and KEPT_BODY filters in modules/filters/mod_request.c.
I did and, frankly, it is not the solution I was looking for. One has to
define a max. body size to be kept. The body is kept in RAM which can
be a problem unless KeptBodySize is rather small. So I developed my
patch further.
It defers now the ap_discard_request_body call as much as possible. This
gives output filters the chance to read the req body. If the client is
expecting a "100 Continue" message it is sent just before the first
line of output.
Is there a chance for the patch to make it into 2.3++? If yes I'll merge
it with the KEPT_BODY stuff.
Currently my httpd passes the current test framework with a few more
patches that are not related to this one (see "2.2.11 & mod_include"
thread) with one exception. Since the request body is read when output
is potentially already on the wire a HTTP_REQUEST_ENTITY_TOO_LARGE
error cannot be sent to the client if it sends the request body in
chunked TE. The only sensible solution that I can think of would be to
always send a 413 response if TE is chunked and a LimitRequestBody is
active.
On Fri 20 Mar 2009, Nick Kew wrote:
> Erm ... that's ringing alarm bells. The client, not the
> server, determines HTTP methods. Or are you talking about
> proxied subrequests here?
I see it a bit different. Subrequests for included documents are made on
behalf of the HTML programmer who wrote the frame. He decides to pass
on the request body and he decides which method to use, IMHO. And yes,
the problem comes from subrequests that are proxied to another server.
Torsten
--
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net
Re: mod_include supporting POST subrequests
Posted by Graham Leggett <mi...@sharp.fm>.
Torsten Foertsch wrote:
> I need the include virtual directive to be able to issue POST requests.
> It should pass the request body to the subrequest. So I came up with
> the attached patch.
>
> It allows to write
>
> <!--#include method="post" virtual="..." --> or
> <!--#include method="inherit" virtual="..." -->
>
> I think the patch is right so far but I wouldn't mind if some more
> experienced eyes had a look.
>
> One problem remains. What if the HTML author uses that method=post thing
> twice? The first call has consumed the post body. The second will try
> to read something but the client has already sent all it can.
>
> Hence, I'd like to
>
> if (!eof_on_input()) {
> rr->headers_in = r->headers_in;
> }
>
> How can I determine if the request body for the current request has
> already been consumed?
>
> Is there a chance that a patch like this will be accepted for inclusion
> in apache?
Something like this has already been added to trunk, take a look at the
KEEP_BODY and KEPT_BODY filters in modules/filters/mod_request.c.
To switch it on, define the KeptBodySize directive:
http://httpd.apache.org/docs/trunk/mod/mod_request.html#keptbodysize
When the body is kept, the request method is maintained across
mod_include, so a POST to foo.shtml will stay a POST through
<!--#include virtual="bar.cgi"-->. You can nest includes as deep as you
like, the body is only set aside once.
Regards,
Graham
--
Re: mod_include supporting POST subrequests
Posted by Nick Kew <ni...@webthing.com>.
On Fri, 20 Mar 2009 20:17:08 +0100
Torsten Foertsch <to...@gmx.net> wrote:
> I need the include virtual directive to be able to issue POST
> requests. It should pass the request body to the subrequest. So I
> came up with the attached patch.
Did you mean to attach something?
> It allows to write
>
> <!--#include method="post" virtual="..." --> or
> <!--#include method="inherit" virtual="..." -->
Erm ... that's ringing alarm bells. The client, not the
server, determines HTTP methods. Or are you talking about
proxied subrequests here?
> One problem remains. What if the HTML author uses that method=post
> thing twice? The first call has consumed the post body. The second
> will try to read something but the client has already sent all it can.
You'd want something like mod_request or mod_form to parse the data
and make them available to other modules in a convenient form.
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/