You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modules-dev@httpd.apache.org by Steve Kemp <st...@steve.org.uk> on 2006/10/13 21:26:21 UTC
Reading POST request without destroying it?
I've been working on a module for filtering incoming requests for
some time now.
I can read the incoming request correctly for most cases, but I'm
having a real problem with POST handling.
My module successfully uses ap_setup_client_block + ap_get_client_block
to read in the payload - but once I've done this it is exhausted and
no longer available to subsequent modules, such as mod_cgi.
Is there a simple way for my module to read/process the incoming
POST data, whilst still allowing the data to be passed on to
mod_cgi, etc?
(Coding against Apache 2.x, rather than 1.3, or 2.2 if that is
relevant.)
Steve
--
http://www.steve.org.uk/
Re: Reading POST request without destroying it?
Posted by Steve Kemp <st...@steve.org.uk>.
On Fri, Oct 13, 2006 at 03:50:17PM -0500, Andrew E. White wrote:
> I just moded the upper case example input filter in the apache module
> source tree.
I missed that one, by looking in modules/filters, not
modules/experimental!
I see how that example works, but I'm still a little unclear on
the contents of the buckets. I'm only wanting post data, and
it isn't immediately obvious what the contents of the buckets would
be in that code.
I guess I will compile it and see.
> Basicly you just want to create a heap brigade queue and copy all
> data into it and pop it back off in order.
Right I see how that works with the creation and the ap_get_brigade
code - then the loop running:
while( !APR_BRIGADE_EMPTY( .. ) )
{ ...
...
apr_brigade_read()
..
..
}
> I can send out some sample code later if you still need help.
It would be appreciated if it is commented better than the
sample code, or explicitly deals with fixup routines and filters
or is post specific.
But thanks a lot for pointing me in the right direction already.
Steve
--
Debian GNU/Linux System Administration
http://www.debian-administration.org/
Re: Reading POST request without destroying it?
Posted by "Andrew E. White" <ae...@uark.edu>.
I just moded the upper case example input filter in the apache module source tree. Basicly you just want to create a heap brigade queue and copy all data into it and pop it back off in order. I can send out some sample code later if you still need help.
Andrew
----- Original Message -----
From: Steve Kemp <st...@steve.org.uk>
Date: Friday, October 13, 2006 3:44 pm
Subject: Re: Reading POST request without destroying it?
To: modules-dev@httpd.apache.org
> On Fri, Oct 13, 2006 at 09:41:21PM +0100, Ivan Ristic wrote:
>
> > Have a look at ModSecurity (http://www.modsecurity.org). It does
> what> you ask for.
>
> My module attempts to do a similar thing:
>
> http://mod_ifier.cvsrepository.org/
>
> (CGI Parsing : http://www.cvsrepository.org/cgi-
> bin/trac/mod_ifier/getfile/mod_ifier/src/cgi.c )
>
> I've looked at the mod_security code but there is a lot of
> complexity there and I'm finding it hard to follow.
> I've seen several GPL modules which use buckets but I'm still not
> sure I understand exactly how they work. I will keep experimenting.
>
> Steve
> --
>
Re: Reading POST request without destroying it?
Posted by Steve Kemp <st...@steve.org.uk>.
On Fri, Oct 13, 2006 at 09:41:21PM +0100, Ivan Ristic wrote:
> Have a look at ModSecurity (http://www.modsecurity.org). It does what
> you ask for.
My module attempts to do a similar thing:
http://mod_ifier.cvsrepository.org/
(CGI Parsing : http://www.cvsrepository.org/cgi-bin/trac/mod_ifier/getfile/mod_ifier/src/cgi.c )
I've looked at the mod_security code but there is a lot of complexity
there and I'm finding it hard to follow.
I've seen several GPL modules which use buckets but I'm still not
sure I understand exactly how they work. I will keep experimenting.
Steve
--
Re: Reading POST request without destroying it?
Posted by Ivan Ristic <iv...@gmail.com>.
On 10/13/06, Steve Kemp <st...@steve.org.uk> wrote:
> On Fri, Oct 13, 2006 at 08:38:44PM +0100, Nick Kew wrote:
>
> > > Is there a simple way for my module to read/process the incoming
> > > POST data, whilst still allowing the data to be passed on to
> > > mod_cgi, etc?
> >
> > That's what input filters are for. How best to do it depends on the
> > nature and size of the requests you're processing.
>
> Thanks.
>
> I'm currently trying to find some decent sample code...
Have a look at ModSecurity (http://www.modsecurity.org). It does what
you ask for.
--
Ivan Ristic
Re: Reading POST request without destroying it?
Posted by Nick Kew <ni...@webthing.com>.
On Friday 13 October 2006 20:42, Steve Kemp wrote:
> On Fri, Oct 13, 2006 at 08:38:44PM +0100, Nick Kew wrote:
> > > Is there a simple way for my module to read/process the incoming
> > > POST data, whilst still allowing the data to be passed on to
> > > mod_cgi, etc?
> >
> > That's what input filters are for. How best to do it depends on the
> > nature and size of the requests you're processing.
>
> Thanks.
>
> I'm currently trying to find some decent sample code... The requests
> I expect to be parsing are "small" in the sense that they will
> typically have less than ten key=value elements.
In that case, you can just collect all the input in a single buffer and
parse it (don't forget to check the size and reject anything ridiculously
big, so you don't invite a DOS attack). Do that in an input filter and
return the data to the caller if you want it to work with CGI.
You have to take some care over the ordering of the processing:
ISTR my old apachetutor article on request processing explains it.
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.prenhallprofessional.com/title/0132409674
Re: Reading POST request without destroying it?
Posted by Steve Kemp <st...@steve.org.uk>.
A while back I asked for help with reading POST data in a module,
such that it would still be available for use by CGI processes etc.
I thought I had a solution using brigades and buckets, but it
seems like I'm still missing the obvious as my data is still removed
by the time mod_cgi/mod_php is invoked.
I've included the code I'm using below, if there is anything
immediately visible I'd appreciate pointers ..
Steve
--
http://www.steve.org.uk/
tatic const char * mod_ifier_parse_read_post_payload(request_rec *r, const char **rbuf)
{
apr_bucket_brigade *bb;
int seen_eos = 0;
apr_status_t rv;
apr_uri_t *uri = &r->parsed_uri;
uri->query = "";
bb = apr_brigade_create(r->pool,r->connection->bucket_alloc);
do
{
apr_bucket *bucket;
rv = ap_get_brigade(r->input_filters,bb,AP_MODE_READBYTES,APR_BLOCK_READ,HUGE_STRING_LEN);
if (rv != APR_SUCCESS)
{
return NULL;
}
while (!APR_BRIGADE_EMPTY(bb))
{
const char *data;
apr_size_t len;
bucket = APR_BRIGADE_FIRST(bb);
if (APR_BUCKET_IS_EOS(bucket))
{
seen_eos = 1;
break;
}
if (APR_BUCKET_IS_FLUSH(bucket))
{
continue;
}
apr_bucket_read(bucket,&data,&len,APR_BLOCK_READ);
uri->query = apr_pstrcat(r->pool,uri->query,data,NULL);
apr_bucket_delete(bucket);
}
apr_brigade_cleanup(bb);
} while (!seen_eos);
return( apr_pstrdup( r->pool, uri->query ) );
}
Re: Reading POST request without destroying it?
Posted by Issac Goldstand <ma...@beamartyr.net>.
If the input is formatted using a "standard" encoding (eg,
multipart/form-data or application/x-www-form-urlencoded) which it
sounds like you are, you might save yourself by using mod_apreq to parse
the input. In Apache2, it's implemented as an input filter, so content
can still be seen by filters after it. It's homepage is
http://httpd.apache.org/apreq/
Even if it's not what you need (and to me, at least, it sounds like it
is), at the very least you'll probably get good sample code for
developing input filters.
Issac
Steve Kemp wrote:
> On Fri, Oct 13, 2006 at 08:38:44PM +0100, Nick Kew wrote:
>
>>> Is there a simple way for my module to read/process the incoming
>>> POST data, whilst still allowing the data to be passed on to
>>> mod_cgi, etc?
>> That's what input filters are for. How best to do it depends on the
>> nature and size of the requests you're processing.
>
> Thanks.
>
> I'm currently trying to find some decent sample code... The requests
> I expect to be parsing are "small" in the sense that they will
> typically have less than ten key=value elements.
>
>
> Steve
Re: Reading POST request without destroying it?
Posted by Steve Kemp <st...@steve.org.uk>.
On Fri, Oct 13, 2006 at 08:38:44PM +0100, Nick Kew wrote:
> > Is there a simple way for my module to read/process the incoming
> > POST data, whilst still allowing the data to be passed on to
> > mod_cgi, etc?
>
> That's what input filters are for. How best to do it depends on the
> nature and size of the requests you're processing.
Thanks.
I'm currently trying to find some decent sample code... The requests
I expect to be parsing are "small" in the sense that they will
typically have less than ten key=value elements.
Steve
--
Re: Reading POST request without destroying it?
Posted by Nick Kew <ni...@webthing.com>.
On Friday 13 October 2006 20:26, Steve Kemp wrote:
> Is there a simple way for my module to read/process the incoming
> POST data, whilst still allowing the data to be passed on to
> mod_cgi, etc?
That's what input filters are for. How best to do it depends on the
nature and size of the requests you're processing.
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.prenhallprofessional.com/title/0132409674