You are viewing a plain text version of this content. The canonical link for it is here.
Posted to apreq-dev@httpd.apache.org by Jeffrey Horner <je...@vanderbilt.edu> on 2005/04/01 23:16:37 UTC
Language bindings other that Perl
Hello all,
I've used the apreq library in some of my c modules, and it works great.
Thank's for all the hard work.
I'm currently working on mod_R, embedding the R statistical language
inside of Apache, and I'd like to use apreq.
My question is can I distribute libapreq with my module. I would not
want user to have to download both mod_R and libapreq to get the full
functionality.
Another questions is how come more languages do not use apreq? What are
their reasons for "rolling their own" form parsers and such?
Thanks again,
--
Jeffrey Horner Computer Systems Analyst School of Medicine
615-322-8606 Department of Biostatistics Vanderbilt University
Re: Language bindings other that Perl
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Jeffrey Horner <je...@vanderbilt.edu> writes:
> Who are the main httpd developers? I'd like to know so that
> I can find some way of raising this issue of placing libapreq
> into the core. It seems obvious to me...
dev@httpd is the place to find them. They're good
people, and I'm sure it'll happen when the time is
right. But I'm a Taoist, so that's always the case ;-)
--
Joe Schaefer
Re: Language bindings other that Perl
Posted by Jeffrey Horner <je...@vanderbilt.edu>.
Joe Schaefer wrote:
> "William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
[...]
> It seems
> like a no-brainer to me that httpd would want something like
> apreq2 in the core, but that's not the feeling I get whenever someone
> asks them about it.
Who are the main httpd developers? I'd like to know so that I can find
some way of raising this issue of placing libapreq into the core. It
seems obvious to me...
--
Jeffrey Horner Computer Systems Analyst School of Medicine
615-322-8606 Department of Biostatistics Vanderbilt University
Re: Language bindings other that Perl
Posted by "admin@rowe-clan.net" <ad...@rowe-clan.net>.
At 08:04 AM 4/14/2005, Joe Schaefer wrote:
>Thanks. I looked it over, but it's still beyond my ken and may
>be a while before I grok it enough to write a functional patch.
>If your chief concern is the spool-file duplication, maybe we
>should shoot for a working patch first, and *then* look for a
>way to eliminate the duplication. Sound reasonable?
Yup - I'll be looking at your apreq_brigade_concat suggestion
early next week, seeing if there is a trivial way to morph the
current body stream (past the ssl filter, before any further
filtering) into a temp file brigade.
Once that works, and libapreq has all the helpers we need, it
should be trivial to wiggle the api to reference a file/mem
brigade for the mime chunks, if they are persistant, or to
rewrite as a file -again- if it's been morphed by a later filter
as a stream.
Bill
Re: Language bindings other that Perl
Posted by Joe Schaefer <jo...@sunstarsys.com>.
"William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
> At 04:39 PM 4/12/2005, Joe Schaefer wrote:
[...]
>
>>I don't recall the exact sequence of HTTP events here, so
>>if you could remind me that might help.
>
> I'd think we would inject it, and then cause it to spool the entire
> contents, in the vicinity of line 503 of ssl_engine_kernel.c (trunk).
Thanks. I looked it over, but it's still beyond my ken and may
be a while before I grok it enough to write a functional patch.
If your chief concern is the spool-file duplication, maybe we
should shoot for a working patch first, and *then* look for a
way to eliminate the duplication. Sound reasonable?
--
Joe Schaefer
Re: Language bindings other that Perl
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 04:39 PM 4/12/2005, Joe Schaefer wrote:
>Joe Schaefer <jo...@sunstarsys.com> writes:
>
>> "William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
>>
>>> At 11:23 AM 4/12/2005, Joe Schaefer wrote:
>>>>"William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
>>>>
>>>>> . mod_ssl triggers renegotiate
>>>>>
>>>>> . body exists ... stream *entire* 102MB data stream into a tempfile
>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>>How did *that* happen?
>>>
>>> It doesn't. That's what MUST happen.
>>
>> If I were implementing that, I'd use apreq_brigade_concat() in
>> the mod_ssl filter.
> ^^^^^^^
>
>Hmm, I haven't thought about ssl negotiation in a long while,
>but perhaps I should write "ap_http_filter" there. I don't recall
>the exact sequence of HTTP events here, so if you could remind me
>that might help.
I'd think we would inject it, and then cause it to spool the entire
contents, in the vicinity of line 503 of ssl_engine_kernel.c (trunk).
Re: Language bindings other that Perl
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Joe Schaefer <jo...@sunstarsys.com> writes:
> "William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
>
>> At 11:23 AM 4/12/2005, Joe Schaefer wrote:
>>>"William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
>>>
>>>> . mod_ssl triggers renegotiate
>>>>
>>>> . body exists ... stream *entire* 102MB data stream into a tempfile
>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>How did *that* happen?
>>
>> It doesn't. That's what MUST happen.
>
> If I were implementing that, I'd use apreq_brigade_concat() in
> the mod_ssl filter.
^^^^^^^
Hmm, I haven't thought about ssl negotiation in a long while,
but perhaps I should write "ap_http_filter" there. I don't recall
the exact sequence of HTTP events here, so if you could remind me
that might help.
--
Joe Schaefer
Re: Language bindings other that Perl
Posted by Joe Schaefer <jo...@sunstarsys.com>.
"William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
> At 11:23 AM 4/12/2005, Joe Schaefer wrote:
>>"William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
>>
>>> . mod_ssl triggers renegotiate
>>>
>>> . body exists ... stream *entire* 102MB data stream into a tempfile
>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>How did *that* happen?
>
> It doesn't. That's what MUST happen.
If I were implementing that, I'd use apreq_brigade_concat() in
the mod_ssl filter. In other words, I wouldn't use mod_apreq2
for this, I'd use libapreq2.
Yes, if mod_apreq2 gets involved in the request later on,
that'd mean duplication in the spool files. It might be
possible to remove that duplication, but I think it'd be
tricky because some intermediary filter might be modifying
the stream.
--
Joe Schaefer
Re: Language bindings other that Perl
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 11:23 AM 4/12/2005, Joe Schaefer wrote:
>"William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
>
>> . mod_ssl triggers renegotiate
>>
>> . body exists ... stream *entire* 102MB data stream into a tempfile
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>How did *that* happen?
It doesn't. That's what MUST happen.
At least, if we ever choose to allow renegotiating client certs
on POST bodies. apache 1.3 allows this; apache 2.0 crippled it,
right now, the existing code *would* stream in 102MB into ram,
until some file based cache is provided.
Which was *exactly* my argument why apreq could be an excellent
provider of this necessary mod_ssl evil :)
Bill
Re: Language bindings other that Perl
Posted by Joe Schaefer <jo...@sunstarsys.com>.
"William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
> . User POSTs a 100MB file upload.
>
> . apreq is invoked to stream this body
>
> . mod_ssl triggers renegotiate
>
> . body exists ... stream *entire* 102MB data stream into a tempfile
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
How did *that* happen?
--
Joe Schaefer
Re: Language bindings other that Perl
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 10:59 AM 4/12/2005, Joe Schaefer wrote:
>"William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
>
>> At 09:23 AM 4/8/2005, Jeffrey Horner wrote:
>>
>>>Well, it seems to me that it should happen. What's holding back the
>>>progress?
>>
>> Nothing, but to present as a patch to httpd/trunk/ and arguments
>> for its inclusion.
>>
>> Arguments, if we streamed large response bodies to a file for the
>> benefit of all filters (either in addition or not to the individual
>> mime file streams) - this would permit large POST bodies to work
>> with SSL renegotiation.
>
>mod_apreq2 is a lazy, transparent filter, so that shouldn't be
>a problem for output filters that use apreq. The range
>of supported per-request phases is supposed to run the full gamut,
>or at least from auth handlers to output filters. Since the
>mod_apreq2 filter always sits behind ap_http_filter, I don't
>see how SSL renegotiation could be negatively impacted by its
>presence.
. User POSTs a 100MB file upload.
. apreq is invoked to stream this body
. mod_ssl triggers renegotiate
. body exists ... stream *entire* 102MB data stream into a tempfile
. apreq writes 100MB file mime body into a tempfile
. handler grabs the body from apreq after validating the cert.
You don't agree that a cumulative 202MB temp file usecase, double
store of the same body, is ineffectual?
Using file buckets, it would be trivial for apreq to create a file
bucket into the common tempfile, representing the mime body. This
would save the redundant double-storage of the uploaded body or
other sorts of body streams.
Thoughts?
Bill
Re: Language bindings other that Perl
Posted by Joe Schaefer <jo...@sunstarsys.com>.
"William A. Rowe, Jr." <wr...@rowe-clan.net> writes:
> At 09:23 AM 4/8/2005, Jeffrey Horner wrote:
>
>>Well, it seems to me that it should happen. What's holding back the
>>progress?
>
> Nothing, but to present as a patch to httpd/trunk/ and arguments
> for its inclusion.
>
> Arguments, if we streamed large response bodies to a file for the
> benefit of all filters (either in addition or not to the individual
> mime file streams) - this would permit large POST bodies to work
> with SSL renegotiation.
mod_apreq2 is a lazy, transparent filter, so that shouldn't be
a problem for output filters that use apreq. The range
of supported per-request phases is supposed to run the full gamut,
or at least from auth handlers to output filters. Since the
mod_apreq2 filter always sits behind ap_http_filter, I don't
see how SSL renegotiation could be negatively impacted by its
presence.
> Next argument - we can use apreq to parse the mod_autoindex query
> string (and, accept these variables via form POST.)
Yes, assuming mod_autoindex expects query strings to appear in
name=value form. The basic design principle behind apreq is
to facilitate the development of a modular filter pipeline.
If someone writes a fantastic php application, and someone
else writes a fantastic mod_python filter, both of which
require access to the POST data, those apps could be chained
together if the mod_python filter is using mod_apreq2. It seems
like a no-brainer to me that httpd would want something like
apreq2 in the core, but that's not the feeling I get whenever someone
asks them about it.
--
Joe Schaefer
Re: Language bindings other that Perl
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 09:23 AM 4/8/2005, Jeffrey Horner wrote:
>Joe Schaefer wrote:
>[...]
>>The hope for apreq2 is/was to distribute it with the httpd server
>>itself (formally libapreq2 ~> libaprutil, mod_apreq2 ~> httpd module, glue code -> mod_perl2), so all apache C modules would have a uniform way to access form-data. Although that hasn't panned out yet, I'm still optimistic that it might.
>
>Well, it seems to me that it should happen. What's holding back the progress?
Nothing, but to present as a patch to httpd/trunk/ and arguments
for its inclusion.
Arguments, if we streamed large response bodies to a file for the
benefit of all filters (either in addition or not to the individual
mime file streams) - this would permit large POST bodies to work
with SSL renegotiation.
Next argument - we can use apreq to parse the mod_autoindex query
string (and, accept these variables via form POST.)
Next argument - closer scrutiny and broader adoption.
I've started to dig into the code, but it will be some time before
I'm ready to present the patch for httpd-2.1-dev. If someone else
beats me to this I will be very pleased to support it.
Bill
Re: Language bindings other that Perl
Posted by Jeffrey Horner <je...@vanderbilt.edu>.
Joe Schaefer wrote:
[...]
> The hope for apreq2 is/was to distribute it with the httpd server
> itself (formally libapreq2 ~> libaprutil, mod_apreq2 ~> httpd module,
> glue code -> mod_perl2), so all apache C modules would have a uniform
> way to access form-data. Although that hasn't panned out yet, I'm
> still optimistic that it might.
Well, it seems to me that it should happen. What's holding back the
progress?
--
Jeffrey Horner Computer Systems Analyst School of Medicine
615-322-8606 Department of Biostatistics Vanderbilt University
Re: Language bindings other that Perl
Posted by "David N. Welton" <da...@dedasys.com>.
Joe Schaefer <jo...@sunstarsys.com> writes:
> I know mod_ruby and mod_dtcl use libapreq, but I think they include
> the library in their own distros.
Well, it's "Apache Rivet" now, but yes, we use it and are happy with
it!
Ciao,
--
David N. Welton
- http://www.dedasys.com/davidw/
Apache, Linux, Tcl Consulting
- http://www.dedasys.com/
Re: Language bindings other that Perl
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Jeffrey Horner <je...@vanderbilt.edu> writes:
> I'm currently working on mod_R, embedding the R statistical language
> inside of Apache, and I'd like to use apreq.
Cool!
> My question is can I distribute libapreq with my module. I would not
> want user to have to download both mod_R and libapreq to get the full
> functionality.
Sure, you can do pretty much whatever you like with the code
(the Apache License is all you need to comply with). However,
if you're talking about apreq2, would it be possible to contribute
the apreq glue code to this project? Feel free to make use of the
glue/r directory in the multi-env-unstable branch (that branch will
wind up moving into trunk before the next developer release).
> Another questions is how come more languages do not use apreq? What
> are their reasons for "rolling their own" form parsers and such?
I know mod_ruby and mod_dtcl use libapreq, but I think they
include the library in their own distros. IIRC php4 cribbed
portions of apreq1's mfd parser instead of using the library's
API, and they quickly earned a few CVE numbers for doing that.
The hope for apreq2 is/was to distribute it with the httpd server
itself (formally libapreq2 ~> libaprutil, mod_apreq2 ~> httpd module,
glue code -> mod_perl2), so all apache C modules would have a uniform
way to access form-data. Although that hasn't panned out yet, I'm
still optimistic that it might.
--
Joe Schaefer