You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Graham Leggett <mi...@sharp.fm> on 2011/12/13 16:19:16 UTC
Proposal: adoption of mod_firehose subproject
Hi all,
I have concluded negotiation with the BBC to open source some httpd modules that I wrote under the AL, and the BBC have very kindly agreed to donate the code to the ASF[1], which I believe would fit well as subprojects of httpd, and would like to know whether httpd would accept these.
To be clear, this isn't a "code dump", my intention is to continue to develop and support this moving forward, and hopefully expand the community around them.
- mod_firehose: "tcpdump for httpd"
Based originally on mod_dumpio.c, mod_firehose is an httpd filter that writes the contents of a request and/or a response to a file or pipe in such a way that the requests can be reconstructed later using a second dedicated tool called "firehose".
It was initially developed to help debug restful services that were secured with client certificates and therefore opaque to other tools like tcpdump or tcpflow, but was then subsequently used to record "dirty traffic" for subsequent replay for the purposes of testing.
The module and the corresponding firehose demultiplexer was used to uncover some of the more tricky bugs in mod_cache, as well as protocol inconsistencies in backend services, and would prove very useful to anyone deploying restful services. We have also intended it to be used to create a "dark live" environment, where live traffic can be split off and diverted to a staging environment to test whether a software update works correctly.
The code is currently packaged as an RPM, wrapped in autotools, and a snapshot is available here:
http://people.apache.org/~minfrin/bbc-donated/mod_firehose/
http://people.apache.org/~minfrin/bbc-donated/firehose/
The corresponding README documenting in more detail is here:
http://people.apache.org/~minfrin/bbc-donated/mod_firehose/README
The code itself is here:
http://people.apache.org/~minfrin/bbc-donated/mod_firehose/mod_firehose.c
http://people.apache.org/~minfrin/bbc-donated/firehose/firehose.c
Obviously the expectation is for the documentation to be completed and fleshed out.
[1] https://issues.apache.org/bugzilla/show_bug.cgi?id=52322
Regards,
Graham
--
Re: Proposal: adoption of mod_firehose subproject
Posted by Jim Jagielski <ji...@jaguNET.com>.
+1!
On Dec 13, 2011, at 10:19 AM, Graham Leggett wrote:
> Hi all,
>
> I have concluded negotiation with the BBC to open source some httpd modules that I wrote under the AL, and the BBC have very kindly agreed to donate the code to the ASF[1], which I believe would fit well as subprojects of httpd, and would like to know whether httpd would accept these.
>
> To be clear, this isn't a "code dump", my intention is to continue to develop and support this moving forward, and hopefully expand the community around them.
>
> - mod_firehose: "tcpdump for httpd"
>
Re: Proposal: adoption of mod_firehose subproject
Posted by Graham Leggett <mi...@sharp.fm>.
On 13 Dec 2011, at 5:19 PM, Graham Leggett wrote:
> - mod_firehose: "tcpdump for httpd"
Based on the support expressed, and the expressed preference that the module be part of httpd, I have updated the documentation for both mod_firehose and firehose and will commit them to trunk shortly.
Further work can continue in svn.
Regards,
Graham
--
Re: Proposal: adoption of mod_firehose subproject
Posted by Arturo 'Buanzo' Busleiman <bu...@buanzo.com.ar>.
It is a wonderful tool, my vote does not count, but please +1 !!
On Tue, Dec 13, 2011 at 12:46 PM, Issac Goldstand <ma...@beamartyr.net> wrote:
> +1 on adopting
>
> On 13/12/2011 17:19, Graham Leggett wrote:
>> Hi all,
>>
>> I have concluded negotiation with the BBC to open source some httpd modules that I wrote under the AL, and the BBC have very kindly agreed to donate the code to the ASF[1], which I believe would fit well as subprojects of httpd, and would like to know whether httpd would accept these.
>>
>> To be clear, this isn't a "code dump", my intention is to continue to develop and support this moving forward, and hopefully expand the community around them.
>>
>> - mod_firehose: "tcpdump for httpd"
>>
>> Based originally on mod_dumpio.c, mod_firehose is an httpd filter that writes the contents of a request and/or a response to a file or pipe in such a way that the requests can be reconstructed later using a second dedicated tool called "firehose".
>>
>> It was initially developed to help debug restful services that were secured with client certificates and therefore opaque to other tools like tcpdump or tcpflow, but was then subsequently used to record "dirty traffic" for subsequent replay for the purposes of testing.
>>
>> The module and the corresponding firehose demultiplexer was used to uncover some of the more tricky bugs in mod_cache, as well as protocol inconsistencies in backend services, and would prove very useful to anyone deploying restful services. We have also intended it to be used to create a "dark live" environment, where live traffic can be split off and diverted to a staging environment to test whether a software update works correctly.
>>
>> The code is currently packaged as an RPM, wrapped in autotools, and a snapshot is available here:
>>
>> http://people.apache.org/~minfrin/bbc-donated/mod_firehose/
>> http://people.apache.org/~minfrin/bbc-donated/firehose/
>>
>> The corresponding README documenting in more detail is here:
>>
>> http://people.apache.org/~minfrin/bbc-donated/mod_firehose/README
>>
>> The code itself is here:
>>
>> http://people.apache.org/~minfrin/bbc-donated/mod_firehose/mod_firehose.c
>> http://people.apache.org/~minfrin/bbc-donated/firehose/firehose.c
>>
>> Obviously the expectation is for the documentation to be completed and fleshed out.
>>
>> [1] https://issues.apache.org/bugzilla/show_bug.cgi?id=52322
>>
>> Regards,
>> Graham
>> --
>
Re: Proposal: adoption of mod_firehose subproject
Posted by Issac Goldstand <ma...@beamartyr.net>.
+1 on adopting
On 13/12/2011 17:19, Graham Leggett wrote:
> Hi all,
>
> I have concluded negotiation with the BBC to open source some httpd modules that I wrote under the AL, and the BBC have very kindly agreed to donate the code to the ASF[1], which I believe would fit well as subprojects of httpd, and would like to know whether httpd would accept these.
>
> To be clear, this isn't a "code dump", my intention is to continue to develop and support this moving forward, and hopefully expand the community around them.
>
> - mod_firehose: "tcpdump for httpd"
>
> Based originally on mod_dumpio.c, mod_firehose is an httpd filter that writes the contents of a request and/or a response to a file or pipe in such a way that the requests can be reconstructed later using a second dedicated tool called "firehose".
>
> It was initially developed to help debug restful services that were secured with client certificates and therefore opaque to other tools like tcpdump or tcpflow, but was then subsequently used to record "dirty traffic" for subsequent replay for the purposes of testing.
>
> The module and the corresponding firehose demultiplexer was used to uncover some of the more tricky bugs in mod_cache, as well as protocol inconsistencies in backend services, and would prove very useful to anyone deploying restful services. We have also intended it to be used to create a "dark live" environment, where live traffic can be split off and diverted to a staging environment to test whether a software update works correctly.
>
> The code is currently packaged as an RPM, wrapped in autotools, and a snapshot is available here:
>
> http://people.apache.org/~minfrin/bbc-donated/mod_firehose/
> http://people.apache.org/~minfrin/bbc-donated/firehose/
>
> The corresponding README documenting in more detail is here:
>
> http://people.apache.org/~minfrin/bbc-donated/mod_firehose/README
>
> The code itself is here:
>
> http://people.apache.org/~minfrin/bbc-donated/mod_firehose/mod_firehose.c
> http://people.apache.org/~minfrin/bbc-donated/firehose/firehose.c
>
> Obviously the expectation is for the documentation to be completed and fleshed out.
>
> [1] https://issues.apache.org/bugzilla/show_bug.cgi?id=52322
>
> Regards,
> Graham
> --
Re: Proposal: adoption of mod_firehose subproject
Posted by Graham Leggett <mi...@sharp.fm>.
On 13 Dec 2011, at 10:37 PM, Christian Folini wrote:
> One thing you can not do with mod_security, though, is
> the following: To log encrypted connections between a
> reverse proxy and the backend applications.
>
> So far it is very hard to prove the Apache proxy
> sends the right stuff if you can not get hold of the
> backend application's logs.
>
> Now I wonder if mod_firehose could solve this problem
> too.
Yes, it can using the FirehoseProxyConnectionInput and FirehoseProxyConnectionOutput options described in http://people.apache.org/~minfrin/bbc-donated/mod_firehose/README.
Regards,
Graham
--
Re: Proposal: adoption of mod_firehose subproject
Posted by Christian Folini <ch...@netnea.com>.
Graham,
Mod_firehose sounds very helpful. I like the record/replay
options. It would be great if you could convince the
developers.
It is possible to do similar stuff with mod_security,
though not in a very easy way, but mod_security still
helps for debugging purposes.
One thing you can not do with mod_security, though, is
the following: To log encrypted connections between a
reverse proxy and the backend applications.
So far it is very hard to prove the Apache proxy
sends the right stuff if you can not get hold of the
backend application's logs.
Now I wonder if mod_firehose could solve this problem
too.
Regards,
Christian Folini
--
First you make it, then it works, then you invite people to
make it better.
-- Eben Moglen, Free Software Foundation
Re: Proposal: adoption of mod_firehose subproject
Posted by Sander Temme <sc...@apache.org>.
On Dec 13, 2011, at 7:19 AM, Graham Leggett wrote:
> - mod_firehose: "tcpdump for httpd"
+1 on adopting.
S.
--
sctemme@apache.org http://www.temme.net/sander/
PGP FP: FC5A 6FC6 2E25 2DFD 8007 EE23 9BB8 63B0 F51B B88A
View my availability: http://tungle.me/sctemme
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Daniel Ruggeri <DR...@primary.net>.
On 3/1/2012 12:11 PM, William A. Rowe Jr. wrote:
> A proposal to adopt mod_firehose is attached.
>
> [X] Option 1: adopt as trunk module
> [ ] Option 2: adopt only as subproject
> [ ] Option 3: do not adopt
+1 to adopt as experimental module in trunk. More discussion should
follow about the in/out format, though (not in this thread, of course).
--
Daniel Ruggeri
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 3/1/2012 12:40 PM, Sander Temme wrote:
>
> Dimpled chad: I would support option 2 if 1 doesn't have traction.
Yup - that's implicit.
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Sander Temme <sc...@apache.org>.
On Mar 1, 2012, at 10:11 AM, William A. Rowe Jr. wrote:
> Let's simply reset this whole mess.
>
> A proposal to adopt mod_firehose is attached.
>
> [X] Option 1: adopt as trunk module
> [ ] Option 2: adopt only as subproject
> [ ] Option 3: do not adopt
Dimpled chad: I would support option 2 if 1 doesn't have traction.
S.
--
sctemme@apache.org http://www.temme.net/sander/
PGP FP: FC5A 6FC6 2E25 2DFD 8007 EE23 9BB8 63B0 F51B B88A
View my availability: http://tungle.me/sctemme
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 3/1/2012 12:11 PM, William A. Rowe Jr. wrote:
>
> A proposal to adopt mod_firehose is attached.
>
> [ ] Option 1: adopt as trunk module
> [X] Option 2: adopt only as subproject
> [ ] Option 3: do not adopt
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 3/1/2012 12:11 PM, William A. Rowe Jr. wrote:
>
> A proposal to adopt mod_firehose is attached.
>
> [ ] Option 1: adopt as trunk module
> [ ] Option 2: adopt only as subproject
> [ ] Option 3: do not adopt
72 hours have passed; the firehose module and utility, as committed, are
accepted as a module to httpd trunk by a plurality. Thank you to all who
voted.
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 3/2/2012 2:14 AM, Nick Kew wrote:
>
> mod_firehose meets a need. But my +1 has to be conditional on
> satisfactory integration of the complementary "firehose" utility
> alongside it, presumably in /bin/ .
That obligation is met. minfrin acknowledges you could do more
than what the firehose extraction utility provides, but what is
there exists, and patches are [ALWAYS] welcome.
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Nick Kew <ni...@webthing.com>.
On 1 Mar 2012, at 18:11, William A. Rowe Jr. wrote:
> Let's simply reset this whole mess.
+1 to that!
I think maybe we have some confusion here because attitudes
have evolved over the years, and modules that would once not
have been accepted to trunk are now welcomed there. Maybe
there would be mileage in revisiting other non-trunk modules?
> A proposal to adopt mod_firehose is attached.
>
> [ ] Option 1: adopt as trunk module
> [ ] Option 2: adopt only as subproject
> [ ] Option 3: do not adopt
Conditional +1 to Option 1.
mod_firehose meets a need. But my +1 has to be conditional on
satisfactory integration of the complementary "firehose" utility
alongside it, presumably in /bin/ .
--
Nick Kew
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Rainer Jung <ra...@kippdata.de>.
On 01.03.2012 19:11, William A. Rowe Jr. wrote:
> Let's simply reset this whole mess.
>
> A proposal to adopt mod_firehose is attached.
>
> [X] Option 1: adopt as trunk module
> [ ] Option 2: adopt only as subproject
> [ ] Option 3: do not adopt
Option 1.
Regards,
Rainer
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Michael Felt <ma...@gmail.com>.
I learned something tonight :)
On Thu, Mar 1, 2012 at 11:37 PM, Greg Stein <gs...@gmail.com> wrote:
> On Thu, Mar 1, 2012 at 16:30, Rich Bowen <rb...@rcbowen.com> wrote:
> >...
> > I've often thought that modules like, say, mod_ftp, would have a much
> > greater chance of being successful if they were in trunk rather than it
> > being several additional steps to obtain.
> >
> > I'm +1 to having this in trunk, but am voting based on the community
> > aspects, rather than the technical ones. I figure the technical aspects
> will
> > work themselves out in trunk ... or they won't, and we'll drop it from a
> > release branch.
>
> Exactly. In the subversion project, we always strive to do development
> directly on trunk (rather than branches). Keeping stuff in trunk gives
> it many more eyeballs and testing. New features might be buggy on
> trunk, but "just don't use it yet" is a good response :-)
>
> Cheers,
> -g
>
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Greg Stein <gs...@gmail.com>.
On Thu, Mar 1, 2012 at 16:30, Rich Bowen <rb...@rcbowen.com> wrote:
>...
> I've often thought that modules like, say, mod_ftp, would have a much
> greater chance of being successful if they were in trunk rather than it
> being several additional steps to obtain.
>
> I'm +1 to having this in trunk, but am voting based on the community
> aspects, rather than the technical ones. I figure the technical aspects will
> work themselves out in trunk ... or they won't, and we'll drop it from a
> release branch.
Exactly. In the subversion project, we always strive to do development
directly on trunk (rather than branches). Keeping stuff in trunk gives
it many more eyeballs and testing. New features might be buggy on
trunk, but "just don't use it yet" is a good response :-)
Cheers,
-g
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Mar 1, 2012, at 4:30 PM, Rich Bowen wrote:
>
> I've often thought that modules like, say, mod_ftp, would have a much greater chance of being successful if they were in trunk rather than it being several additional steps to obtain.
>
Yeppers.
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Rich Bowen <rb...@rcbowen.com>.
On Mar 1, 2012, at 4:02 PM, Greg Stein wrote:
> Modules do not have to be tested *before* they appear in trunk. That's
> putting the cart before the horse. Part of the development process
> (while in trunk) is doing the testing portion. And hey... if it never
> gets tested, then it gets marked as "experimental" and we all move on.
This is why I'm not understanding why this particular module (or set of modules) is getting this level of debate and scrutiny. We're talking about adding them to trunk, not in a release. Presumably we wouldn't put them in a release if there was a problem with them.
I've often thought that modules like, say, mod_ftp, would have a much greater chance of being successful if they were in trunk rather than it being several additional steps to obtain.
I'm +1 to having this in trunk, but am voting based on the community aspects, rather than the technical ones. I figure the technical aspects will work themselves out in trunk ... or they won't, and we'll drop it from a release branch.
--
Rich Bowen
rbowen@rcbowen.com :: @rbowen
rbowen@apache.org
Re: removal of mod_noloris?
Posted by Igor Galić <i....@brainsware.org>.
----- Original Message -----
>
> On Mar 5, 2012, at 12:01 PM, William A. Rowe Jr. wrote:
>
> > On 3/2/2012 2:16 AM, Nick Kew wrote:
> >>
> >> Mod_noloris was a quick&dirty fix to a rather serious problem. It
> >> was
> >> superseded when Stefan produced a better fix, so there's no
> >> expectation now that mod_noloris will ever 'graduate'. I don't
> >> think
> >> that's a model for most incoming modules!
> >
> > In that case, is it time for mod_noloris be axed from the httpd
> > tree?
>
> +1
+1
--
Igor Galić
Tel: +43 (0) 664 886 22 883
Mail: i.galic@brainsware.org
URL: http://brainsware.org/
GPG: 6880 4155 74BD FD7C B515 2EA5 4B1D 9E08 A097 C9AE
Re: removal of mod_noloris?
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Mar 5, 2012, at 12:01 PM, William A. Rowe Jr. wrote:
> On 3/2/2012 2:16 AM, Nick Kew wrote:
>>
>> Mod_noloris was a quick&dirty fix to a rather serious problem. It was
>> superseded when Stefan produced a better fix, so there's no
>> expectation now that mod_noloris will ever 'graduate'. I don't think
>> that's a model for most incoming modules!
>
> In that case, is it time for mod_noloris be axed from the httpd tree?
+1
Re: removal of mod_noloris?
Posted by Jeff Trawick <tr...@gmail.com>.
On Mon, Mar 5, 2012 at 12:11 PM, Eric Covener <co...@gmail.com> wrote:
> On Mon, Mar 5, 2012 at 12:01 PM, William A. Rowe Jr.
> <wr...@rowe-clan.net> wrote:
>> On 3/2/2012 2:16 AM, Nick Kew wrote:
>>>
>>> Mod_noloris was a quick&dirty fix to a rather serious problem. It was
>>> superseded when Stefan produced a better fix, so there's no
>>> expectation now that mod_noloris will ever 'graduate'. I don't think
>>> that's a model for most incoming modules!
>>
>> In that case, is it time for mod_noloris be axed from the httpd tree?
>
>
> +1
+1
Re: removal of mod_noloris?
Posted by Eric Covener <co...@gmail.com>.
On Mon, Mar 5, 2012 at 12:01 PM, William A. Rowe Jr.
<wr...@rowe-clan.net> wrote:
> On 3/2/2012 2:16 AM, Nick Kew wrote:
>>
>> Mod_noloris was a quick&dirty fix to a rather serious problem. It was
>> superseded when Stefan produced a better fix, so there's no
>> expectation now that mod_noloris will ever 'graduate'. I don't think
>> that's a model for most incoming modules!
>
> In that case, is it time for mod_noloris be axed from the httpd tree?
+1
Re: removal of mod_noloris?
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 3/6/2012 5:18 PM, Igor Galić wrote:
>
>
> ----- Original Message -----
>> On 06 Mar 2012, at 4:46 PM, Jeff Trawick wrote:
>>
>>>> I'm fine with that.
>>>>
>>>> What about the folks who voted to keep it last time the subject
>>>> arose?
>>>> http://www.gossamer-threads.com/lists/apache/users/383407
>>>> http://www.gossamer-threads.com/lists/apache/dev/383405
>>>
>>> Do we just need to offer Dirk an alternate URL for downloading it?
>>
>> How about some kind of "attic" for old modules or old code? Something
>> like this:
>>
>> trunk
>> tags
>> branches
>> attic
>
> Can we put 1.3 there too ;)
:)
If there will be no further changes, svn rm is our friend. Simply publish
a pointer to repos@rev if you want simple access to the non-editable resource.
The only reason to leave a file active in svn is for purpose of collaboration
and modification. Nick stated mod_noloris has been absorbed by a replacement
effort and offers nothing further for the project.
Re: removal of mod_noloris?
Posted by Igor Galić <i....@brainsware.org>.
----- Original Message -----
> On 06 Mar 2012, at 4:46 PM, Jeff Trawick wrote:
>
> >> I'm fine with that.
> >>
> >> What about the folks who voted to keep it last time the subject
> >> arose?
> >> http://www.gossamer-threads.com/lists/apache/users/383407
> >> http://www.gossamer-threads.com/lists/apache/dev/383405
> >
> > Do we just need to offer Dirk an alternate URL for downloading it?
>
> How about some kind of "attic" for old modules or old code? Something
> like this:
>
> trunk
> tags
> branches
> attic
Can we put 1.3 there too ;)
> Regards,
> Graham
> --
>
>
--
Igor Galić
Tel: +43 (0) 664 886 22 883
Mail: i.galic@brainsware.org
URL: http://brainsware.org/
GPG: 6880 4155 74BD FD7C B515 2EA5 4B1D 9E08 A097 C9AE
Re: removal of mod_noloris?
Posted by Graham Leggett <mi...@sharp.fm>.
On 06 Mar 2012, at 4:46 PM, Jeff Trawick wrote:
>> I'm fine with that.
>>
>> What about the folks who voted to keep it last time the subject arose?
>> http://www.gossamer-threads.com/lists/apache/users/383407
>> http://www.gossamer-threads.com/lists/apache/dev/383405
>
> Do we just need to offer Dirk an alternate URL for downloading it?
How about some kind of "attic" for old modules or old code? Something like this:
trunk
tags
branches
attic
Regards,
Graham
--
Re: removal of mod_noloris?
Posted by Jeff Trawick <tr...@gmail.com>.
On Tue, Mar 6, 2012 at 9:26 AM, Nick Kew <ni...@webthing.com> wrote:
> On Mon, 05 Mar 2012 11:01:50 -0600
> "William A. Rowe Jr." <wr...@rowe-clan.net> wrote:
>
>> On 3/2/2012 2:16 AM, Nick Kew wrote:
>> >
>> > Mod_noloris was a quick&dirty fix to a rather serious problem. It was
>> > superseded when Stefan produced a better fix, so there's no
>> > expectation now that mod_noloris will ever 'graduate'. I don't think
>> > that's a model for most incoming modules!
>>
>> In that case, is it time for mod_noloris be axed from the httpd tree?
>
>
> I'm fine with that.
>
> What about the folks who voted to keep it last time the subject arose?
> http://www.gossamer-threads.com/lists/apache/users/383407
> http://www.gossamer-threads.com/lists/apache/dev/383405
Do we just need to offer Dirk an alternate URL for downloading it?
Re: removal of mod_noloris?
Posted by Nick Kew <ni...@webthing.com>.
On Mon, 05 Mar 2012 11:01:50 -0600
"William A. Rowe Jr." <wr...@rowe-clan.net> wrote:
> On 3/2/2012 2:16 AM, Nick Kew wrote:
> >
> > Mod_noloris was a quick&dirty fix to a rather serious problem. It was
> > superseded when Stefan produced a better fix, so there's no
> > expectation now that mod_noloris will ever 'graduate'. I don't think
> > that's a model for most incoming modules!
>
> In that case, is it time for mod_noloris be axed from the httpd tree?
I'm fine with that.
What about the folks who voted to keep it last time the subject arose?
http://www.gossamer-threads.com/lists/apache/users/383407
http://www.gossamer-threads.com/lists/apache/dev/383405
--
Nick Kew
removal of mod_noloris?
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 3/2/2012 2:16 AM, Nick Kew wrote:
>
> Mod_noloris was a quick&dirty fix to a rather serious problem. It was
> superseded when Stefan produced a better fix, so there's no
> expectation now that mod_noloris will ever 'graduate'. I don't think
> that's a model for most incoming modules!
In that case, is it time for mod_noloris be axed from the httpd tree?
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Nick Kew <ni...@webthing.com>.
On 1 Mar 2012, at 21:27, William A. Rowe Jr. wrote:
> On 3/1/2012 3:02 PM, Greg Stein wrote:
>> Modules do not have to be tested *before* they appear in trunk. That's
>> putting the cart before the horse. Part of the development process
>> (while in trunk) is doing the testing portion. And hey... if it never
>> gets tested, then it gets marked as "experimental" and we all move on.
>
> In fact there is an modules/experimental/ tree; mod_noloris is currently
> one such module.
Mod_noloris was a quick&dirty fix to a rather serious problem. It was
superseded when Stefan produced a better fix, so there's no
expectation now that mod_noloris will ever 'graduate'. I don't think
that's a model for most incoming modules!
--
Nick Kew
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 3/1/2012 3:02 PM, Greg Stein wrote:
> Modules do not have to be tested *before* they appear in trunk. That's
> putting the cart before the horse. Part of the development process
> (while in trunk) is doing the testing portion. And hey... if it never
> gets tested, then it gets marked as "experimental" and we all move on.
In fact there is an modules/experimental/ tree; mod_noloris is currently
one such module. Of course, "if it never gets tested" is a handwave.
There's obviously no way for the pmc to assert committers have tested it.
The only filter is the acceptance vote.
This submitted module was not committed to modules/experimental/, but
rather in modules/debugging/
If your desire was to mark this module experimental, you may want to
refine your vote to be more explicit.
As a diagnostic tool this module is toxic in resource consumption, so
in theory, any bugs in this tool are unlikely to cause more pain than
using the tool in the first place.
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Greg Stein <gs...@gmail.com>.
Modules do not have to be tested *before* they appear in trunk. That's
putting the cart before the horse. Part of the development process
(while in trunk) is doing the testing portion. And hey... if it never
gets tested, then it gets marked as "experimental" and we all move on.
Cheers,
-g
On Thu, Mar 1, 2012 at 15:05, Michael Felt <ma...@gmail.com> wrote:
> Seems dangerous to even comment in this flow - but as I am all about
> thinking "testing" at the moment - is there any thought about how to test
> this. From a packaging point of view I would expect tooling to be able to
> test are "included" functions. As a user I would expect anything in trunk
> (what I would call main) to be guaranteed.
>
> I cannot have an opinion about the reasoning for placing something in, or
> not in "trunk", and I would expect something to at least have gone through
> some sort of testing process - live testing - before committing anything to
> a product/service. Before testing was completed I would only dare speaking
> of an intention to add.
>
> Isnt it something along the lines of: "The proof of the pudding is the
> eating".
>
> To me this is just mod_foo, and as far as I know it has never been tested.
> (If it is already in trunk maybe I have already compiled it and just do not
> know it :p) - and that alone would make me postpone a non-reversible
> decision.
>
> Makes me think of what someone old and wise said to me when I was young: you
> (or she) only has to say Yes, or even (yes) once.
>
>
> On Thu, Mar 1, 2012 at 8:33 PM, Greg Stein <gs...@gmail.com> wrote:
>>
>>
>> On Mar 1, 2012 1:29 PM, "Jim Jagielski" <ji...@jagunet.com> wrote:
>> >
>> >
>> > On Mar 1, 2012, at 1:11 PM, William A. Rowe Jr. wrote:
>> >
>> > > Let's simply reset this whole mess.
>> > >
>> > > A proposal to adopt mod_firehose is attached.
>> > >
>> > > [X] Option 1: adopt as trunk module
>> > > [ ] Option 2: adopt only as subproject
>> > > [ ] Option 3: do not adopt
>>
>> +1 for Option 1.
>
>
Re: TESTING of mod_firehose MODULE
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 3/1/2012 2:05 PM, Michael Felt wrote:
> Seems dangerous to even comment in this flow - but as I am all about thinking "testing" at
> the moment - is there any thought about how to test this. From a packaging point of view I
> would expect tooling to be able to test are "included" functions. As a user I would expect
> anything in trunk (what I would call main) to be guaranteed.
Changing the subject (please do this yourself next time)...
1. it is in trunk; if you've checked out trunk you likely compiled it and
have bin/firehose and modules/mod_firehose.so - and can review the doc page.
2. if it weren't in trunk, the proposer included links to the module. Just
drop it in and review it.
3. it's open source. The httpd legacy releases, current stable release and
trunk development branch come with exactly one guarantee - if it's broke,
you have all the pieces.
Maybe I'm not understanding your question?
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Michael Felt <ma...@gmail.com>.
Seems dangerous to even comment in this flow - but as I am all about
thinking "testing" at the moment - is there any thought about how to test
this. From a packaging point of view I would expect tooling to be able to
test are "included" functions. As a user I would expect anything in trunk
(what I would call main) to be guaranteed.
I cannot have an opinion about the reasoning for placing something in, or
not in "trunk", and I would expect something to at least have gone through
some sort of testing process - live testing - before committing anything to
a product/service. Before testing was completed I would only dare speaking
of an intention to add.
Isnt it something along the lines of: "The proof of the pudding is the
eating".
To me this is just mod_foo, and as far as I know it has never been tested.
(If it is already in trunk maybe I have already compiled it and just do not
know it :p) - and that alone would make me postpone a non-reversible
decision.
Makes me think of what someone old and wise said to me when I was young:
you (or she) only has to say Yes, or even (yes) once.
On Thu, Mar 1, 2012 at 8:33 PM, Greg Stein <gs...@gmail.com> wrote:
>
> On Mar 1, 2012 1:29 PM, "Jim Jagielski" <ji...@jagunet.com> wrote:
> >
> >
> > On Mar 1, 2012, at 1:11 PM, William A. Rowe Jr. wrote:
> >
> > > Let's simply reset this whole mess.
> > >
> > > A proposal to adopt mod_firehose is attached.
> > >
> > > [X] Option 1: adopt as trunk module
> > > [ ] Option 2: adopt only as subproject
> > > [ ] Option 3: do not adopt
>
> +1 for Option 1.
>
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Greg Stein <gs...@gmail.com>.
On Mar 1, 2012 1:29 PM, "Jim Jagielski" <ji...@jagunet.com> wrote:
>
>
> On Mar 1, 2012, at 1:11 PM, William A. Rowe Jr. wrote:
>
> > Let's simply reset this whole mess.
> >
> > A proposal to adopt mod_firehose is attached.
> >
> > [X] Option 1: adopt as trunk module
> > [ ] Option 2: adopt only as subproject
> > [ ] Option 3: do not adopt
+1 for Option 1.
Re: [RE-VOTE] adoption of mod_firehose MODULE
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Mar 1, 2012, at 1:11 PM, William A. Rowe Jr. wrote:
> Let's simply reset this whole mess.
>
> A proposal to adopt mod_firehose is attached.
>
> [X] Option 1: adopt as trunk module
> [ ] Option 2: adopt only as subproject
> [ ] Option 3: do not adopt
>
[RE-VOTE] adoption of mod_firehose MODULE
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
Let's simply reset this whole mess.
A proposal to adopt mod_firehose is attached.
[ ] Option 1: adopt as trunk module
[ ] Option 2: adopt only as subproject
[ ] Option 3: do not adopt
[Prior to this vote, option 2 had previously passed with minfrin, issac,
sctemme, jim in support. Subsequently, wrowe endorsed option 2, while
minfrin, jim introduced option 1. Please vote.]
On 12/13/2011 9:19 AM, Graham Leggett wrote:
> Hi all,
>
> I have concluded negotiation with the BBC to open source some httpd modules that I wrote under the AL, and the BBC have very kindly agreed to donate the code to the ASF[1], which I believe would fit well as subprojects of httpd, and would like to know whether httpd would accept these.
>
> To be clear, this isn't a "code dump", my intention is to continue to develop and support this moving forward, and hopefully expand the community around them.
>
> - mod_firehose: "tcpdump for httpd"
>
> Based originally on mod_dumpio.c, mod_firehose is an httpd filter that writes the contents of a request and/or a response to a file or pipe in such a way that the requests can be reconstructed later using a second dedicated tool called "firehose".
>
> It was initially developed to help debug restful services that were secured with client certificates and therefore opaque to other tools like tcpdump or tcpflow, but was then subsequently used to record "dirty traffic" for subsequent replay for the purposes of testing.
>
> The module and the corresponding firehose demultiplexer was used to uncover some of the more tricky bugs in mod_cache, as well as protocol inconsistencies in backend services, and would prove very useful to anyone deploying restful services. We have also intended it to be used to create a "dark live" environment, where live traffic can be split off and diverted to a staging environment to test whether a software update works correctly.
>
> The code is currently packaged as an RPM, wrapped in autotools, and a snapshot is available here:
>
> http://people.apache.org/~minfrin/bbc-donated/mod_firehose/
> http://people.apache.org/~minfrin/bbc-donated/firehose/
>
> The corresponding README documenting in more detail is here:
>
> http://people.apache.org/~minfrin/bbc-donated/mod_firehose/README
>
> The code itself is here:
>
> http://people.apache.org/~minfrin/bbc-donated/mod_firehose/mod_firehose.c
> http://people.apache.org/~minfrin/bbc-donated/firehose/firehose.c
>
> Obviously the expectation is for the documentation to be completed and fleshed out.
>
> [1] https://issues.apache.org/bugzilla/show_bug.cgi?id=52322
>
> Regards,
> Graham
> --
>
>
Re: Proposal: adoption of mod_firehose subproject
Posted by Graham Leggett <mi...@sharp.fm>.
On 13 Dec 2011, at 8:35 PM, William A. Rowe Jr. wrote:
> A silly question perhaps, but was the tcpdump/wireshark file format considered?
> If not, was there a reason to invent a new representational format?
>
> It seems like the functionality you describe, at the httpd-internals visibility,
> emitted in a tcpdump-compatible representation, would be a godsend. All the
> GUI inspection tools already exist.
It wasn't considered no, we were after something that was very simple, was text based so we could open up a capture file with a text viewer and look at the result without necessarily running it through firehose to demux it.
It does sound like a very useful thing to do though. Having looked at the packet format, it seems like all the format types are different layer 2 packets of various kinds, when I'm after something a lot simpler (just request fragments muxed together). Something definitely worth looking into in more depth.
Regards,
Graham
--
Re: Proposal: adoption of mod_firehose subproject
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 12/13/2011 9:19 AM, Graham Leggett wrote:
>
> - mod_firehose: "tcpdump for httpd"
>
> Based originally on mod_dumpio.c, mod_firehose is an httpd filter that writes the contents of a request and/or a response to a file or pipe in such a way that the requests can be reconstructed later using a second dedicated tool called "firehose".
>
> It was initially developed to help debug restful services that were secured with client certificates and therefore opaque to other tools like tcpdump or tcpflow, but was then subsequently used to record "dirty traffic" for subsequent replay for the purposes of testing.
>
> The module and the corresponding firehose demultiplexer was used to uncover some of the more tricky bugs in mod_cache, as well as protocol inconsistencies in backend services, and would prove very useful to anyone deploying restful services. We have also intended it to be used to create a "dark live" environment, where live traffic can be split off and diverted to a staging environment to test whether a software update works correctly.
A silly question perhaps, but was the tcpdump/wireshark file format considered?
If not, was there a reason to invent a new representational format?
It seems like the functionality you describe, at the httpd-internals visibility,
emitted in a tcpdump-compatible representation, would be a godsend. All the
GUI inspection tools already exist.