Re: filter design

["Roy T. Fielding" <fi...@kiwi.ICS.UCI.EDU>]

In message <87...@pobox.com>, Ben Hyde writes:
>
>In a case like this I pesonally would be reluctant to veto in 
>the absense of a significant commitement of time to dive into 
>the issue.

That's basically what I meant.

Voting on anything implies a commitment to work on the project.

For simple coding issues, a veto is fairly easy to define and the
solution is generally apparent from the reason for the veto.  Likewise,
if you simply think that a given piece of functionality should not be in
the server, that is pretty easy to describe and (obviously) wouldn't
require an alternative solution.  Those would be the anti-cruft feature
vetos that have done us very well over time.

In this case, however, we all agree that the functionality is worthwhile
having in the server.  We even agree (mostly) on whereabouts in the server
it should occur.  What we disagree on is the interface for interconnecting
and ordering filters, and quite possibly on the initial scope of what
filters should be able to handle.  The problem is we can't just apply one
patch and then take turns "fixing" the results, since there is nothing
iterative about the bucket brigades interface -- it is all or nothing.

I am a less tolerant of the "wait and see" approach to design than I used
to be.  I still don't like the module hook macros because they make the
run-time interface very hard to understand, but I have little chance of
fixing that at this stage without a version 3.0.  I have a sinking feeling
that adding content filters to the maze will make Apache completely
unstable in real-world environments, but a feeling isn't enough.

And, in reality, there is no need for me to veto anything here.  Greg
and Ryan will think about the issues whether or not there is a pending
veto, and I assume long after the code (in whatever form) is applied.
If there is a performance problem, it is much easier for me to demonstrate
it on a checked-out cvs copy of Apache (or in user complaints) than by
applying one of the proposed patches.  But I owe it to both of them to
explain what I think the issues are up front, rather than just blasting
away at it when we get closer to a gold release.

It's a pity that I can't convince people that the bucket brigades
interface is the only one that works, but it isn't all that surprising.
It took me a year of fooling around in Ada95 before I finally came up
with the right design for HTTP layering. *shrug*   And the only reason
I know it is the right design is because the things I want to do with it
require a 100% solution.  At no time whatsoever, under any circumstances,
can a filter be allowed to lose metadata by denying to others the source
of the content it is passing along.  Allow one ignorant filter in the mix
and you lose the ability for all sources on the server to be treated as
first-class resources.

But, if you start out with the supposition that 2.0 filters will be just
as ignorant as 1.3 content generators when it comes to HTTP, then Greg's
patch might work.  I'm not convinced that the memory allocation works,
or is even remotely efficient, but that is something we can fix later.
The thing I don't think we'll be able to fix later is all of the modules
once they start implementing ignorant filters.  CGI hell, all over again.

....Roy

Re: filter design

[rb...@covalent.net]

> It's a pity that I can't convince people that the bucket brigades
> interface is the only one that works, but it isn't all that surprising.

Roy, you have convinced me.  I actually have the beginning of this patch
written.  I started over about three days ago.  I just want to take it
slow to make sure it is right.  This means the patch could take a few
weeks.  I could post what I have, but it is not even close to working
yet.  The design is coming along however.

> But, if you start out with the supposition that 2.0 filters will be just
> as ignorant as 1.3 content generators when it comes to HTTP, then Greg's
> patch might work.  I'm not convinced that the memory allocation works,
> or is even remotely efficient, but that is something we can fix later.
> The thing I don't think we'll be able to fix later is all of the modules
> once they start implementing ignorant filters.  CGI hell, all over again.

This is my fear too.

Would it help if I posted the VERY incomplete patch I have been working on
recently.  It doesn't touch the core server AT ALL.  All it does currently
is begin to implement rwmem buckets.  I have already re-designed them
three times and I need to clean them up a bit more.  Roy, they look more
or less like exactly what you posted last November.

What I really need to add now, are the functions to write directly to
rwmem buffers (although this could wait).  Then, we just modify http_core,
and the first patch is done.

If people would like to see the current code (understanding that it is
VERY much in development, please let me know).  Roy, I would be interested
in knowing if I do understand your bucket brigades, so if you have time to
review two small files, that would be much appreciated.  There are
comments already, but more are still needed.

Ryan

_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------