Re: cvs commit: apr-serf/docs notes-filter-chains.txt

[Aaron Bannert <aa...@clove.org>]

On Wed, Aug 28, 2002 at 11:45:39AM -0000, gstein@apache.org wrote:
> gstein      2002/08/28 04:45:39
> 
>   Added:       docs     notes-filter-chains.txt
>   Log:
>   A transcript of a conversation that Justin and I just had on IRC. This
>   still needs to be massaged down to its key points.

[originally only sent to gstein by accident, this is a resend to the
list... stupid reply-to headers... :) ]

It's going to take awhile to catch up on all this, but one quick comment:       
                                                                                
Filter interaction should not think in terms of bytes.

The app should not request any thing in terms of "bytes" it should
request a type of data. "Give me body data" or "Give me abstract things
until one of them can be identifed as a body data thingy" or something
along those lines. This rids us of the need to deal with "spillover"
or any other buffering problems at the filter level, so we only have to
think about it in terms of specifical data types.

Only once a filter has received a type of data that it knows about should
it inspect that unit, and if it happens to be a type of data that can
be thought of in terms of bytes than that's fine, but only at that
deeper level. Am I making any sense here?

-aaron

Re: cvs commit: apr-serf/docs notes-filter-chains.txt

[Greg Stein <gs...@lyra.org>]

On Wed, Aug 28, 2002 at 04:24:48PM -0700, Justin Erenkrantz wrote:
> On Wed, Aug 28, 2002 at 12:08:59PM -0700, Aaron Bannert wrote:
> > Filter interaction should not think in terms of bytes.
> 
> Push-based filters can't think like that.  We don't know what the
> end result is going to be.  All we know is that we received something
> from the socket - we churn it through the filters, and something
> comes out on the other end.

Right.

> The key is here is all of the filters are push - not pull.  A pull
> is merely a call to special code that triggers the socket to go read
> and push data down the pipe (if it doesn't have any data in its
> 'spillage').  Hopefully, something comes out on the other end.
> 
> Notice the bit at the end where Greg proposes that each socket
> *read* gets its own independent thread.  Now that I think about it,
> what happens if we do a pull and not enough data is generated to
> make it through to the end of the filters?  Eek.  That pull call
> returns nothing?

Oh no... the serf_pull_data() would just keep telling the socket to keep
shoving data until it got enough spilled out the end.

> Specifically to what you were thinking about, I think we could have
> a HTTP protocol filter that morphs from the bytes to the
> HTTP-specific entities.  However, if we generalize for a second,
> pull calls must also offer a byte-oriented API to help those filters
> that need byte-orientation (think SSL).
> 
> So, it sounds like we need a hybrid approach.  An API could be
> provided that allows saying, "Gimme the entire body" or "Gimme the
> headers" as well as "Gimme X bytes."
> 
> In fact, I wonder if we have two types of filter chains.  One that
> has the byte-orientation (for SSL et al - connection-level) and
> another for the entity-orientation (once the protocol is fixed and
> determined).
> 
> The bottom-half is byte-oriented, and the top-half is
> entity-oriented.  -- justin

Nah. One request type is simply byte-oriented.

In any case, I think the entity-based approach needs some more Aaron-thought
(see my other email).

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

Re: cvs commit: apr-serf/docs notes-filter-chains.txt

[Justin Erenkrantz <je...@apache.org>]

On Wed, Aug 28, 2002 at 12:08:59PM -0700, Aaron Bannert wrote:
> Filter interaction should not think in terms of bytes.

Push-based filters can't think like that.  We don't know what the
end result is going to be.  All we know is that we received something
from the socket - we churn it through the filters, and something
comes out on the other end.

The key is here is all of the filters are push - not pull.  A pull
is merely a call to special code that triggers the socket to go read
and push data down the pipe (if it doesn't have any data in its
'spillage').  Hopefully, something comes out on the other end.

Notice the bit at the end where Greg proposes that each socket
*read* gets its own independent thread.  Now that I think about it,
what happens if we do a pull and not enough data is generated to
make it through to the end of the filters?  Eek.  That pull call
returns nothing?

Specifically to what you were thinking about, I think we could have
a HTTP protocol filter that morphs from the bytes to the
HTTP-specific entities.  However, if we generalize for a second,
pull calls must also offer a byte-oriented API to help those filters
that need byte-orientation (think SSL).

So, it sounds like we need a hybrid approach.  An API could be
provided that allows saying, "Gimme the entire body" or "Gimme the
headers" as well as "Gimme X bytes."

In fact, I wonder if we have two types of filter chains.  One that
has the byte-orientation (for SSL et al - connection-level) and
another for the entity-orientation (once the protocol is fixed and
determined).

The bottom-half is byte-oriented, and the top-half is
entity-oriented.  -- justin