more on events and parsing (was: Re: PLEASE READ: Filter I/O)

[Greg Stein <gs...@lyra.org>]

On Mon, Jun 26, 2000 at 05:28:11AM +0100, Tony Finch wrote:
>...
> Um. That's incoherent. I meant something more like:
> 
> > I think this is also true for linked filters: if a filter is passed
> > a block of data that's unparsable because a syntactic element is
> > missing its end (presumably because the end will be passed down in
> > the next chunk) then the filter will have to hold back the start of
> > the syntactic element for next time. The link scheme ends up with
> > exactly the same problems that the hook scheme is designed to
> > address from the start with its event-driven design. The existing
> > link scheme will have to aggregate blocks of data in order to fix
> > this problem which implies copying.

Per my other note: I'm not going to attempt to refute the "partial input"
problem. That exists. No question. The issue is with the "partial output"
which would be needed to avoid the "put it all on the heap" problem.

> For an example of holding back part of a syntactic element for
> processing next time around, see Greg's version of mod_gargle which
> has space for a byte in its private data area. More complicated
> filters will need more complicated holding spaces and will end up
> re-re-re-implementing the iovec-like-thing that the hook scheme is
> built on.

All that needs to be saved is a parse state and a string of text
representing the partial, unparsed token. An iovec is not necessary -- we're
talking about unused textual input. I might grant that somebody could come
up with a scheme that might need something more complicated, but I'm at a
loss to figure out what that might be. A filter takes text, transforms it,
and outputs text. The iovec is simple an optimized way of passing that text
along. When it is realized that you have a partial parse token, then you
save the partial token, remember your parse state, and return [and await to
be called again].

Regardless... as I mentioned in the other note, consider the following code:


{
    char buf[100];
    int n;

    n = sprintf(buf, "...", ...);
    ap_rwrite(buf, n, r);
}

In the above code, both filter schemes will need to copy "buf" to the heap
for later usage.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/