Re: [PATCH] allocate properly-sized buffer for header

["Roy T. Fielding" <fi...@ebuilt.com>]

> > Can you put in words and some simple examples what's wrong with the API?
> 
> The problem is that it is impossible to do a lot of small writes to a
> brigade efficiently.  Take as an example, code that needs to add 20 five
> byte buffers to brigade.  There are two options:
> 
> 1)  bigbuff = apr_pstrcat(p, buf1, buf2, buf3, buf4, buf5, ...);
>     b = apr_bucket_make_pool(p, bigbuff);
>     APR_INSERT_BRIGADE_TAIL(bb, b);
> 
> 2)  b = apr_bucket_make_heap(buf1);
>     APR_INSERT_BRIGADE_TAIL(bb, b);
>     b = apr_bucket_make_heap(buf2);
>     APR_INSERT_BRIGADE_TAIL(bb, b);
>     b = apr_bucket_make_heap(buf3);
>     APR_INSERT_BRIGADE_TAIL(bb, b);
>     ...

Why write to the brigade instead of the top filter?
The problem with this type of optimization is that the brigade code
cannot know the optimal size of a bucket -- only the next
filter in the chain can know, since what is optimal will depend
on what kind of processing is done next.

I think that if we are suffering from wasted cycles and pallocs
due to premature brigade formations, then we should try it the other
way -- always allocate the bucket structure off the stack, use a
simple next pointer to connect brigades, and force the filter that
needs to setaside the data to do so in a way that coalesces the
bucket data.  That was the main difference between the design we
are using now and the one Greg proposed prior to the filters meeting.

....Roy

Re: [PATCH] allocate properly-sized buffer for header

[rb...@covalent.net]

> > > The problem with this type of optimization is that the brigade code
> > > cannot know the optimal size of a bucket -- only the next
> > > filter in the chain can know, since what is optimal will depend
> > > on what kind of processing is done next.
> > 
> > However, the programmer knows what kind of data they are dealing with.  If
> > it is bufferable, then it should be written to the brigade using a
> > buffering API.  Otherwise, it should be written using a direct bucket API,
> > IMO.
> 
> Hmmm... but I don't want to write to the brigade.  I want to write to
> the filter stack just like I would write to any file handle.  All of
> the complexity should be handled within the filter implementations
> and not exposed to the users of the filter, IMO.

Great, you and I agree here.  I have a patch, that am working on, that
uses the apr_brigade_* calls, but those are kept hidden from the
programmer.  Obviously those are exported from apr-util, but the module
writer should never actually use them, unless they absolutely want to,
which most people won't.

> > > I think that if we are suffering from wasted cycles and pallocs
> > > due to premature brigade formations, then we should try it the other
> > > way -- always allocate the bucket structure off the stack, use a
> > > simple next pointer to connect brigades, and force the filter that
> > > needs to setaside the data to do so in a way that coalesces the
> > > bucket data.  That was the main difference between the design we
> > > are using now and the one Greg proposed prior to the filters meeting.
> > 
> > However, we don't always want to coalesce bucket data.  I am picturing
> > this case.
> > 
> > file bucket 9k -> 10 byte pool bucket -> 10 byte pool bucket.
> > 
> > We want to coalesce the 2 10 byte buckets, but we don't want to coalesce
> > the file bucket.  If the buckets are allocated off the stack, how do you
> > keep the file bucket around?
> 
> I don't.  Every buffering mechanism needs a threshold against which
> it writes to the next output (if it isn't blocked by waiting for
> something like an end-of-record) or writing to a large processing
> buffer if it is blocked.  That prevents latency from getting too high,
> and provides the intermediate files that can be identified and
> cached just like a proxy does caching.

Okay, I think I see what you are saying, and I think I see how this
works.  I need to think about it more.  Thanks for the clarification.  I
may implement some small pieces of this in the code I am currently working
on, but not the buckets allocated on the stack.  That will have to wait.

Ryan
_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------

Re: [PATCH] allocate properly-sized buffer for header

["Roy T. Fielding" <fi...@ebuilt.com>]

> > Why write to the brigade instead of the top filter?
> 
> Because we aren't always writing to the top filter.  Sometimes, we are
> writing to a lower filter.  This is the problem with current
> implementation.

I meant the top from the perspective of the writer (i.e., a lower
filter only knows about filters below it).  Sorry for the confusion.

> > The problem with this type of optimization is that the brigade code
> > cannot know the optimal size of a bucket -- only the next
> > filter in the chain can know, since what is optimal will depend
> > on what kind of processing is done next.
> 
> However, the programmer knows what kind of data they are dealing with.  If
> it is bufferable, then it should be written to the brigade using a
> buffering API.  Otherwise, it should be written using a direct bucket API,
> IMO.

Hmmm... but I don't want to write to the brigade.  I want to write to
the filter stack just like I would write to any file handle.  All of
the complexity should be handled within the filter implementations
and not exposed to the users of the filter, IMO.

> > I think that if we are suffering from wasted cycles and pallocs
> > due to premature brigade formations, then we should try it the other
> > way -- always allocate the bucket structure off the stack, use a
> > simple next pointer to connect brigades, and force the filter that
> > needs to setaside the data to do so in a way that coalesces the
> > bucket data.  That was the main difference between the design we
> > are using now and the one Greg proposed prior to the filters meeting.
> 
> However, we don't always want to coalesce bucket data.  I am picturing
> this case.
> 
> file bucket 9k -> 10 byte pool bucket -> 10 byte pool bucket.
> 
> We want to coalesce the 2 10 byte buckets, but we don't want to coalesce
> the file bucket.  If the buckets are allocated off the stack, how do you
> keep the file bucket around?

I don't.  Every buffering mechanism needs a threshold against which
it writes to the next output (if it isn't blocked by waiting for
something like an end-of-record) or writing to a large processing
buffer if it is blocked.  That prevents latency from getting too high,
and provides the intermediate files that can be identified and
cached just like a proxy does caching.

> I have tried to allocate the brigades off the stack, but we rely on the
> cleanups to ensure that buckets are freed properly.
> 
> I guess I don't see what you are thinking.

If the bucket structure is allocated on the stack, then it is cleaned
or reused when the caller returns.  The bucket data would either be
sent out the filter tail (if it suffices for a large write) or
copied to some other buffer as part of the coalescing of small
writes.  We are left with no need for cleanups, aside from the
basic ones for the filter chain and the pools.

This is hard to describe because I am still thinking of
buckets in terms of the abstract model and not the current bucket
code.  In the abstract model, bucket structures are like windows --
they point to segments of data that are managed somewhere else.
They don't really contain anything -- they just provide a sequence
of instructions in how the data stream should be constructed at
whatever point it needs to be constructed (that point is either
when it becomes desirable to write to a non-brigade interface
or when some filter needs to do per-character processing).

....Roy

Re: [PATCH] allocate properly-sized buffer for header

[rb...@covalent.net]

> > The problem is that it is impossible to do a lot of small writes to a
> > brigade efficiently.  Take as an example, code that needs to add 20 five
> > byte buffers to brigade.  There are two options:
> > 
> > 1)  bigbuff = apr_pstrcat(p, buf1, buf2, buf3, buf4, buf5, ...);
> >     b = apr_bucket_make_pool(p, bigbuff);
> >     APR_INSERT_BRIGADE_TAIL(bb, b);
> > 
> > 2)  b = apr_bucket_make_heap(buf1);
> >     APR_INSERT_BRIGADE_TAIL(bb, b);
> >     b = apr_bucket_make_heap(buf2);
> >     APR_INSERT_BRIGADE_TAIL(bb, b);
> >     b = apr_bucket_make_heap(buf3);
> >     APR_INSERT_BRIGADE_TAIL(bb, b);
> >     ...
> 
> Why write to the brigade instead of the top filter?

Because we aren't always writing to the top filter.  Sometimes, we are
writing to a lower filter.  This is the problem with current
implementation.

> The problem with this type of optimization is that the brigade code
> cannot know the optimal size of a bucket -- only the next
> filter in the chain can know, since what is optimal will depend
> on what kind of processing is done next.

However, the programmer knows what kind of data they are dealing with.  If
it is bufferable, then it should be written to the brigade using a
buffering API.  Otherwise, it should be written using a direct bucket API,
IMO.

> 
> I think that if we are suffering from wasted cycles and pallocs
> due to premature brigade formations, then we should try it the other
> way -- always allocate the bucket structure off the stack, use a
> simple next pointer to connect brigades, and force the filter that
> needs to setaside the data to do so in a way that coalesces the
> bucket data.  That was the main difference between the design we
> are using now and the one Greg proposed prior to the filters meeting.

However, we don't always want to coalesce bucket data.  I am picturing
this case.

file bucket 9k -> 10 byte pool bucket -> 10 byte pool bucket.

We want to coalesce the 2 10 byte buckets, but we don't want to coalesce
the file bucket.  If the buckets are allocated off the stack, how do you
keep the file bucket around?

I have tried to allocate the brigades off the stack, but we rely on the
cleanups to ensure that buckets are freed properly.

I guess I don't see what you are thinking.

Ryan

_______________________________________________________________________________
Ryan Bloom                        	rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------