You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Paul Querna <ch...@force-elite.com> on 2005/02/18 10:12:52 UTC

Use of brigade_split...

First, let me say that apr_brigade_split is evil.

It is used in several filters to break up input, or temporarily store 
some buckets. The problem is that it allocates a new brigade, commonly 
out of the request pool.

This is fine, if you only call it once, but for streaming filters, where 
your filter might be called hundreds of times, it will leak a brigade 
every time.

This was the cause of the memory 'leak' in PR 33382 (Proxying Windows 
Media Player Streams).  With just a quick look in the source, I believe 
there is also a bug in the mod_deflate input filter. Consider the code 
in mod_deflate.c line 813:

     if (!APR_BRIGADE_EMPTY(ctx->proc_bb)) {
         apr_bucket_brigade *newbb;

         /* May return APR_INCOMPLETE which is fine by us. */
         apr_brigade_partition(ctx->proc_bb, readbytes, &bkt);

         newbb = apr_brigade_split(ctx->proc_bb, bkt);
         APR_BRIGADE_CONCAT(bb, ctx->proc_bb);
         APR_BRIGADE_CONCAT(ctx->proc_bb, newbb);
     }

While this code works, it does not provide any reuse of the bucket 
brigade *newbb. The inflate filter will effectively leak one brigade 
every time it is called.  To fix it, a brigade should be added to the 
filter context, and recycled. Instead of using brigade_split, the 
buckets should be moved to this persistent brigade.

I count 11 [1] uses of apr_brigade_split in the trunk source tree.  If 
anyone has some spare time, I believe many of them will reveal that they 
  have the possibility to leak too.  I will try to investigate them all 
sooner or later, but some help on this hunt would be nice :)

-Paul

[1]
./modules/filters/mod_deflate.c:819
./modules/http/chunk_filter.c:89
./server/core_filters.c:187
./server/core_filters.c:627
./server/core_filters.c:644
./server/core_filters.c:661
./server/core_filters.c:677
./server/core_filters.c:697
./server/core_filters.c:762
./server/mpm/experimental/perchild/perchild.c:1572
./server/protocol.c:1201

Re: Use of brigade_split...

Posted by Cliff Woolley <jw...@virginia.edu>.
On Fri, 18 Feb 2005, Paul Querna wrote:

> First, let me say that apr_brigade_split is evil.
>
> It is used in several filters to break up input, or temporarily store
> some buckets. The problem is that it allocates a new brigade, commonly
> out of the request pool.

Feel free to write a variant that (also) takes as a parameter the brigade
into which to split the second half.  :)

Re: Use of brigade_split...

Posted by Joe Orton <jo...@redhat.com>.
On Fri, Feb 18, 2005 at 01:12:52AM -0800, Paul Querna wrote:
> First, let me say that apr_brigade_split is evil.

Specifically the combination of apr_brigade_split() and an
apr_brigade_destroy() which is a noop, yes; #23567 is another example of
a leak caused by this.

I'd also concluded that eliminating use of apr_brigade_split() was
probably the easiest way out of this hole, it's still fairly disruptive
though... 

joe