Re: cvs commit: httpd-test/perl-framework/c-modules/test_pass_brigade mod_test_pass_brigade.c

[Cliff Woolley <jw...@virginia.edu>]

On Wed, 29 Sep 2004 jorton@apache.org wrote:

> jorton      2004/09/29 08:03:59
>
>   Modified:    perl-framework/c-modules/test_pass_brigade
>                         mod_test_pass_brigade.c
>   Log:
>   Prevent death by memory consumption in an --enable-pool-debug/-lefence
>   build: allocate and {ab,re}use a single brigade structure rather than
>   allocating one out of r->pool for each block sent (see also "why are
>   brigades allocated out of pools this is insane" threads).

Um, I actually never saw any such threads.  Where were they?

Anyway, there actually was a brief period of time about two years ago when
they were not allocated out of pools.  I was under the mistaken impression
that that was still the case, but apparently brianp reverted his own
commit about a week afterward for some reason I can no longer recall
(presumably it broke something and it was easier at the time to revert
than fix because a new release of httpd was pending).

Brian, do you remember what the deal was there?

--Cliff

Re: cvs commit: httpd-test/perl-framework/c-modules/test_pass_brigade mod_test_pass_brigade.c

[Joe Orton <jo...@redhat.com>]

On Wed, Sep 29, 2004 at 01:19:24PM -0400, Cliff Woolley wrote:
> On Wed, 29 Sep 2004 jorton@apache.org wrote:
> 
> > jorton      2004/09/29 08:03:59
> >
> >   Modified:    perl-framework/c-modules/test_pass_brigade
> >                         mod_test_pass_brigade.c
> >   Log:
> >   Prevent death by memory consumption in an --enable-pool-debug/-lefence
> >   build: allocate and {ab,re}use a single brigade structure rather than
> >   allocating one out of r->pool for each block sent (see also "why are
> >   brigades allocated out of pools this is insane" threads).
> 
> Um, I actually never saw any such threads.  Where were they?

OK, I paraphrased somewhat, but:

recent: http://marc.theaimsgroup.com/?l=apache-httpd-dev&m=108732600304014&w=2
historic: http://marc.theaimsgroup.com/?l=apache-httpd-dev&m=104039770718467&w=2

> Anyway, there actually was a brief period of time about two years ago when
> they were not allocated out of pools.  I was under the mistaken impression
> that that was still the case, but apparently brianp reverted his own
> commit about a week afterward for some reason I can no longer recall
> (presumably it broke something and it was easier at the time to revert
> than fix because a new release of httpd was pending).

The fact is that now, we can't fix it in 2.0 because there are output
filters which presume they can apr_brigade_destroy() the brigade passed
in, and there are other filters which presume they can reuse a brigade
which they passed down.

Many of the filters in httpd will allocate new brigades when you pass a
FLUSH bucket up (e.g. PR23567, a slow-running CGI), which leads to
memory usage proportional to number of FLUSH buckets sent etc etc...

joe