You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Stefan Fritsch <sf...@sfritsch.de> on 2007/12/03 23:13:46 UTC
Apache memory usage
Hi,
there is still the problem that during a request, many bucket brigades
being created which are only cleaned up after the request is
finished, see
http://issues.apache.org/bugzilla/show_bug.cgi?id=23567 . There was
some discussion about retaining ownership of a brigade when
ap_pass_brigade() is called, and then reusing the brigades [1]. But
this does not seem to be implemented in 2.2, yet.
But I found two locations where the creation of a new brigade could be
avoided:
- In buffer_output()/ap_old_write_filter(), it is possible to keep the
brigade around and reuse it after the next flush.
- In ap_http_chunk_filter(), a new brigade is created for every flush
bucket. But it is not really necessary if the flush bucket is the
last bucket in the brigade.
I don't know yet whether the saving is significant in practice. But it
saves two brigades on every combination of ap_rwrite()/ap_rflush()
and mod_php uses ap_r*() quite a lot.
A patch is attached. It's against 2.2.6 but applies to trunk.
Cheers,
Stefan
[1]
http://mail-archives.apache.org/mod_mbox/httpd-dev/200504.mbox/<9b5d906c83b6650b9512eb9e7ac96be1%40ricilake.net>
Re: Apache memory usage
Posted by Plüm,
Rüdiger,
VF-Group <ru...@vodafone.com>.
> -----Ursprüngliche Nachricht-----
>
> Im Auftrag von Justin Erenkrantz
> Gesendet: Dienstag, 11. Dezember 2007 11:05
> An: dev@httpd.apache.org
> Betreff: Re: Apache memory usage
>
>
> On Dec 10, 2007 10:42 PM, Stefan Fritsch wrote:
> > Yes, that works as well. I wanted to avoid calling
> APR_BRIGADE_PREPEND
> > when the temporary brigade is empty. But I don't really know whether
> > APR_BRIGADE_PREPEND is so expensive that this makes sense. An
> > alternative would be to check this using APR_BRIGADE_EMPTY.
>
> FWIW, as the brigade is a ring (read: dual-linked list), prepend or
> append are both constant-time operations - on the order of maybe 2-3
> instructions.
>
> IOW, it's cheap. -- justin
Additionally, the first thing the macro does is checking whether the
brigade is empty :-).
Regards
Rüdiger
Re: Apache memory usage
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
On Dec 10, 2007 10:42 PM, Stefan Fritsch <sf...@sfritsch.de> wrote:
> Yes, that works as well. I wanted to avoid calling APR_BRIGADE_PREPEND
> when the temporary brigade is empty. But I don't really know whether
> APR_BRIGADE_PREPEND is so expensive that this makes sense. An
> alternative would be to check this using APR_BRIGADE_EMPTY.
FWIW, as the brigade is a ring (read: dual-linked list), prepend or
append are both constant-time operations - on the order of maybe 2-3
instructions.
IOW, it's cheap. -- justin
Re: Apache memory usage
Posted by Plüm,
Rüdiger,
VF-Group <ru...@vodafone.com>.
> -----Ursprüngliche Nachricht-----
> Von: Stefan Fritsch
> Gesendet: Montag, 10. Dezember 2007 22:43
> An: dev@httpd.apache.org
> Betreff: Re: Apache memory usage
>
>
> On Sunday 09 December 2007, Ruediger Pluem wrote:
> > But I think your patch to server/protocol.c can be done much
> > simpler. Can you try the following and let us know if this helps as
> > well:
> >
> >
> > Index: server/protocol.c
> > ===================================================================
> > --- server/protocol.c (Revision 602730)
> > +++ server/protocol.c (Arbeitskopie)
> > @@ -1397,9 +1397,7 @@
> > * can simply insert our buffered data at the front and
> > * pass the whole bundle down the chain.
> > */
> > - APR_BRIGADE_CONCAT(ctx->bb, bb);
> > - bb = ctx->bb;
> > - ctx->bb = NULL;
> > + APR_BRIGADE_PREPEND(bb, ctx->bb);
> > }
> >
> > return ap_pass_brigade(f->next, bb);
> >
>
> Yes, that works as well. I wanted to avoid calling
> APR_BRIGADE_PREPEND
> when the temporary brigade is empty. But I don't really know whether
> APR_BRIGADE_PREPEND is so expensive that this makes sense. An
> alternative would be to check this using APR_BRIGADE_EMPTY.
Committed patches for both issues to trunk
http://svn.apache.org/viewvc?view=rev&revision=602735
http://svn.apache.org/viewvc?view=rev&revision=603227
and proposed them for backport
http://svn.apache.org/viewvc?view=rev&revision=603231
Regards
Rüdiger
Re: Apache memory usage
Posted by Stefan Fritsch <sf...@sfritsch.de>.
On Sunday 09 December 2007, Ruediger Pluem wrote:
> But I think your patch to server/protocol.c can be done much
> simpler. Can you try the following and let us know if this helps as
> well:
>
>
> Index: server/protocol.c
> ===================================================================
> --- server/protocol.c (Revision 602730)
> +++ server/protocol.c (Arbeitskopie)
> @@ -1397,9 +1397,7 @@
> * can simply insert our buffered data at the front and
> * pass the whole bundle down the chain.
> */
> - APR_BRIGADE_CONCAT(ctx->bb, bb);
> - bb = ctx->bb;
> - ctx->bb = NULL;
> + APR_BRIGADE_PREPEND(bb, ctx->bb);
> }
>
> return ap_pass_brigade(f->next, bb);
>
Yes, that works as well. I wanted to avoid calling APR_BRIGADE_PREPEND
when the temporary brigade is empty. But I don't really know whether
APR_BRIGADE_PREPEND is so expensive that this makes sense. An
alternative would be to check this using APR_BRIGADE_EMPTY.
Thanks for the review.
Cheers,
Stefan
Re: Apache memory usage
Posted by Ruediger Pluem <rp...@apache.org>.
On 12/09/2007 10:03 PM, Stefan Fritsch wrote:
> Hi,
>
> On Monday 03 December 2007, Stefan Fritsch wrote:
>> But I found two locations where the creation of a new brigade could
>> be avoided:
>>
>> - In buffer_output()/ap_old_write_filter(), it is possible to keep
>> the brigade around and reuse it after the next flush.
>>
>> - In ap_http_chunk_filter(), a new brigade is created for every
>> flush bucket. But it is not really necessary if the flush bucket is
>> the last bucket in the brigade.
>>
>> I don't know yet whether the saving is significant in practice. But
>> it saves two brigades on every combination of
>> ap_rwrite()/ap_rflush() and mod_php uses ap_r*() quite a lot.
>>
>> A patch is attached. It's against 2.2.6 but applies to trunk.
>
> I now got a report that this patch helped in one case where the system
> used to run out of memory pretty quickly [1]. Could someone please
> review the patch? Maybe it's not too late for 2.2.7 yet?
Your first patch to chunk_filter.c seems reasonable to me (committed
in r602735, http://svn.apache.org/viewvc?view=rev&revision=602735 to trunk).
Thanks for this good catch.
But I think your patch to server/protocol.c can be done much simpler.
Can you try the following and let us know if this helps as well:
Index: server/protocol.c
===================================================================
--- server/protocol.c (Revision 602730)
+++ server/protocol.c (Arbeitskopie)
@@ -1397,9 +1397,7 @@
* can simply insert our buffered data at the front and
* pass the whole bundle down the chain.
*/
- APR_BRIGADE_CONCAT(ctx->bb, bb);
- bb = ctx->bb;
- ctx->bb = NULL;
+ APR_BRIGADE_PREPEND(bb, ctx->bb);
}
return ap_pass_brigade(f->next, bb);
Regards
Rüdiger
Re: Apache memory usage
Posted by Stefan Fritsch <sf...@sfritsch.de>.
Hi,
On Monday 03 December 2007, Stefan Fritsch wrote:
> But I found two locations where the creation of a new brigade could
> be avoided:
>
> - In buffer_output()/ap_old_write_filter(), it is possible to keep
> the brigade around and reuse it after the next flush.
>
> - In ap_http_chunk_filter(), a new brigade is created for every
> flush bucket. But it is not really necessary if the flush bucket is
> the last bucket in the brigade.
>
> I don't know yet whether the saving is significant in practice. But
> it saves two brigades on every combination of
> ap_rwrite()/ap_rflush() and mod_php uses ap_r*() quite a lot.
>
> A patch is attached. It's against 2.2.6 but applies to trunk.
I now got a report that this patch helped in one case where the system
used to run out of memory pretty quickly [1]. Could someone please
review the patch? Maybe it's not too late for 2.2.7 yet?
Cheers,
Stefan
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=421557