You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Stefan Fritsch <sf...@sfritsch.de> on 2009/07/28 19:35:25 UTC
Segfault with fix for CVE-2009-1891
Hi,
I have backported r791454 to 2.2.3 in Debian 4.0 and have received a
report [1] about segfaults with mod_deflate and mod_php (5.2.0). As
far as I understand it, the reason is that mod_php uses ap_rwrite
which creates transient buckets. When the connection is closed by the
client, these buckets sometimes stay in the bucket brigade when
ap_pass_brigade returns an error for the compressed data of an
earlier bucket. If deflate_out_filter gets called again with the same
brigade, the memory of the transient buckets is no longer valid,
causing a segfault.
This patch seems to fix the issue:
--- apache2-2.2.3~/modules/filters/mod_deflate.c
+++ apache2-2.2.3/modules/filters/mod_deflate.c
@@ -512,6 +512,7 @@
APR_BRIGADE_INSERT_TAIL(ctx->bb, bkt);
rv = ap_pass_brigade(f->next, ctx->bb);
if (rv != APR_SUCCESS) {
+ apr_brigade_cleanup(bb);
return rv;
}
continue;
@@ -543,6 +544,7 @@
/* Send what we have right now to the next filter. */
rv = ap_pass_brigade(f->next, ctx->bb);
if (rv != APR_SUCCESS) {
+ apr_brigade_cleanup(bb);
return rv;
}
}
I could not reproduce the segfault with Debian 5.0, containing Apache
2.2.9 and php 5.2.8. The same for Debian unstable with 2.2.12 and php
5.2.10. Therefore, I also tried to take the whole mod_deflate.c from
2.2.9 into 2.2.3, but it did not fix the segfaults [2].
Is there some change from 2.2.3 to 2.2.9 (which cannot be in
mod_deflate.c) that fixes the issue? Or is it just coincidence that
2.2.9 and 2.2.12 do not segfault and should the above patch be
included in the next 2.2.x?
Cheers,
Stefan
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=537665
[2] to reproduce, use this script
<?php
for ($i = 0; $i < 100000; $i++ ) {
echo "blah\n";
echo $i;
}
echo "blubb\n";
and request it with
curl -is -H 'Accept-Encoding: gzip' http://localhost/x.php |head -10
With 2.2.3, I get a segfault for about 30% of the requests.
Re: Segfault with fix for CVE-2009-1891
Posted by Joe Orton <jo...@redhat.com>.
On Mon, Aug 03, 2009 at 01:09:35PM +0200, Ruediger Pluem wrote:
> On 08/03/2009 12:52 PM, Joe Orton wrote:
> > On Tue, Jul 28, 2009 at 07:35:25PM +0200, Stefan Fritsch wrote:
> >> I have backported r791454 to 2.2.3 in Debian 4.0 and have received a
> >> report [1] about segfaults with mod_deflate and mod_php (5.2.0). As
...
> > This sounds exactly like:
> >
> > https://issues.apache.org/bugzilla/show_bug.cgi?id=36780
...
> Yes, but AFAIU Stefan it doesn't happen with 2.2.11 which does not
> contain this patch :-).
PR 36780 is triggered by an output filter returning an error, so, it
doesn't seem surprising (in retrospect!) that r791454 makes it much
easier to trigger - the core output filter will now return errors in
very common cases (an aborted connection).
Regards, Joe
Re: Segfault with fix for CVE-2009-1891
Posted by Ruediger Pluem <rp...@apache.org>.
On 08/03/2009 12:52 PM, Joe Orton wrote:
> On Tue, Jul 28, 2009 at 07:35:25PM +0200, Stefan Fritsch wrote:
>> Hi,
>>
>> I have backported r791454 to 2.2.3 in Debian 4.0 and have received a
>> report [1] about segfaults with mod_deflate and mod_php (5.2.0). As
>> far as I understand it, the reason is that mod_php uses ap_rwrite
>> which creates transient buckets. When the connection is closed by the
>> client, these buckets sometimes stay in the bucket brigade when
>> ap_pass_brigade returns an error for the compressed data of an
>> earlier bucket. If deflate_out_filter gets called again with the same
>> brigade, the memory of the transient buckets is no longer valid,
>> causing a segfault.
>
> This sounds exactly like:
>
> https://issues.apache.org/bugzilla/show_bug.cgi?id=36780
>
> I've proposed the fix for backport:
>
> https://issues.apache.org/bugzilla/attachment.cgi?id=24087
Yes, but AFAIU Stefan it doesn't happen with 2.2.11 which does not
contain this patch :-).
Nevertheless backporting this is good.
Regards
Rüdiger
Re: Segfault with fix for CVE-2009-1891
Posted by Joe Orton <jo...@redhat.com>.
On Tue, Jul 28, 2009 at 07:35:25PM +0200, Stefan Fritsch wrote:
> Hi,
>
> I have backported r791454 to 2.2.3 in Debian 4.0 and have received a
> report [1] about segfaults with mod_deflate and mod_php (5.2.0). As
> far as I understand it, the reason is that mod_php uses ap_rwrite
> which creates transient buckets. When the connection is closed by the
> client, these buckets sometimes stay in the bucket brigade when
> ap_pass_brigade returns an error for the compressed data of an
> earlier bucket. If deflate_out_filter gets called again with the same
> brigade, the memory of the transient buckets is no longer valid,
> causing a segfault.
This sounds exactly like:
https://issues.apache.org/bugzilla/show_bug.cgi?id=36780
I've proposed the fix for backport:
https://issues.apache.org/bugzilla/attachment.cgi?id=24087
Regards, Joe
Re: Segfault with fix for CVE-2009-1891
Posted by Stefan Fritsch <sf...@sfritsch.de>.
>> Right, it is not really helpful, but as you seem to be able to reproduce
>> the issue can you please create a backtrace on your own, preferably with
>> an unstripped and -g compiled php (which doesn't seem to be the case in
>> the
>> current backtrace) .
>
> Backtrace is attached.
I forgot to mention that mod_deflate.c:537 is actually line 524, because I
have applied Rüdiger Plüm's patch for no deflate for HEAD requests.
Re: Segfault with fix for CVE-2009-1891
Posted by Ruediger Pluem <rp...@apache.org>.
On 07/30/2009 10:48 AM, Stefan Fritsch wrote:
>> Right, it is not really helpful, but as you seem to be able to reproduce
>> the issue can you please create a backtrace on your own, preferably with
>> an unstripped and -g compiled php (which doesn't seem to be the case in
>> the
>> current backtrace) .
>
> Backtrace is attached. Looking at it again, the brigade that contains the
> transient bucket does not seem to belong to the oldwrite filter. It's the
> brigade that is used by mod_php to pass the eos bucket. But since mod_php
> only creates eos buckets and uses ap_fwrite for everything else, I don't
> know where the transient bucket comes from.
I guess this is because they reuse the brigade for sending the EOS bucket
(which is fine). They should simply cleanup this brigade before sending
the EOS bucket down the chain.
Regards
Rüdiger
Re: Segfault with fix for CVE-2009-1891
Posted by Stefan Fritsch <sf...@sfritsch.de>.
> Right, it is not really helpful, but as you seem to be able to reproduce
> the issue can you please create a backtrace on your own, preferably with
> an unstripped and -g compiled php (which doesn't seem to be the case in
> the
> current backtrace) .
Backtrace is attached. Looking at it again, the brigade that contains the
transient bucket does not seem to belong to the oldwrite filter. It's the
brigade that is used by mod_php to pass the eos bucket. But since mod_php
only creates eos buckets and uses ap_fwrite for everything else, I don't
know where the transient bucket comes from.
Re: Segfault with fix for CVE-2009-1891
Posted by Ruediger Pluem <rp...@apache.org>.
On 07/29/2009 01:52 PM, Stefan Fritsch wrote:
>
> A backtrace is available in the bug report, but it does not reveal much:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=537665#30
Right, it is not really helpful, but as you seem to be able to reproduce
the issue can you please create a backtrace on your own, preferably with
an unstripped and -g compiled php (which doesn't seem to be the case in the
current backtrace) .
Regards
Rüdiger
Re: Segfault with fix for CVE-2009-1891
Posted by Stefan Fritsch <sf...@sfritsch.de>.
Ruediger Pluem wrote:
>> far as I understand it, the reason is that mod_php uses ap_rwrite
>> which creates transient buckets. When the connection is closed by the
>> client, these buckets sometimes stay in the bucket brigade when
>> ap_pass_brigade returns an error for the compressed data of an
>> earlier bucket. If deflate_out_filter gets called again with the same
>> brigade, the memory of the transient buckets is no longer valid,
>> causing a segfault.
>
> IMHO the mod_deflate filter shouldn't be called with the same brigade
> again or at least the brigade should have been set aside in this case.
> So that should never happen.
I was not really able to figure out, how or why this happens. What I
actually see is that the brigade in mod_deflate contains one heap bucket
and one transient bucket where the latter points to invalid memory.
Therefore I assumed the bucket was around from the last call to
buffer_output. But php does strange things (which I don't understand)
involving longjmp to handle errors, maybe that's got something to do with
it.
> I don't know. Maybe at setaside is now done somewhere were it didn't
> happen before. I guess the best approach is to add the above patch to
> your backport and dig further.
>
> Nevertheless a backtrace of the segfault would help in any case to
> understand
> better what is going on and why it might not happen on more recent
> versions.
A backtrace is available in the bug report, but it does not reveal much:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=537665#30
Re: Segfault with fix for CVE-2009-1891
Posted by Ruediger Pluem <rp...@apache.org>.
On 07/28/2009 07:35 PM, Stefan Fritsch wrote:
> Hi,
>
> I have backported r791454 to 2.2.3 in Debian 4.0 and have received a
> report [1] about segfaults with mod_deflate and mod_php (5.2.0). As
> far as I understand it, the reason is that mod_php uses ap_rwrite
> which creates transient buckets. When the connection is closed by the
> client, these buckets sometimes stay in the bucket brigade when
> ap_pass_brigade returns an error for the compressed data of an
> earlier bucket. If deflate_out_filter gets called again with the same
> brigade, the memory of the transient buckets is no longer valid,
> causing a segfault.
IMHO the mod_deflate filter shouldn't be called with the same brigade
again or at least the brigade should have been set aside in this case.
So that should never happen.
>
> This patch seems to fix the issue:
>
> --- apache2-2.2.3~/modules/filters/mod_deflate.c
> +++ apache2-2.2.3/modules/filters/mod_deflate.c
> @@ -512,6 +512,7 @@
> APR_BRIGADE_INSERT_TAIL(ctx->bb, bkt);
> rv = ap_pass_brigade(f->next, ctx->bb);
> if (rv != APR_SUCCESS) {
> + apr_brigade_cleanup(bb);
> return rv;
> }
> continue;
> @@ -543,6 +544,7 @@
> /* Send what we have right now to the next filter. */
> rv = ap_pass_brigade(f->next, ctx->bb);
> if (rv != APR_SUCCESS) {
> + apr_brigade_cleanup(bb);
> return rv;
> }
> }
>
>
> I could not reproduce the segfault with Debian 5.0, containing Apache
> 2.2.9 and php 5.2.8. The same for Debian unstable with 2.2.12 and php
> 5.2.10. Therefore, I also tried to take the whole mod_deflate.c from
> 2.2.9 into 2.2.3, but it did not fix the segfaults [2].
>
> Is there some change from 2.2.3 to 2.2.9 (which cannot be in
> mod_deflate.c) that fixes the issue? Or is it just coincidence that
> 2.2.9 and 2.2.12 do not segfault and should the above patch be
> included in the next 2.2.x?
I don't know. Maybe at setaside is now done somewhere were it didn't
happen before. I guess the best approach is to add the above patch to
your backport and dig further.
Nevertheless a backtrace of the segfault would help in any case to understand
better what is going on and why it might not happen on more recent versions.
Regards
Rüdiger
Re: Segfault with fix for CVE-2009-1891
Posted by Jeff Trawick <tr...@gmail.com>.
On Tue, Jul 28, 2009 at 6:24 PM, Akins, Brian <Br...@turner.com>wrote:
> On 7/28/09 1:35 PM, "Stefan Fritsch" <sf...@sfritsch.de> wrote:
>
> > I have backported r791454 to 2.2.3 in Debian 4.0 and have received a
> > report [1] about segfaults with mod_deflate and mod_php (5.2.0).
>
> Isn't php only officially supported via fastcgi?
Both mod_php and FastCGI are considered strategic for PHP, according to some
folks intimately involved in the engine. (I asked based on your previous,
similar comment on the list ;) )
Re: Segfault with fix for CVE-2009-1891
Posted by Stefan Fritsch <sf...@sfritsch.de>.
William A. Rowe, Jr. wrote:
> One helpful detail, Stefan, would be if this is worker-specific or can
> be reproduced with prefork. That helps narrow down the number of places
> to consider your question.
This happened with prefork, Debian supports mod_php only with prefork.
> As I recall, we have several filter bug fixes in deflate over the past
> years, so it's not a really surprising result.
But I have also tried the 2.2.9 mod_deflate.c with 2.2.3, and it didn't
fix the issue. Therefore the fix would have to be in some other source
file.
Re: Segfault with fix for CVE-2009-1891
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Akins, Brian wrote:
> On 7/28/09 1:35 PM, "Stefan Fritsch" <sf...@sfritsch.de> wrote:
>
>> I have backported r791454 to 2.2.3 in Debian 4.0 and have received a
>> report [1] about segfaults with mod_deflate and mod_php (5.2.0).
>
> Isn't php only officially supported via fastcgi?
Last I read, the PHP license has no warranty.
One helpful detail, Stefan, would be if this is worker-specific or can
be reproduced with prefork. That helps narrow down the number of places
to consider your question.
As I recall, we have several filter bug fixes in deflate over the past
years, so it's not a really surprising result.
Re: Segfault with fix for CVE-2009-1891
Posted by "Akins, Brian" <Br...@turner.com>.
On 7/28/09 1:35 PM, "Stefan Fritsch" <sf...@sfritsch.de> wrote:
> I have backported r791454 to 2.2.3 in Debian 4.0 and have received a
> report [1] about segfaults with mod_deflate and mod_php (5.2.0).
Isn't php only officially supported via fastcgi?
--
Brian Akins