You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2011/08/31 18:27:20 UTC
DO NOT REPLY [Bug 51747] New: mod_fcgid loads entire request into
memory while passing to FCGI backend
https://issues.apache.org/bugzilla/show_bug.cgi?id=51747
Bug #: 51747
Summary: mod_fcgid loads entire request into memory while
passing to FCGI backend
Product: Apache httpd-2
Version: 2.2.19
Platform: PC
OS/Version: Linux
Status: NEW
Severity: normal
Priority: P2
Component: mod_fcgid
AssignedTo: bugs@httpd.apache.org
ReportedBy: dominic.benson@thirdlight.com
Classification: Unclassified
Created attachment 27448
--> https://issues.apache.org/bugzilla/attachment.cgi?id=27448
Patch to prevent memory explosion when fcgid hands off to FCGI backend
While FcgidMaxRequestInMem limits the amount of memory used by the server while
the client request is being sent, the process of reading the file (via
APR_BUCKET_NEXT) to write via proc_write_ipc causes the entire contents to be
in memory in the server process simultaneously.
The problem seems to be that, while data is passed to the FCGI process in small
chunks, the apr_bucket_brigade keeps all its buckets as the file is read, and
they morph into heap type.
Buckets should be removed from the brigade as the data is delivered, per the
guidelines for writing output filters.
Steps to reproduce:
httpd 2.2.19 on x86 with mod_fcgid 2.3.6
Create a file of 2.5GB+
Set FcgidMaxRequestLen to at least the size of this file
Send the file in a request that will be handled by fcgid
What happens:
A few seconds after the request body is transferred (depending on server
speed), the httpd child handling the request segfaults, as it exceeds the
process address limit
What should happen:
The httpd process should stay at modest memory usage throughout, and the
request should be processed by the FCGI backend
The attached patch prevents the end-of-request explosion. I think it probably
ought to have some more checks on the added bucket/brigade processing, though.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 51747] mod_fcgid loads entire request into memory while passing
to FCGI backend
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=51747
William A. Rowe Jr. <wr...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |nickdragomir@gmail.com
--- Comment #1 from William A. Rowe Jr. <wr...@apache.org> ---
*** Bug 52282 has been marked as a duplicate of this bug. ***
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 51747] mod_fcgid loads entire request into memory while passing
to FCGI backend
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=51747
Benjamin Coddington <bc...@uvm.edu> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |bcodding@uvm.edu
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 51747] mod_fcgid loads entire request into memory while passing
to FCGI backend
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=51747
Dominic Benson <do...@thirdlight.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #27448|0 |1
is obsolete| |
--- Comment #2 from Dominic Benson <do...@thirdlight.com> ---
Created attachment 30006
--> https://issues.apache.org/bugzilla/attachment.cgi?id=30006&action=edit
Updated patch against 2.3.7
The original patch doesn't apply correctly against 2.3.7, as the addition of:
if (APR_BUCKET_IS_METADATA(e)) {
continue;
}
causes an infinite loop (the APR_BUCKET_REMOVE hadn't happened at that point).
This patch corrects this; now the APR_BUCKET_REMOVE is the first action in the
loop, so it ought to be more robust against future changes.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
DO NOT REPLY [Bug 51747] mod_fcgid loads entire request into memory
while passing to FCGI backend
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=51747
William A. Rowe Jr. <wr...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords| |PatchAvailable
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 51747] mod_fcgid loads entire request into memory while passing
to FCGI backend
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=51747
Dominic Benson <do...@thirdlight.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #30006|0 |1
is obsolete| |
--- Comment #3 from Dominic Benson <do...@thirdlight.com> ---
Created attachment 30093
--> https://issues.apache.org/bugzilla/attachment.cgi?id=30093&action=edit
Revised patch fixing a bug introduced in the previous update for 2.3.7
The previous updated patch against 2.3.7 had a bug that it could run into a
blocking read that would never return, owing to the bucket not being in the
brigade at the point at which the data was read.
This issue seems to only affect SSL at certain file sizes, but those are likely
only loosely related as issues.
This updated version restores the order so that the read is prior to the
removal from the original brigade, while still avoiding the issue when applying
the original patch (written for 2.3.6) against 2.3.7 that metadata buckets
would cause an infinite loop.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 51747] mod_fcgid loads entire request into memory while passing
to FCGI backend
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=51747
--- Comment #4 from Dis <ap...@sigkill.net> ---
"Me too" and the Mar 21 patch fixed it. (For future searchers, I hit this on
Ubuntu through the owncloud user migration plugin - it can't upload the large
dataset successfully. Without this patch, it coredumps but with it applied it
worked.)
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org