You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2014/11/26 15:46:53 UTC
[Bug 57268] New: apache process crashes when downloading large file
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
Bug ID: 57268
Summary: apache process crashes when downloading large file
Product: Apache httpd-2
Version: 2.4.10
Hardware: PC
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: mpm_event
Assignee: bugs@httpd.apache.org
Reporter: jaroslav@thinline.cz
When using event mpm, Apache processes randomly crash when serving request to
download large file. It seems the crash occurs when file download takes long
time (downloading 256MB file on 10Mbit line triggers the crash, but downloading
the same file on gbit LAN works fine.) It also seems the crash only occurs when
the server is busy serving other requests.
This is how I am able to reproduce the issue on my test server:
- create small file in document root (index.html with content "index") and
large file large.bin (dd if=/dev/zero of=large.bin bs=$((1024**2)) count=256)
- run ab -q -n 2000000 -c 20 'http://[a:b:c:d::1]/index.html' on the server
- run wget 'http://[a:b:c:d::1]/large.bin' on my workstation (~5 minutes long
download)
The server transmits some data and then wget reports "Connection closed at byte
xxx. Retrying." The amount of transmitted data changes every try, sometimes the
crash occurs after few seconds, sometimes it takes few minutes, in few cases
the file downloaded without causing Apache to crash.
When the server was idle - only wget running, no ab - server didn't crash in
any of my tries.
When the crash occurs, Apache logs this into error log:
[Wed Nov 26 14:38:14.982708 2014] [core:notice] [pid 12215:tid 140446535845760]
AH00052: child pid 16079 exit signal Segmentation fault (11).
In some cases kernel logs message like this one into dmesg (message from
different server, therefore different pid):
[1943221.021819] apache2[31073]: segfault at 7f3c380c52d8 ip 00007f3c59d66ab2
sp 00007f3c3afece30 error 4 in mod_mpm_event.so[7f3c59d60000+e000]
We are using Apache downloaded from Debian Jessie repository (apache2-bin),
together with libapache2-mod-fcgid, php5-cgi, suexec (apache2-suexec-custom)
and libapache2-mod-geoip. (PHP is not involved in processing any of the files
mentioned above.) List of active modules follows (from server-info): core.c,
event.c, http_core.c, mod_access_compat.c, mod_alias.c, mod_auth_basic.c,
mod_authn_core.c, mod_authn_file.c, mod_authz_core.c, mod_authz_host.c,
mod_authz_user.c, mod_autoindex.c, mod_deflate.c, mod_dir.c, mod_env.c,
mod_fcgid.c, mod_filter.c, mod_geoip.c, mod_info.c, mod_log_config.c,
mod_logio.c, mod_mime.c, mod_negotiation.c, mod_setenvif.c, mod_so.c,
mod_status.c, mod_unique_id.c, mod_unixd.c, mod_version.c, mod_watchdog.c
If I recall correctly, testing server is running in default configuration from
Debian package - let me know if you need some configuration details.
At the moment it seems switching to mpm_worker works around the problem.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
Yann Ylavic <yl...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #32246|r1638879 + r1639960 |r1638879 + r1640031
description| |
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
--- Comment #10 from jaroslav@thinline.cz ---
Thank you for the help.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
Eric Covener <co...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #5 from Eric Covener <co...@gmail.com> ---
(In reply to jaroslav from comment #4)
> Hello, thank you for the reply.
>
> Apache 2.4.10 in Debian is already shipped with patches r1638879 + r1640031
> applied, so these don't fix this crash.
>
> I applied the other one - r1639614 - on Debian source tree and so far it
> seems the issue is fixed with it. I am no longer able to reproduce the
> crash, even after multiple tries.
>
> If you still need the backtrade or any other information, let me know...
Sorry to take you up on it, but it would be nice to see a backtrace w/o
r1639614 just to confirm. It is a bit odd that it helps for a crash
mid-response.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
--- Comment #1 from Yann Ylavic <yl...@gmail.com> ---
There are 2 pending patches about mpm_event possible crashes in trunk (follow
up attachements).
Can you apply those in your environment, and preferably test them separatly so
that we can determine which one (if not both) should be backported?
Otherwise, can you provide a backtrace of the crash using gdb and a core file
(http://httpd.apache.org/dev/debugging.html may help).
You will probably need to add "CoreDumpDirectory /tmp" in httpd.conf, and start
httpd with unlimited core files (ulimit -c unlimited).
Once you have got a core(.pid) file in /tmp :
$ gdb /path/to/httpd -c /tmp/core.xxxx
% gdb) thread apply all bt
Then you could provide the output here.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
--- Comment #6 from jaroslav@thinline.cz ---
Created attachment 32247
--> https://issues.apache.org/bugzilla/attachment.cgi?id=32247&action=edit
backtrace
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
--- Comment #8 from Eric Covener <co...@gmail.com> ---
(In reply to jaroslav from comment #7)
> No problem, added it into attachements. The testing server has apache2-dbg
> package (debugging symbols) installed, hope it helps.
thanks, does actually seem consistent at least on the surface.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
Yann Ylavic <yl...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |ylavic.dev@gmail.com
--- Comment #2 from Yann Ylavic <yl...@gmail.com> ---
Created attachment 32245
--> https://issues.apache.org/bugzilla/attachment.cgi?id=32245&action=edit
r1639614
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
Yann Ylavic <yl...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |FIXED
--- Comment #9 from Yann Ylavic <yl...@gmail.com> ---
Both r1639614 and r1638879+r1640031 backported to upcoming 2.4.11.
Thanks Jaroslav for reporting and testing.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
jaroslav@thinline.cz changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #7 from jaroslav@thinline.cz ---
No problem, added it into attachements. The testing server has apache2-dbg
package (debugging symbols) installed, hope it helps.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
jaroslav@thinline.cz changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #4 from jaroslav@thinline.cz ---
Hello, thank you for the reply.
Apache 2.4.10 in Debian is already shipped with patches r1638879 + r1640031
applied, so these don't fix this crash.
I applied the other one - r1639614 - on Debian source tree and so far it seems
the issue is fixed with it. I am no longer able to reproduce the crash, even
after multiple tries.
If you still need the backtrade or any other information, let me know...
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
Yann Ylavic <yl...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 57268] apache process crashes when downloading large file
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=57268
--- Comment #3 from Yann Ylavic <yl...@gmail.com> ---
Created attachment 32246
--> https://issues.apache.org/bugzilla/attachment.cgi?id=32246&action=edit
r1638879 + r1639960
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org