You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2015/10/08 14:10:06 UTC
[Bug 58488] New: Stoped PHP-FPM pool in multi-pool system with
comunication over UDS exhausts connection and causes mod_proxy_fcgi crash
https://bz.apache.org/bugzilla/show_bug.cgi?id=58488
Bug ID: 58488
Summary: Stoped PHP-FPM pool in multi-pool system with
comunication over UDS exhausts connection and causes
mod_proxy_fcgi crash
Product: Apache httpd-2
Version: 2.4.9
Hardware: PC
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: mod_proxy_fcgi
Assignee: bugs@httpd.apache.org
Reporter: carlos.nieto@seap.minhap.es
In a multi-pool PHP-FPM when one of the PHP-FPM pools is stopped, but apache
continues processing requests towards the pool, the pools of others
applications begins to fails.
In all the application's pools communication is realized over UDS.
There are registered segmentation faults in Apache logs.
[Tue Oct 06 12:52:14.272956 2015] [core:notice] [pid 8563:tid 140096632240128]
AH00052: child pid 13967 exit signal Segmentation fault (11)
Also the /var/log/messages show segfaults.
Oct 6 12:21:30 phcaeproma01 kernel: httpd[13647]: segfault at 55 ip
00007f6ac91aaab2 sp 00007f6ac17fa860 error 6 in
mod_proxy_fcgi.so[7f6ac91a8000+4000]
This direction, corresponds to:
addr2line -e /opt/apache-2.4/modules/mod_proxy.so -fCi 0x6AB2
ap_proxy_connection_create
/home/software/php/httpd-2.4.9/modules/proxy/proxy_util.c:2790
Line 2790 in proxy_util.c correspons to:
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, APLOGNO(00962)
"%s: connection complete to %pI (%s)",
proxy_function, backend_addr, conn->hostname);
This line, make be sanitized in similar form to bug
https://bz.apache.org/bugzilla/show_bug.cgi?id=56858
But the source of error, i think, is the stablishment of the connection, when
connections are UDS, it seems to me, that mod_proxy_fcgi does not detect that
the PHP-FPM pool is closed and continues making connections instead of return a
503 or similar error condition. The UDS file is present, but nobody hear the
request, and the connections are continuosly creating until some condition
causes no more connections can be created, and the above ap_log_error receives
a null connection.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 58488] Stoped PHP-FPM pool in multi-pool system with
comunication over UDS exhausts connection and causes mod_proxy_fcgi crash
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58488
--- Comment #3 from Luca Toscano <to...@gmail.com> ---
Extra question: since 2.4.9 is a bit old, have you tried to reproduce with a
more recent version of httpd?
I am asking this because I noticed this entry in the changelog
(https://www.apache.org/dist/httpd/CHANGES_2.4) for 2.4.10:
*) mod_proxy_fcgi: Don't segfault when failing to connect to the backend.
(regression in 2.4.9 release) [Jeff Trawick]
https://svn.apache.org/r1592998
If you still have patience and time it would be really great if you could test
at least 2.4.10 and see if it solves the problem (or maybe only the patch
mentioned above).
Also 2.4.9 -> 2.4.12 brought a lot of mod_proxy_fcgi fixes. The more recent
version of httpd the better :)
Thanks!
Luca
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 58488] Stoped PHP-FPM pool in multi-pool system with
comunication over UDS exhausts connection and causes mod_proxy_fcgi crash
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58488
Luca Toscano <to...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |toscano.luca@gmail.com
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 58488] Stoped PHP-FPM pool in multi-pool system with
comunication over UDS exhausts connection and causes mod_proxy_fcgi crash
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58488
carlos.nieto@seap.minhap.es changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |carlos.nieto@seap.minhap.es
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 58488] Stoped PHP-FPM pool in multi-pool system with
comunication over UDS exhausts connection and causes mod_proxy_fcgi crash
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58488
--- Comment #1 from Luca Toscano <to...@gmail.com> ---
Hi Carlos,
really sorry for the delay. This bug sounds really bad, would you mind to add
some basic data about your httpd version and maybe some steps about how to
repro it?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 58488] Stoped PHP-FPM pool in multi-pool system with
comunication over UDS exhausts connection and causes mod_proxy_fcgi crash
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58488
--- Comment #4 from Luca Toscano <to...@gmail.com> ---
Hi Carlos,
any news?
Luca
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 58488] Stoped PHP-FPM pool in multi-pool system with
comunication over UDS exhausts connection and causes mod_proxy_fcgi crash
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58488
Szőgyényi Gábor <sz...@freemail.hu> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |szg0000@freemail.hu
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 58488] Stoped PHP-FPM pool in multi-pool system with
comunication over UDS exhausts connection and causes mod_proxy_fcgi crash
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58488
--- Comment #2 from carlos.nieto@seap.minhap.es ---
Of course Luca, httpd version was 2.4.9, you can repro the bug in a
configuration with at least two PHP-FPM pools backends and a apache frontend
communicating over UDS. Configuration is something like this:
<VirtualHost *:80>
ServerName poolA.test.com
DocumentRoot "/opt/poolA/htdocs/"
<Proxy "unix:/dev/shm/poolA.sock|fcgi://php-fpm-poolA.local">
ProxySet min=0
ProxySet acquire=20
ProxySet connectiontimeout=100ms
ProxySet retry=0
ProxySet timeout=300
Require all granted
</Proxy>
# Sintaxis 2.4.10
<FilesMatch \.php$>
SetHandler "proxy:fcgi://php-fpm-poolA.local"
Require all granted
</FilesMatch>
RedirectMatch ^/$ /index.php
</VirtualHost>
<VirtualHost *:80>
ServerName poolB.test.com
DocumentRoot "/opt/poolB/htdocs/"
<Proxy "unix:/dev/shm/poolB.sock|fcgi://php-fpm-poolB.local">
ProxySet min=0
ProxySet acquire=20
ProxySet connectiontimeout=100ms
ProxySet retry=0
ProxySet timeout=300
Require all granted
</Proxy>
# Sintaxis 2.4.10
<FilesMatch \.php$>
SetHandler "proxy:fcgi://php-fpm-poolB.local"
Require all granted
</FilesMatch>
RedirectMatch ^/$ /index.php
</VirtualHost>
So, you can shutdown one of the PHP-FPM pools, by example poolB, killing their
FPM processes, and inject requests to both pools, the active pool poolA
responds, and the no active pool, poolB, returns 503 errors.
But if you persist the requests to both pool, then sometime crash happens and
both pools fails.
I have observed this behavior in a production system with high load in a pool
with we have shutdowned for maintenance and then past a time, the other pools
have begun these failures, so, instead of a simple shutdown, we had to set up a
maintenance page.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org