You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2017/06/29 11:03:35 UTC

[Bug 61237] New: apache2 2.4.26 segfaults when http2 module is enabled

https://bz.apache.org/bugzilla/show_bug.cgi?id=61237

            Bug ID: 61237
           Summary: apache2 2.4.26 segfaults when http2 module is enabled
           Product: Apache httpd-2
           Version: 2.4.26
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: regression
          Priority: P2
         Component: mod_http2
          Assignee: bugs@httpd.apache.org
          Reporter: ondrej@sury.org
  Target Milestone: ---

Hi,

several users have reported that apache2 segfaults usually in PHP when http2
module is enabled in apache2 2.4.26.

The original bugreports are here, I am copying just some traces:

~~~~
Jun 28 16:30:14 mail kernel: [707770.889003] apache2[102233]: segfault at
7f89e25c6010 ip 00007f89e1e8cde0 sp 00007fff56430e00 error 4 in
libapr-1.so.0.5.2 (deleted)[7f89e1e72000+31000]
Jun 28 16:30:44 mail kernel: [707800.310376] apache2[106852]: segfault at
600000001 ip 00007fd3da2df4e2 sp 00007fd3c1d326e8 error 4 in
libphp7.0.so[7fd3da0b1000+392000]
Jun 28 16:30:55 mail kernel: [707812.141281] apache2[107200]: segfault at
600000001 ip 00007fd3da2e0b04 sp 00007fd3c25359b0 error 4 in
libphp7.0.so[7fd3da0b1000+392000]
Jun 28 16:32:06 mail kernel: [707882.272427] apache2[107928]: segfault at
7f6c00000048 ip 00007f6cc1ba3042 sp 00007f6ca9d5a9b0 error 4 in
libphp7.0.so[7f6cc195d000+392000]
Jun 28 16:32:43 mail kernel: [707919.758973] apache2[108367]: segfault at
55c900000036 ip 00007f6cc1c0057a sp 00007f6caa55b9e0 error 4 in
libphp7.0.so[7f6cc195d000+392000]
Jun 28 16:33:32 mail kernel: [707968.278623] apache2[108605]: segfault at
7f6c0000001a ip 00007f6cc1c00548 sp 00007f6ca95576f0 error 4 in
libphp7.0.so[7f6cc195d000+392000]
Jun 28 16:33:37 mail kernel: [707973.490387] apache2[108814]: segfault at
7f6e474ea5a4 ip 00007f6cc1bbdb5f sp 00007f6ca9d5a8d0 error 4 in
libphp7.0.so[7f6cc195d000+392000]
Jun 28 16:33:47 mail kernel: [707984.145976] apache2[107830]: segfault at
3330322e30 ip 00007f6cc1b8cb04 sp 00007f6caad5c710 error 4 in
libphp7.0.so[7f6cc195d000+392000]
Jun 28 16:36:04 mail kernel: [708120.747497] apache2[109092]: segfault at
600000001 ip 00007f6cc1b8cb04 sp 00007f6ca9557570 error 4 in
libphp7.0.so[7f6cc195d000+392000]
Jun 28 16:36:14 mail kernel: [708130.828956] apache2[109624]: segfault at 0 ip
(null) sp 00007f6ca95576e8 error 14 in apache2[55c92a09e000+9e000]
Jun 28 16:37:15 mail kernel: [708191.943459] apache2[109674]: segfault at 1406
ip 00007f6cc1ba2ea1 sp 00007f6ca9557080 error 4 in
libphp7.0.so[7f6cc195d000+392000]
Jun 28 16:38:18 mail kernel: [708254.981575] apache2[110142]: segfault at
600000001 ip 00007f6cc1b8cb04 sp 00007f6ca9559860 error 4 in
libphp7.0.so[7f6cc195d000+392000]
Jun 28 16:38:21 mail kernel: [708257.266126] apache2[110454]: segfault at
60600000000 ip 00007f6cc1b8cb04 sp 00007f6ca9557290 error 4 in
libphp7.0.so[7f6cc195d000+392000]
Jun 28 16:40:55 mail kernel: [708411.291971] apache2[112048]: segfault at
7fe23b4ea5ac ip 00007fe0b5b55bfa sp 00007fe09dd5a8d0 error 4 in
libphp7.0.so[7fe0b58f5000+392000]
Jun 28 16:41:45 mail kernel: [708461.406343] apache2[111959]: segfault at 0 ip
00007fe0b5b3817b sp 00007fe09e55af58 error 4 in
libphp7.0.so[7fe0b58f5000+392000]
Jun 28 16:44:28 mail kernel: [708624.787914] apache2[114082]: segfault at
600000001 ip 00007fe0b5b24b04 sp 00007fe09dd5a860 error 4 in
libphp7.0.so[7fe0b58f5000+392000]
Jun 28 16:44:54 mail kernel: [708650.426822] apache2[114418]: segfault at
7fe23b4ea5a4 ip 00007fe0b5b55bfa sp 00007fe09e55b8d0 error 4 in
libphp7.0.so[7fe0b58f5000+392000]
Jun 28 16:46:07 mail kernel: [708723.584629] apache2[114737]: segfault at
600000001 ip 00007fe0b5b24b04 sp 00007fe09d557570 error 4 in
libphp7.0.so[7fe0b58f5000+392000]
Jun 28 16:47:09 mail kernel: [708785.630073] apache2[115322]: segfault at c ip
00007fe0b5b5cd67 sp 00007fe09d557110 error 4 in
libphp7.0.so[7fe0b58f5000+392000]
Jun 28 16:51:56 mail kernel: [709073.089354] apache2[117357]: segfault at
7fe000000000 ip 00007fe0b5b24b04 sp 00007fe09d5597d0 error 4 in
libphp7.0.so[7fe0b58f5000+392000]
~~~~

~~~~
[Thu Jun 29 06:55:23.235009 2017] [core:notice] [pid 6435] AH00051: child pid
16045 exit signal Segmentation fault (11), possible coredump in /etc/apache2
*** Error in `/usr/sbin/apache2': double free or corruption (fasttop):
0x00007faca40608d0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7908b)[0x7facd0d7508b]
/lib/x86_64-linux-gnu/libc.so.6(+0x82c3a)[0x7facd0d7ec3a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7facd0d82d2c]
/usr/lib/php/20160303/xdebug.so(xdebug_llist_remove+0x4f)[0x7facc75766af]
/usr/lib/php/20160303/xdebug.so(xdebug_execute_ex+0x474)[0x7facc7566d84]
/usr/lib/apache2/modules/libphp7.1.so(zend_execute+0x188)[0x7faccd1eb458]
/usr/lib/apache2/modules/libphp7.1.so(zend_execute_scripts+0xd3)[0x7faccd1464a3]
/usr/lib/apache2/modules/libphp7.1.so(+0x30805d)[0x7faccd1ed05d]
/usr/sbin/apache2(ap_run_handler+0x40)[0x55e894d0d7c0]
/usr/sbin/apache2(ap_invoke_handler+0xc6)[0x55e894d0dd56]
/usr/sbin/apache2(ap_process_async_request+0x363)[0x55e894d25a33]
/usr/sbin/apache2(ap_process_request+0x10)[0x55e894d25b40]
/usr/lib/apache2/modules/mod_http2.so(+0x26d09)[0x7facce218d09]
/usr/sbin/apache2(ap_run_process_connection+0x40)[0x55e894d17630]
/usr/lib/apache2/modules/mod_http2.so(+0x281d7)[0x7facce21a1d7]
/usr/lib/apache2/modules/mod_http2.so(+0x2b6a3)[0x7facce21d6a3]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76da)[0x7facd10ca6da]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x5f)[0x7facd0e04d7f]
~~~~

~~~~
[Thu Jun 29 09:53:11.623671 2017] [core:notice] [pid 25242] AH00051: child pid
27364 exit signal Segmentation fault (11), possible coredump in
/data01/apacheSegFault
[Thu Jun 29 09:53:14.973912 2017] [core:warn] [pid 25242] AH00045: child
process 27665 still did not exit, sending a SIGTERM
[Thu Jun 29 09:53:16.975994 2017] [core:warn] [pid 25242] AH00045: child
process 27665 still did not exit, sending a SIGTERM
[Thu Jun 29 09:53:18.978067 2017] [core:warn] [pid 25242] AH00045: child
process 27665 still did not exit, sending a SIGTERM
[Thu Jun 29 09:53:20.980150 2017] [core:error] [pid 25242] AH00046: child
process 27665 still did not exit, sending a SIGKILL
[Thu Jun 29 09:53:21.981314 2017] [mpm_prefork:notice] [pid 25242] AH00169:
caught SIGTERM, shutting down
[Thu Jun 29 09:54:13.146363 2017] [wsgi:warn] [pid 28557] mod_wsgi: Compiled
for Python/2.7.11+.
[Thu Jun 29 09:54:13.146393 2017] [wsgi:warn] [pid 28557] mod_wsgi: Runtime
using Python/2.7.12.
[Thu Jun 29 09:54:13.149165 2017] [mpm_prefork:notice] [pid 28557] AH00163:
Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2k mod_wsgi/4.5.2 Python/2.7.12
mod_perl/2.0.9 Perl/v5.22.1 configured -- resuming normal operations
~~~~
+
~~~~
[Thu Jun 29 09:47:20.978227 2017] [http2:warn] [pid 22221] [client
***.***.***.***:65284] AH10020: h2_session(29,IDLE,0): session cleanup
triggered by p
ool cleanup. this should have happened earlier already.
[Thu Jun 29 09:47:20.978244 2017] [http2:warn] [pid 22221] [client
***.***.***.***:65284] AH03199: h2_session(29,IDLE,0): connection disappeared
without
proper goodbye, clients will be confused, should not happen

[Thu Jun 29 09:50:41.725088 2017] [php7:error] [pid 24245] [client
***.***.***.***:55746] PHP Fatal error: Invalid opcode 137/16/8. in
/data01/var/www/prod/vendor/zendframework/zendframework1/library/Zend/Config.php
on line 112, referer: https://lernziele.charite.de/llpMobil/
[Thu Jun 29 09:51:20.074313 2017] [http2:warn] [pid 23933] [client
***.***.***.***:55748] AH10020: h2_session(31,IDLE,0): session cleanup
triggered by pool cleanup. this should have happened earlier already.
~~~~

And here is screenshot from gdb:
https://user-images.githubusercontent.com/195299/27682022-5868e1a6-5cc1-11e7-9c33-cfef81e23ad5.png


It looks like http2 module corrupts memory in other modules (or something like
that).

Ondrej

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 61237] apache2 2.4.26 segfaults when http2 module is enabled

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=61237

--- Comment #3 from mark burdett <mf...@gmail.com> ---
.. and the solution for this was to migrate mod_php to php_fpm and prefork mpm
to event mpm.

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 61237] apache2 2.4.26 segfaults when http2 module is enabled

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=61237

joel@joelj.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |joel@joelj.org

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 61237] apache2 2.4.26 segfaults when http2 module is enabled

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=61237

--- Comment #2 from mark burdett <mf...@gmail.com> ---
Also, looking at the error log, the PHP errors indicate vital configuration
required to make the app work were lost, so very unpredictable things could
happen...

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 61237] apache2 2.4.26 segfaults when http2 module is enabled

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=61237

mark burdett <mf...@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mfburdett@gmail.com

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 61237] apache2 2.4.26 segfaults when http2 module is enabled

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=61237

--- Comment #1 from mark burdett <mf...@gmail.com> ---
My symptoms were segfaults, run away load, and one URL actually spat out the
raw PHP source code - instead of a webpage - before I shut the server down.

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 61237] apache2 2.4.26 segfaults when http2 module is enabled

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=61237

Stefan Eissing <st...@eissing.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #4 from Stefan Eissing <st...@eissing.org> ---
As a workaround, Apache 2.4.26 users can use the configuration directive

  H2MaxWorkers 1

to go back to the single-threaded execution model of previous httpd version.

In 2.4.27, release imminent, HTTP/2 will no longer be supported together with
mpm_prefork. 

This is done in a way that is least likely interrupting existing installations.
A warning will be logged and the HTTP/2 protocol will not be negotiated for any
connection. Otherwise the server will work as before.

As to deep technical discussion of why the httpd team chose to disable this
combination, please refer to the dev mailing list archive. Thanks.

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org