You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Jakov Sosic <ja...@srce.hr> on 2011/08/01 17:21:52 UTC

[users@httpd] Program terminated with signal 7, Bus error.

Hi.

I run RHEL v5.6 and can see this kind of entries in my apache error_log:

[Sun Jul 31 23:27:44 2011] [notice] child pid 6974 exit signal Bus error
(7), possible coredump in /tmp/httpd-gdb-dump
[Sun Jul 31 23:27:44 2011] [notice] child pid 6976 exit signal Bus error
(7), possible coredump in /tmp/httpd-gdb-dump
[Sun Jul 31 23:27:44 2011] [notice] child pid 6979 exit signal Bus error
(7), possible coredump in /tmp/httpd-gdb-dump


I've turned on coredump as you can see, and installed lots of debuginfo
packages (although not all of them).


After the core is generated, this is what I get with gdb:

warning: no loadable sections found in added symbol-file system-supplied
DSO at 0x7ffff930f000
Core was generated by `/usr/sbin/httpd'.
Program terminated with signal 7, Bus error.
#0  0x00002b66739c139b in memcpy () from /lib64/libc.so.6
(gdb) bt
#0  0x00002b66739c139b in memcpy () from /lib64/libc.so.6
#1  0x00002b6677e46584 in php_ob_append (str=0x2b6685903000 <Address
0x2b6685903000 out of bounds>, str_length=2270) at
/usr/include/bits/string3.h:51
#2  php_b_body_write (str=0x2b6685903000 <Address 0x2b6685903000 out of
bounds>, str_length=2270) at /usr/src/debug/php-5.3.3/main/output.c:682
#3  0x00002b6677e4a7ba in _php_stream_passthru (stream=0x2b667ea474b8)
at /usr/src/debug/php-5.3.3/main/streams/streams.c:1220
#4  0x00002b6677de0839 in zif_readfile (ht=<value optimized out>,
return_value=0x2b667ea4a2c8, return_value_ptr=<value optimized out>,
    this_ptr=<value optimized out>, return_value_used=<value optimized
out>) at /usr/src/debug/php-5.3.3/ext/standard/file.c:1442
#5  0x00002b667d2458ad in phar_readfile (ht=1,
return_value=0x2b667ea4a2c8, return_value_ptr=0x0, this_ptr=0x0,
return_value_used=0)
    at /usr/src/debug/php-5.3.3/ext/phar/func_interceptors.c:320
#6  0x00002b6677ed27c9 in zend_do_fcall_common_helper_SPEC
(execute_data=0x2b667eedc050) at
/usr/src/debug/php-5.3.3/Zend/zend_vm_execute.h:316
#7  0x00002b6677ea888b in execute (op_array=0x2b66808c0f80) at
/usr/src/debug/php-5.3.3/Zend/zend_vm_execute.h:107
#8  0x00002b6677e84e35 in zend_execute_scripts (type=8, retval=0x0,
file_count=3) at /usr/src/debug/php-5.3.3/Zend/zend.c:1194
#9  0x00002b6677e35d28 in php_execute_script
(primary_file=0x7ffff9284b40) at /usr/src/debug/php-5.3.3/main/main.c:2260
#10 0x00002b6677f0d52d in php_handler (r=0x2b6686b30100) at
/usr/src/debug/php-5.3.3/sapi/apache2handler/sapi_apache2.c:669
#11 0x00002b6671a45a0a in ap_run_handler (r=0x2b6686b30100) at
/usr/src/debug/httpd-2.2.3/server/config.c:157
#12 0x00002b6671a48e98 in ap_invoke_handler (r=0x2b6686b30100) at
/usr/src/debug/httpd-2.2.3/server/config.c:375
#13 0x00002b6671a53938 in ap_process_request (r=0x2b6686b30100) at
/usr/src/debug/httpd-2.2.3/modules/http/http_request.c:258
#14 0x00002b6671a50b70 in ap_process_http_connection (c=0x2b6686b1dee0)
at /usr/src/debug/httpd-2.2.3/modules/http/http_core.c:184
#15 0x00002b6671a4cc92 in ap_run_process_connection (c=0x2b6686b1dee0)
at /usr/src/debug/httpd-2.2.3/server/connection.c:43
#16 0x00002b6671a577e9 in child_main (child_num_arg=<value optimized
out>) at /usr/src/debug/httpd-2.2.3/server/mpm/prefork/prefork.c:640
#17 0x00002b6671a57a7a in make_child (s=0x2b6671c99d70, slot=26) at
/usr/src/debug/httpd-2.2.3/server/mpm/prefork/prefork.c:736
#18 0x00002b6671a582dd in ap_mpm_run (_pconf=<value optimized out>,
plog=<value optimized out>, s=<value optimized out>)
    at /usr/src/debug/httpd-2.2.3/server/mpm/prefork/prefork.c:871
#19 0x00002b6671a32e48 in main (argc=1, argv=0x7ffff9285138) at
/usr/src/debug/httpd-2.2.3/server/main.c:717
(gdb) quit



Is there any way I can debug this any further? Crashes occur randomly,
and apache service is not affected, pages are being served regulary
before and after this kind of errors. Hardware is out of question
because I have this problem on more than one server...


Any idea what to do next?


-- 
Jakov Sosic
www.srce.hr

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] Program terminated with signal 7, Bus error.

Posted by Jeroen Geilman <je...@adaptr.nl>.
On 2011-08-01 20:11, Jakov Sosic wrote:
> On 08/01/2011 06:49 PM, Jeroen Geilman wrote:
>> On 2011-08-01 17:21, Jakov Sosic wrote:
>>> Hi.
>>>
>>> I run RHEL v5.6 and can see this kind of entries in my apache error_log:
>>>
>>> [Sun Jul 31 23:27:44 2011] [notice] child pid 6974 exit signal Bus error
>>> (7), possible coredump in /tmp/httpd-gdb-dump
>>> [Sun Jul 31 23:27:44 2011] [notice] child pid 6976 exit signal Bus error
>>> (7), possible coredump in /tmp/httpd-gdb-dump
>>> [Sun Jul 31 23:27:44 2011] [notice] child pid 6979 exit signal Bus error
>>> (7), possible coredump in /tmp/httpd-gdb-dump
>> Which apache or third-party modules are you running ?
>> What VERSION of apache httpd are you running ?
>> This is not optional information.
> I said RHEL v5.6 (x86_64 architecture), so versions are:

I don't assume package versions from the mere mention of an OS.

>
> RHEL packaged and patched:
> --------------------------
> Apache - httpd-2.2.3-45.el5_6.1
> PHP - php53-5.3.3-1.el5_6.1
>
> EPEL packages:
> ---------------------------------
> mod_extract_forwarded-2.0.2-2.el5
>
>
> 3rd party (compiled and packaged by me):
> ----------------------------------------
> php-eaccelerator - 0.9.6.1
> php-pecl-memcache - 2.2.6
> php-mcrypt - 5.3.3
> php-sqlite - 5.3.3
>
> mod_jk-ap20 - 1.2.30-1jpp
>
>
> I suspect it's php too. But how to locate which module is to blame?

The dump tells you it's happening in

	php_ob_append

(apart from strcpy in libc6, which will probably not be the reason, or your OS wouldn't run at all)

See what provides that function, and go down the list from the dump if 
it's not clear what goes wrong.


>
>>> Is there any way I can debug this any further? Crashes occur randomly,
>>> and apache service is not affected, pages are being served regulary
>>> before and after this kind of errors. Hardware is out of question
>>> because I have this problem on more than one server...
>> It's PHP.
>>
>> It's always PHP.
> It's possible. How can I debug it further? Maybe I should ask on some
> other list because I can feel the negativity from your comments...

What negativity ?
It is quite customary to provide - at the very least - the apache 
version and versions of any relevant modules, complete log entries with 
nothing elided, and steps to reproduce the problem if that is at all 
possible.
That you spout "RHEL 5.6" and leave us to deduce the rest can only lead 
to misunderstandings and, at worst, misdiagnoses.

Wouldn't you rather move in the right direction from the start ?

> I've shown with my first post that I'm not lazy and can do basic research.

Nobody is doubting that, or, in fact, anything whatsoever.
You need to provide sufficient information for the people on this list 
to have a chance at diagnosing what's wrong.

I don't see why you need to be so defensive about it.

> I'm sorry if you understood my question as an attack on apache project.

I did not - I understood it as a lack of required information.


-- 
J.


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] Program terminated with signal 7, Bus error.

Posted by Jakov Sosic <ja...@srce.hr>.
On 08/01/2011 06:49 PM, Jeroen Geilman wrote:
> On 2011-08-01 17:21, Jakov Sosic wrote:
>> Hi.
>>
>> I run RHEL v5.6 and can see this kind of entries in my apache error_log:
>>
>> [Sun Jul 31 23:27:44 2011] [notice] child pid 6974 exit signal Bus error
>> (7), possible coredump in /tmp/httpd-gdb-dump
>> [Sun Jul 31 23:27:44 2011] [notice] child pid 6976 exit signal Bus error
>> (7), possible coredump in /tmp/httpd-gdb-dump
>> [Sun Jul 31 23:27:44 2011] [notice] child pid 6979 exit signal Bus error
>> (7), possible coredump in /tmp/httpd-gdb-dump
> 
> Which apache or third-party modules are you running ?
> What VERSION of apache httpd are you running ?
> This is not optional information.

I said RHEL v5.6 (x86_64 architecture), so versions are:

RHEL packaged and patched:
--------------------------
Apache - httpd-2.2.3-45.el5_6.1
PHP - php53-5.3.3-1.el5_6.1

EPEL packages:
---------------------------------
mod_extract_forwarded-2.0.2-2.el5


3rd party (compiled and packaged by me):
----------------------------------------
php-eaccelerator - 0.9.6.1
php-pecl-memcache - 2.2.6
php-mcrypt - 5.3.3
php-sqlite - 5.3.3

mod_jk-ap20 - 1.2.30-1jpp


I suspect it's php too. But how to locate which module is to blame?


>> Is there any way I can debug this any further? Crashes occur randomly,
>> and apache service is not affected, pages are being served regulary
>> before and after this kind of errors. Hardware is out of question
>> because I have this problem on more than one server...
> 
> It's PHP.
> 
> It's always PHP.

It's possible. How can I debug it further? Maybe I should ask on some
other list because I can feel the negativity from your comments... I've
shown with my first post that I'm not lazy and can do basic research.
I'm sorry if you understood my question as an attack on apache project.


Thank you,




-- 
Jakov Sosic
www.srce.hr

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] Program terminated with signal 7, Bus error.

Posted by Jeroen Geilman <je...@adaptr.nl>.
On 2011-08-01 17:21, Jakov Sosic wrote:
> Hi.
>
> I run RHEL v5.6 and can see this kind of entries in my apache error_log:
>
> [Sun Jul 31 23:27:44 2011] [notice] child pid 6974 exit signal Bus error
> (7), possible coredump in /tmp/httpd-gdb-dump
> [Sun Jul 31 23:27:44 2011] [notice] child pid 6976 exit signal Bus error
> (7), possible coredump in /tmp/httpd-gdb-dump
> [Sun Jul 31 23:27:44 2011] [notice] child pid 6979 exit signal Bus error
> (7), possible coredump in /tmp/httpd-gdb-dump

Which apache or third-party modules are you running ?
What VERSION of apache httpd are you running ?
This is not optional information.

>
> I've turned on coredump as you can see, and installed lots of debuginfo
> packages (although not all of them).

Most conspicuously, not the apache one.

That will make it difficult to follow where it fails but luckily, it's 
not apache that fails here.

>
>
> After the core is generated, this is what I get with gdb:
>
> warning: no loadable sections found in added symbol-file system-supplied
> DSO at 0x7ffff930f000
> Core was generated by `/usr/sbin/httpd'.
> Program terminated with signal 7, Bus error.
> #0  0x00002b66739c139b in memcpy () from /lib64/libc.so.6
> (gdb) bt
> #0  0x00002b66739c139b in memcpy () from /lib64/libc.so.6
> #1  0x00002b6677e46584 in php_ob_append (str=0x2b6685903000<Address
> 0x2b6685903000 out of bounds>, str_length=2270) at
> /usr/include/bits/string3.h:51
> #2  php_b_body_write (str=0x2b6685903000<Address 0x2b6685903000 out of
> bounds>, str_length=2270) at /usr/src/debug/php-5.3.3/main/output.c:682
> #3  0x00002b6677e4a7ba in _php_stream_passthru (stream=0x2b667ea474b8)
> at /usr/src/debug/php-5.3.3/main/streams/streams.c:1220
> #4  0x00002b6677de0839 in zif_readfile (ht=<value optimized out>,
> return_value=0x2b667ea4a2c8, return_value_ptr=<value optimized out>,
>      this_ptr=<value optimized out>, return_value_used=<value optimized
> out>) at /usr/src/debug/php-5.3.3/ext/standard/file.c:1442
> #5  0x00002b667d2458ad in phar_readfile (ht=1,
> return_value=0x2b667ea4a2c8, return_value_ptr=0x0, this_ptr=0x0,
> return_value_used=0)
>      at /usr/src/debug/php-5.3.3/ext/phar/func_interceptors.c:320
> #6  0x00002b6677ed27c9 in zend_do_fcall_common_helper_SPEC
> (execute_data=0x2b667eedc050) at
> /usr/src/debug/php-5.3.3/Zend/zend_vm_execute.h:316
> #7  0x00002b6677ea888b in execute (op_array=0x2b66808c0f80) at
> /usr/src/debug/php-5.3.3/Zend/zend_vm_execute.h:107
> #8  0x00002b6677e84e35 in zend_execute_scripts (type=8, retval=0x0,
> file_count=3) at /usr/src/debug/php-5.3.3/Zend/zend.c:1194
> #9  0x00002b6677e35d28 in php_execute_script
> (primary_file=0x7ffff9284b40) at /usr/src/debug/php-5.3.3/main/main.c:2260
> #10 0x00002b6677f0d52d in php_handler (r=0x2b6686b30100) at
> /usr/src/debug/php-5.3.3/sapi/apache2handler/sapi_apache2.c:669
> #11 0x00002b6671a45a0a in ap_run_handler (r=0x2b6686b30100) at
> /usr/src/debug/httpd-2.2.3/server/config.c:157
> #12 0x00002b6671a48e98 in ap_invoke_handler (r=0x2b6686b30100) at
> /usr/src/debug/httpd-2.2.3/server/config.c:375
> #13 0x00002b6671a53938 in ap_process_request (r=0x2b6686b30100) at
> /usr/src/debug/httpd-2.2.3/modules/http/http_request.c:258
> #14 0x00002b6671a50b70 in ap_process_http_connection (c=0x2b6686b1dee0)
> at /usr/src/debug/httpd-2.2.3/modules/http/http_core.c:184
> #15 0x00002b6671a4cc92 in ap_run_process_connection (c=0x2b6686b1dee0)
> at /usr/src/debug/httpd-2.2.3/server/connection.c:43
> #16 0x00002b6671a577e9 in child_main (child_num_arg=<value optimized
> out>) at /usr/src/debug/httpd-2.2.3/server/mpm/prefork/prefork.c:640
> #17 0x00002b6671a57a7a in make_child (s=0x2b6671c99d70, slot=26) at
> /usr/src/debug/httpd-2.2.3/server/mpm/prefork/prefork.c:736
> #18 0x00002b6671a582dd in ap_mpm_run (_pconf=<value optimized out>,
> plog=<value optimized out>, s=<value optimized out>)
>      at /usr/src/debug/httpd-2.2.3/server/mpm/prefork/prefork.c:871
> #19 0x00002b6671a32e48 in main (argc=1, argv=0x7ffff9285138) at
> /usr/src/debug/httpd-2.2.3/server/main.c:717
> (gdb) quit

That is very old - 16 releases behind the current stable version.

> Is there any way I can debug this any further? Crashes occur randomly,
> and apache service is not affected, pages are being served regulary
> before and after this kind of errors. Hardware is out of question
> because I have this problem on more than one server...

It's PHP.

It's always PHP.

-- 
J.


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org