[users@httpd] Apache dies with segfault

[Lischuk Mikhail <r0...@itx.com.ua>]

Hi everyone! 

My problem started for no reason. Just at some moment httpd started dying
with segfault. 

First I upgraded to the 2.2.21 version (I was using 2.2.20) - it didn't
help. 

Second, I noticed that all my Perl sites are alive on that same server, so
I've downloaded and recompiled PHP. It didnt help. Neither helped
commenting PHP out of httpd.conf 

Third, I enabled core file generation and applied gdb: 
Core was generated by `/usr/local/apache2/bin/httpd -k start'.
Program terminated with signal 11, Segmentation fault.
#0 0xb7b08c11 in strstr () from /lib/libc.so.6
(gdb) where
#0 0xb7b08c11 in strstr () from /lib/libc.so.6
#1 0xb7a5a25d in sch_out_filter () from
/usr/local/apache2/modules/mod_log.so
#2 0x08084068 in ap_remove_output_filter ()
#3 0x0807248e in default_handler ()
#4 0x08078926 in ap_run_child_init ()
#5 0x08079068 in ap_invoke_handler ()
#6 0x08095bd7 in ap_die ()
#7 0x08092a21 in ap_process_http_async_connection ()
#8 0x0808027f in
ap_run_create_connection ()
#9 0x080806c0 in ap_lingering_close ()
#10 0x080add18 in make_child ()
#11 0x080aded5 in perform_idle_server_maintenance ()
#12 0x080ae0f1 in ap_mpm_run ()
#13 0x080ae55c in ap_mpm_run ()
#14 0x08062ea9 in main ()

The problem is, I have analysed about 10 different types of core file. One
kind of them had only 1 string in stack, about "memory cannot be accessed".
Some of them were mentioning different functions, meaningless to me as to
non-developer. But the one I've given above is what httpd produces in 100%
of cases for the las 6 hours.
I became suspicious of that mod_log.so, so I just commented it out in
httpd.conf - an voila! Everything's working now, and besides all logs are
alive despite absent mod_log
Then I remembered that when I was updating from 2.2.10 to 2.2.20 `apachectl
-t` told me about 10+ modules that were not DSO any more, so I had to
comment them out.
My guess is that for some reason my installation became unaware of module
conflict and I
came to situation when I had both embedded and shared
mod_log at the same time.
But, of course, I may be wrong.
Anyways, my problem is gone, so this is more of FYI if someone finds this
useful. Also I attach httpd-generated stack trace from error_log, when it
dies with signal 6

P.S. still wondering why all the Perl sites were OK while all PHP sites
were dead. Long live mod_perl for its memory management?

BR

Re: [users@httpd] Apache dies with segfault

[Lischuk Mikhail <r0...@itx.com.ua>]

On Mon, 19 Sep 2011 16:54:49 +0100, Nick Kew <ni...@webthing.com> wrote:
> On Mon, 19 Sep 2011 18:08:45 +0300
> Lischuk Mikhail <r0...@itx.com.ua> wrote:
> 
> 
>> #1 0xb7a5a25d in sch_out_filter () from
>> /usr/local/apache2/modules/mod_log.so
> 
> An innocent name, but there's no mod_log.so in a
> regular apache install (mod_log_config is the
> 'standard' logging module).
> 
> Find out where that comes from.  It looks like the culprit.

Oh! I've checked it, and it looks like it has appeared right when I started
having problems with Apache.

12K 2011-09-18 10:47 mod_log.so

Looks like you are right, and I somehow got pwned.
Thank you, Nick! I will look into that issue.

-- 
Best regards,
Lischuk Mikhail

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: [users@httpd] Apache dies with segfault

[Nick Kew <ni...@webthing.com>]

On Mon, 19 Sep 2011 18:08:45 +0300
Lischuk Mikhail <r0...@itx.com.ua> wrote:


> #1 0xb7a5a25d in sch_out_filter () from
> /usr/local/apache2/modules/mod_log.so

An innocent name, but there's no mod_log.so in a
regular apache install (mod_log_config is the
'standard' logging module).

Find out where that comes from.  It looks like the culprit.

-- 
Nick Kew

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org