You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by "Mark P. Fister" <mf...@ebay.com> on 2002/02/06 08:15:13 UTC
[Q] SIGSEGV After fork()
Dear mod_perl experts:
Collectively, we've been at this for more than two weeks and have searched
various mod_perl archives, all to no avail.
Symptom:
=======
SIGSEGV after fork(). Very reproducible. Memory corruption gets moved
around if the codebase changes.
Tried:
=====
* disable rule: EXPAT
* gdb
* strace -f
* MOD_PERL_TRACE=all
***NOTE***
Read no further unless you're interested in helping, since the rest is
debug/more info.
Thanks! :)
Software:
========
Linux:
-----
2.4.13-ac5-20011101-1 #1 SMP Thu Nov 1 16:07:19 CST 2001 i686 unknown
Apache:
------
Server version: Apache/1.3.23 (Unix)
Server built: Feb 5 2002 22:16:11
Server's Module Magic Number: 19990320:11
Server compiled with....
-D HAVE_MMAP
-D HAVE_SHMGET
-D USE_SHMGET_SCOREBOARD
-D USE_MMAP_FILES
-D HAVE_FCNTL_SERIALIZED_ACCEPT
-D HAVE_SYSVSEM_SERIALIZED_ACCEPT
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D HTTPD_ROOT="/home/mfister/vendor/apache-1.3.23"
-D SUEXEC_BIN="/home/mfister/vendor/apache-1.3.23/sbin/suexec"
-D DEFAULT_PIDLOG="www/logs/httpd.pid"
-D DEFAULT_SCOREBOARD="www/logs/httpd.scoreboard"
-D DEFAULT_LOCKFILE="www/logs/httpd.lock"
-D DEFAULT_XFERLOG="www/logs/access_log"
-D DEFAULT_ERRORLOG="www/logs/error_log"
-D TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
-D ACCESS_CONFIG_FILE="conf/access.conf"
-D RESOURCE_CONFIG_FILE="conf/srm.conf"
Perl:
----
Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration:
Platform:
osname=linux, osvers=2.4.13-ac5-20011101-1, archname=i686-linux
uname='linux <server_name_deleted> 2.4.13-ac5-20011101-1 #1 smp thu nov 1 16:07:19 cst 2001 i686 unknown '
config_args='-des -Doptimize=-g -Dusedevel -Uinstallusrbinperl -Ubincompat5005 -Uusemymalloc -Dcc=gcc -pipe -g -I/usr/include/db3 -Dcccdlflags=-fPIC -Dinstallprefix=/home/mfister/vendor/perl-5.6.1 -Dprefix=/home/mfister/vendor/perl-5.6.1 -Dcf_email=mfister@ebay.com -Dinstallman1dir=/home/mfister/vendor/perl-5.6.1/man/man1 -Dinstallman3dir=/home/mfister/vendor/perl-5.6.1/man/man3 -Dman1dir=/home/mfister/vendor/perl-5.6.1/man/man1 -Dman3dir=/home/mfister/vendor/perl-5.6.1/man/man3 -Dd_dosuid=n -Dd_semctl_semun -Di_db -Di_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Uuselargefiles'
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
useperlio=undef d_sfio=undef uselargefiles=undef usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
Compiler:
cc='gcc -pipe -g -I/usr/include/db3', ccflags ='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include',
optimize='-g',
cppflags='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include'
ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.1 2.96-85)', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=4
alignbytes=4, usemymalloc=n, prototype=define
Linker and Libraries:
ld='gcc -pipe -g -I/usr/include/db3', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -ldl -lm -lc -lcrypt -lutil
perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
libc=/lib/libc-2.2.4.so, so=so, useshrplib=false, libperl=libperl.a
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: DEBUGGING
Built under linux
Compiled at Feb 4 2002 13:40:29
@INC:
/home/mfister/vendor/perl-5.6.1/lib/5.6.1/i686-linux
/home/mfister/vendor/perl-5.6.1/lib/5.6.1
/home/mfister/vendor/perl-5.6.1/lib/site_perl/5.6.1/i686-linux
/home/mfister/vendor/perl-5.6.1/lib/site_perl/5.6.1
/home/mfister/vendor/perl-5.6.1/lib/site_perl
mod_perl:
--------
version: 1.26
built with: perl Makefile.PL USE_APXS=1 \
WITH_APXS=/home/mfister/vendor/apache-1.3.23/sbin/apxs \
EVERYTHING=1 CCFLAGS='-pipe -fPIC'
Modules:
-------
*** fairly long list *** Will send if necessary ***
Debug Info:
==========
Stack Trace:
===========
#0 __pthread_mutex_lock (mutex=0x8bf04999) at mutex.c:99
#1 0x401b9cc8 in __libc_free (mem=0x4046cc18) at malloc.c:3152
#2 0x403ce028 in Perl_safesysfree (where=0x4046cc18) at util.c:158
#3 0x403f20d8 in Perl_sv_clear (sv=0x8198f60) at sv.c:3827
#4 0x403f2473 in Perl_sv_free (sv=0x8198f60) at sv.c:3950
#5 0x403f80e1 in do_clean_all (sv=0x8198f60) at sv.c:8411
#6 0x403e9c5e in S_visit (f=0x403f8094 <do_clean_all>) at sv.c:162
#7 0x403e9ce2 in Perl_sv_clean_all () at sv.c:193
#8 0x4038594a in perl_destruct (my_perl=0x809a9a8) at perl.c:665
#9 0x4035629c in perl_shutdown (s=0x0, p=0x0) at mod_perl.c:294
#10 0x40356be6 in mp_dso_unload (data=0x808e714) at mod_perl.c:489
#11 0x08050f34 in run_cleanups (c=0x809c8ac) at alloc.c:1713
#12 0x0804f5fa in ap_clear_pool (a=0x808e714) at alloc.c:538
#13 0x08062128 in standalone_main (argc=7, argv=0xbffff294) at http_main.c:5014
#14 0x08062cb2 in main (argc=7, argv=0xbffff294) at http_main.c:5401
#15 0x40155627 in __libc_start_main (main=0x80627d4 <main>, argc=7,
ubp_av=0xbffff294, init=0x804e3e4 <_init>, fini=0x807aa40 <_fini>,
rtld_fini=0x4000dcd4 <_dl_fini>, stack_end=0xbffff28c)
at ../sysdeps/generic/libc-start.c:129
***NOTE*** the following gdb session was gleaned from sv.c and refers to
the freed memory location (0x4046cc18) above:
(gdb) p *((XPV*)(sv)->sv_any)
$13 = {xpv_pv = 0x4046cc18 "", xpv_cur = 135562488, xpv_len = 135617180}
strace -f:
---------
open("/v/01/home/mfister/ulist/run/ulist/logs/access_log", O_WRONLY|O_APPEND|O_C
REAT, 0644) = 3
fcntl64(3, F_DUPFD, 15) = 17
close(3) = 0
chdir("/") = 0
fork() = 32715
[pid 32715] getpid() = 32715
[pid 32715] getrlimit(0x3, 0xbffff154) = 0
[pid 32715] setrlimit(RLIMIT_STACK, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFIN
ITY}) = 0
[pid 32715] setsid() = 32715
[pid 32715] close(0) = 0
[pid 32715] open("/dev/null", O_RDONLY) = 0
[pid 32715] close(1) = 0
[pid 32715] open("/dev/null", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 1
[pid 32715] getpid() = 32715
[pid 32715] time(NULL) = 1012968984
[pid 32715] close(17) = 0
[pid 32715] close(16) = 0
[pid 32714] _exit(0) = ?
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No su
ch file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such
file or directory)
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RT_0], 8) = 0
--- SIGSEGV (Segmentation fault) ---
Apache's config.status:
----------------------
CC="gcc -pipe -g -I/usr/include/db1" \
CFLAGS="-fPIC" \
LIBS="-lpthread -ldb" \
./configure \
"--with-layout=Apache" \
"--prefix=/home/mfister/vendor/apache-1.3.23" \
"--sbindir=/home/mfister/vendor/apache-1.3.23/sbin" \
"--mandir=/home/mfister/vendor/apache-1.3.23/man" \
"--datadir=/home/mfister/vendor/apache-1.3.23/www" \
"--iconsdir=/home/mfister/vendor/apache-1.3.23/www/icons" \
"--htdocsdir=/home/mfister/vendor/apache-1.3.23/www/html" \
"--cgidir=/home/mfister/vendor/apache-1.3.23/www/cgi-bin" \
"--localstatedir=/home/mfister/vendor/apache-1.3.23/www/logs" \
"--runtimedir=/home/mfister/vendor/apache-1.3.23/www/logs" \
"--logfiledir=/home/mfister/vendor/apache-1.3.23/www/logs" \
"--proxycachedir=/home/mfister/vendor/apache-1.3.23/www/cache" \
"--enable-module=all" \
"--enable-shared=max" \
"--disable-rule=WANTHSREGEX" \
"--disable-rule=EXPAT" \
"--enable-module=auth_dbm" \
"--with-perl=/home/mfister/vendor/perl-5.6.1/bin/perl" \
MOD_PERL_TRACE=all
------------------
*** stderr of parent Apache process:
perl_parse args: '/dev/null' ...allocating perl interpreter...ok
constructing perl interpreter...ok
ok
running perl interpreter...ok
mod_perl: 0 END blocks encountered during server startup
loading perl module 'Apache'...loading perl module 'Apache::Constants::Exports'...ok
ok
mod_perl: calling perl_startup()
init `PerlHandler' stack
perl_cmd_push_handlers: @PerlHandler, 'AMPRPC::Server'
pushing `AMPRPC::Server' into `PerlHandler' handlers
init `PerlChildInitHandler' stack
perl_cmd_push_handlers: @PerlChildInitHandler, 'AMPRPC::Server::handle_init'
pushing `AMPRPC::Server::handle_init' into `PerlChildInitHandler' handlers
init `PerlChildExitHandler' stack
perl_cmd_push_handlers: @PerlChildExitHandler, 'AMPRPC::Server::handle_exit'
pushing `AMPRPC::Server::handle_exit' into `PerlChildExitHandler' handlers
init `PerlTransHandler' stack
perl_cmd_push_handlers: @PerlTransHandler, 'ApacheRequestWatcher::start'
pushing `ApacheRequestWatcher::start' into `PerlTransHandler' handlers
init `PerlLogHandler' stack
perl_cmd_push_handlers: @PerlLogHandler, 'ApacheRequestWatcher::stop'
pushing `ApacheRequestWatcher::stop' into `PerlLogHandler' handlers
perl_cmd_pass_env: arg=`LC_ALL'
perl_cmd_pass_env: arg=`NLS_LANG'
perl_cmd_pass_env: arg=`DN_RELBASE'
perl_cmd_pass_env: arg=`PERL5LIB'
perl_cmd_pass_env: arg=`SQUID_PORT'
perl_cmd_pass_env: arg=`DBG'
perl_cmd_pass_env: arg=`UID'
perl_cmd_pass_env: arg=`USER'
perl_cmd_pass_env: arg=`CONFIG_FILE'
perl_cmd_pass_env: arg=`ORACLE_HOME'
perl_cmd_pass_env: arg=`LD_LIBRARY_PATH'
perl_cmd_pass_env: arg=`PATH'
PerlRequire: arg=`AMPRPC/AMPRPCStartup.pl'
attempting to require `AMPRPC/AMPRPCStartup.pl'
I will use localhost:11222 to reach CCS server
I will use localhost:10222 to reach IMGSUBMIT server
I will use product_au:ulist/tsilu (hqtest0.deja.com/1521:UTF8) to reach ORACLE server
init `PerlHandler' stack
perl_cmd_push_handlers: @PerlHandler, 'Apache::Status'
pushing `Apache::Status' into `PerlHandler' handlers
*** stderr once error_log is stderr:
loading perl module 'Apache'...ok
perl_startup: perl aleady running...ok
cleanup_sv: SvREFCNT(0x8128804)==1
cleanup_sv: SvREFCNT(0x80aa590)==1
cleanup_sv: SvREFCNT(0x80aa4e8)==1
cleanup_sv: SvREFCNT(0x80aa758)==1
cleanup_sv: SvREFCNT(0x80aa770)==1
cleanup_sv: SvREFCNT(0x811b328)==1
Time::HiRes dl handle == 0x812d118
POSIX dl handle == 0x813e3d8
Fcntl dl handle == 0x8231f08
Storable dl handle == 0x8221c18
HTMunger dl handle == 0x8235ed8
Data::Dumper dl handle == 0x825dbd0
Sys::Hostname dl handle == 0x8273a28
IO dl handle == 0x82909c0
DBI dl handle == 0x836e4f0
Image::Magick dl handle == 0x87c4ca8
MD5 dl handle == 0x87f3008
DBD::Oracle dl handle == 0x8cab508
running 3 END blocks for perl_shutdown
destructing and freeing Perl interpreter (level=2)...
--
\_/} Mark P. Fister Java, Java, everywhere, and all \_/}
\_/} eBay, Inc. the cups did shrink; Java, Java \_/}
\_/} Austin, TX everywhere, nor any drop to drink! \_/}
Re: [Q] SIGSEGV After fork()
Posted by Ged Haywood <ge...@www2.jubileegroup.co.uk>.
Hi there,
On Wed, 6 Feb 2002, Mark P. Fister wrote:
> Collectively, we've been at this for more than two weeks and have searched
> various mod_perl archives, all to no avail.
:(
> SIGSEGV after fork(). Very reproducible. Memory corruption gets moved
> around if the codebase changes.
[snip]
> Perl:
[snip]
> config_args='-des -Doptimize=-g -Dusedevel -Uinstallusrbinperl -Ubincompat5005 -Uusemymalloc -Dcc=gcc -pipe -g -I/usr/include/db3 -Dcccdlflags=-fPIC -Dinstallprefix=/home/mfister/vendor/perl-5.6.1 -Dprefix=/home/mfister/vendor/perl-5.6.1 -Dcf_email=mfister@ebay.com -Dinstallman1dir=/home/mfister/vendor/perl-5.6.1/man/man1 -Dinstallman3dir=/home/mfister/vendor/perl-5.6.1/man/man3 -Dman1dir=/home/mfister/vendor/perl-5.6.1/man/man1 -Dman3dir=/home/mfister/vendor/perl-5.6.1/man/man3 -Dd_dosuid=n -Dd_semctl_semun -Di_db -Di_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Uuselargefiles'
You might try usemymalloc.
> Compiler:
> optimize='-g',
Hmmmm...
> ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.1 2.96-85)', gccosandvers=''
You've obviously read the docs, so I take it the same compiler built
Aapche, mod_perl and Perl. Have you tried this on RH6.2 with the
compiler that came with that?
> Stack Trace:
> ===========
> #0 __pthread_mutex_lock (mutex=0x8bf04999) at mutex.c:99
> #1 0x401b9cc8 in __libc_free (mem=0x4046cc18) at malloc.c:3152
> #2 0x403ce028 in Perl_safesysfree (where=0x4046cc18) at util.c:158
> #3 0x403f20d8 in Perl_sv_clear (sv=0x8198f60) at sv.c:3827
> #4 0x403f2473 in Perl_sv_free (sv=0x8198f60) at sv.c:3950
> #5 0x403f80e1 in do_clean_all (sv=0x8198f60) at sv.c:8411
> #6 0x403e9c5e in S_visit (f=0x403f8094 <do_clean_all>) at sv.c:162
> #7 0x403e9ce2 in Perl_sv_clean_all () at sv.c:193
> #8 0x4038594a in perl_destruct (my_perl=0x809a9a8) at perl.c:665
> #9 0x4035629c in perl_shutdown (s=0x0, p=0x0) at mod_perl.c:294
> #10 0x40356be6 in mp_dso_unload (data=0x808e714) at mod_perl.c:489
Have you tried a statically linked mod_perl?
73,
Ged.