You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Marc Gracia <mg...@oasyssoft.com> on 2004/11/05 13:38:44 UTC
How to get a core dump
Hi everybody.
I have a problem on a production cluster with a somewhat big mod_perl
app, and I just cannot get any clue of what is happening.
The problem is that the servers just exit with Segmentation fault
randomly.
The problem is rare, hapens 10/20 times each day in each of the 6
frontends, which globaly processes about 1.000.000 daily hits.
The global stats show about 30 Internal server errors daily, I don't
know if a segfault can cause an Internal Server Error on the client (I
suppose not, if the server dies, cannot send the 505), but the numbers
don't match anyway.
I think a coredump will help me understand why it segfaults, but I don't
know how to make apache dump a coredump, I've tried a lot of recipes
found on internet with any success. Making things more complicated, this
problems only happens on the production systems, (Suppose only on some
pages...) so I cannot reproduce it on my test system.
So, my question is... There is any way to force apache to dump a
coredump file? I suppose I'm forgotting something but I really
desperate...
A secondary question is, some of the servers transforms all UTF8 strings
with "garbage" when some Segfault happens (Seems like double-encoded
UTF8, the page shows 3 or 4 chars for every UTF8 char...). The only way
to solve this is reboot the machine completely.
Is that related to this same problem? Or is an obscure UTF8 perl/Apache
problem?
Many thanks,
I'm using mod_perl 1.29 with apache 1.3.31.
My perl conf:
Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
Platform:
osname=linux, osvers=2.4.20-2.48smp,
archname=i386-linux-thread-multi
uname='linux str'
config_args='-des -Doptimize=-O2 -march=i386 -mcpu=i686 -g
-Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red
Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux
-Dvendorprefix=/usr -Dsiteprefix=/usr
-Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads
-Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db
-Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
-Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less
-isr'
hint=recommended, useposix=true, d_sigaction=define
usethreads=define use5005threads=undef'
useithreads=define usemultiplicity=
useperlio= d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=un uselongdouble=
usemymalloc=, bincompat5005=undef
Compiler:
cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBUGGING -fno-strict-aliasing -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
optimize='',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING
-fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
ccversion='', gccversion='3.2.2 20030213 (Red Hat Linux 8.0
3.2.2-1)', gccosandvers=''
gccversion='3.2.2 200302'
intsize=e, longsize= , ptrsize=p, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long'
k', ivsize=4'
ivtype='long'
known_ext, nvtype='double'
o_nonbl', nvsize=, Off_t='', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='gcc'
l', ldflags =' -L/usr/local/lib'
ldf'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil
perllibs=
libc=/lib/libc-2.3.1.so, so=so, useshrplib=true, libperl=libper
gnulibc_version='2.3.1'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic
-Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE'
cccdlflags='-fPIC'
ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s
Unicode/Normalize XS/A'
Characteristics of this binary (from libperl):
Compile-time options: DEBUGGING MULTIPLICITY USE_ITHREADS
USE_LARGE_FILES PERL_IMPLICIT_CONTEXT
Locally applied patches:
MAINT18379
Built under linux
Compiled at Feb 18 2003 22:19:53
%ENV:
PERL5LIB="/usr/eBD/perl"
@INC:
/usr/eBD/perl/i386-linux-thread-multi
/usr/eBD/perl
/usr/lib/perl5/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/5.8.0
/usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/site_perl/5.8.0
/usr/lib/perl5/site_perl
/usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.8.0
/usr/lib/perl5/vendor_perl
/usr/lib/perl5/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/5.8.0
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: How to get a core dump
Posted by Stas Bekman <st...@stason.org>.
Marc Gracia wrote:
[...]
> So, my question is... There is any way to force apache to dump a
> coredump file? I suppose I'm forgotting something but I really
> desperate...
Please read:
http://perl.apache.org/docs/2.0/devel/debug/c.html#Getting_the_core_File_Dumped
> A secondary question is, some of the servers transforms all UTF8 strings
> with "garbage" when some Segfault happens (Seems like double-encoded
> UTF8, the page shows 3 or 4 chars for every UTF8 char...). The only way
> to solve this is reboot the machine completely.
> Is that related to this same problem? Or is an obscure UTF8 perl/Apache
> problem?
Not sure what is the problem here. Any difference if you upgrade to perl
5.8.5? UTF8 support is constantly being improved by newer perl versions.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: How to get a core dump
Posted by Marc Gracia <mg...@oasyssoft.com>.
Yes, I've just taked in account that it happens at clean-up time, at
perl shutdown.
So I suppose that's nothing wrong with this.
Don't know why, I supposed it was related to the GTopLimit automatic
server kill. Now I see that it can be true.
My intention was to try to find out if this coredumps where related to
the sudden change of UTF8 codification, which is the main problem I have
now. Once this happens the only solution is to reboot the entire
machine.
Some Segfault like that can lead to a glibc or iconv library corruption?
The fact that a reboot is needed makes me suspect that glibc is
involved. It's right this assumption?
Many thanks again.
On dl, 2004-11-08 at 15:48, David Hodgkinson wrote:
> On 8 Nov 2004, at 14:39, Marc Gracia wrote:
> >> So, my question is... There is any way to force apache to dump a
> >> coredump file? I suppose I'm forgotting something but I really
> >> desperate...
>
> Yes.
>
> As root, you need to do the ulimit magic and then start the server.
>
> My question is: do you *really* need to debug this? For sure, the
> process
> has got its knickers in a twist, but this is happening at cleanup time
> anyway and not during any request.
>
> I'm sure you have better things to worry about.
>
> Cheers,
>
> Dave
>
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: How to get a core dump
Posted by David Hodgkinson <da...@rockitfactory.com>.
On 8 Nov 2004, at 14:39, Marc Gracia wrote:
>> So, my question is... There is any way to force apache to dump a
>> coredump file? I suppose I'm forgotting something but I really
>> desperate...
Yes.
As root, you need to do the ulimit magic and then start the server.
My question is: do you *really* need to debug this? For sure, the
process
has got its knickers in a twist, but this is happening at cleanup time
anyway and not during any request.
I'm sure you have better things to worry about.
Cheers,
Dave
--
Dave Hodgkinson
CTO, Rockit Factory Ltd.
http://www.rockitfactory.com/
Web sites for rock bands
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: How to get a core dump
Posted by Stas Bekman <st...@stason.org>.
Marc Gracia wrote:
> Sorry, here the backtrace...
>
> #0 0x4018de75 in Perl_hv_free_ent (my_perl=0x85ba6d8, hv=0xd103fd0,
> entry=0xcfdb698) at hv.c:1592
> #1 0x4018e11b in S_hfreeentries (my_perl=0x85ba6d8, hv=0x85ba6d8) at
> hv.c:1681
> #2 0x4018e182 in Perl_hv_undef (my_perl=0x85ba6d8, hv=0xd103fd0) at
> hv.c:1707
> #3 0x401a4367 in Perl_sv_clear (my_perl=0x85ba6d8, sv=0xd103fd0) at
> sv.c:5051
> #4 0x401a49e4 in Perl_sv_free (my_perl=0x85ba6d8, sv=0xd103fd0) at
> sv.c:5226
> #5 0x4019be83 in do_clean_all (my_perl=0x85ba6d8, sv=0xd103fd0) at
> sv.c:422
> #6 0x4019bba2 in S_visit (my_perl=0x85ba6d8, f=0x4019be40
> <do_clean_all>)
> at sv.c:314
> #7 0x4019bee3 in Perl_sv_clean_all (my_perl=0x85ba6d8) at sv.c:440
> #8 0x4012fa04 in perl_destruct (my_perl=0x85ba6d8) at perl.c:796
> #9 0x400be0f3 in perl_shutdown (s=0x809979c, p=0xa1d9fd4) at
> mod_perl.c:294
> #10 0x400c0ca1 in perl_child_exit (s=0x809979c, p=0xa1d9fd4) at
> mod_perl.c:965
> #11 0x400c0958 in perl_child_exit_cleanup (data=0xd103fd0) at
> mod_perl.c:933
It's hard to tell whether this is a bug in perl or some module that you
use. Shutdown bugs are really hard to deal with. If you could reproduce
that at will, trying to eliminate modules, programs should help to find
the guilty party. Since things happen at the server shutdown, you could
set a low limit on the max number of requests served before retiring the
process.
Also could you try to upgrade to 5.8.5 just to check that may be it's some
bug that was fixed in perl since 5.8.0?
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: How to get a core dump
Posted by Marc Gracia <mg...@oasyssoft.com>.
Sorry, here the backtrace...
#0 0x4018de75 in Perl_hv_free_ent (my_perl=0x85ba6d8, hv=0xd103fd0,
entry=0xcfdb698) at hv.c:1592
#1 0x4018e11b in S_hfreeentries (my_perl=0x85ba6d8, hv=0x85ba6d8) at
hv.c:1681
#2 0x4018e182 in Perl_hv_undef (my_perl=0x85ba6d8, hv=0xd103fd0) at
hv.c:1707
#3 0x401a4367 in Perl_sv_clear (my_perl=0x85ba6d8, sv=0xd103fd0) at
sv.c:5051
#4 0x401a49e4 in Perl_sv_free (my_perl=0x85ba6d8, sv=0xd103fd0) at
sv.c:5226
#5 0x4019be83 in do_clean_all (my_perl=0x85ba6d8, sv=0xd103fd0) at
sv.c:422
#6 0x4019bba2 in S_visit (my_perl=0x85ba6d8, f=0x4019be40
<do_clean_all>)
at sv.c:314
#7 0x4019bee3 in Perl_sv_clean_all (my_perl=0x85ba6d8) at sv.c:440
#8 0x4012fa04 in perl_destruct (my_perl=0x85ba6d8) at perl.c:796
#9 0x400be0f3 in perl_shutdown (s=0x809979c, p=0xa1d9fd4) at
mod_perl.c:294
#10 0x400c0ca1 in perl_child_exit (s=0x809979c, p=0xa1d9fd4) at
mod_perl.c:965
#11 0x400c0958 in perl_child_exit_cleanup (data=0xd103fd0) at
mod_perl.c:933
#12 0x08051976 in run_cleanups (c=0xa1da164) at alloc.c:1936
#13 0x08050104 in ap_clear_pool (a=0xa1d9fd4) at alloc.c:650
#14 0x08050178 in ap_destroy_pool (a=0xa1d9fd4) at alloc.c:680
#15 0x0805ea10 in clean_child_exit (code=0) at http_main.c:519
#16 0x08061ef5 in child_main (child_num_arg=7) at http_main.c:4558
#17 0x08062659 in make_child (s=0x809979c, slot=7, now=1099922742)
at http_main.c:5051
#18 0x080629bc in perform_idle_server_maintenance () at http_main.c:5236
#19 0x08063068 in standalone_main (argc=1, argv=0xbffff0c4) at
http_main.c:5499
---Type <return> to continue, or q <return> to quit---
#20 0x080636cf in main (argc=1, argv=0xbffff0c4) at http_main.c:5767
#21 0x42015574 in __libc_start_main () from /lib/tls/libc.so.6
On dv, 2004-11-05 at 13:38, Marc Gracia wrote:
> Hi everybody.
> I have a problem on a production cluster with a somewhat big mod_perl
> app, and I just cannot get any clue of what is happening.
>
> The problem is that the servers just exit with Segmentation fault
> randomly.
> The problem is rare, hapens 10/20 times each day in each of the 6
> frontends, which globaly processes about 1.000.000 daily hits.
> The global stats show about 30 Internal server errors daily, I don't
> know if a segfault can cause an Internal Server Error on the client (I
> suppose not, if the server dies, cannot send the 505), but the numbers
> don't match anyway.
>
> I think a coredump will help me understand why it segfaults, but I don't
> know how to make apache dump a coredump, I've tried a lot of recipes
> found on internet with any success. Making things more complicated, this
> problems only happens on the production systems, (Suppose only on some
> pages...) so I cannot reproduce it on my test system.
>
> So, my question is... There is any way to force apache to dump a
> coredump file? I suppose I'm forgotting something but I really
> desperate...
>
> A secondary question is, some of the servers transforms all UTF8 strings
> with "garbage" when some Segfault happens (Seems like double-encoded
> UTF8, the page shows 3 or 4 chars for every UTF8 char...). The only way
> to solve this is reboot the machine completely.
> Is that related to this same problem? Or is an obscure UTF8 perl/Apache
> problem?
>
> Many thanks,
> I'm using mod_perl 1.29 with apache 1.3.31.
> My perl conf:
>
> Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
> Platform:
> osname=linux, osvers=2.4.20-2.48smp,
> archname=i386-linux-thread-multi
> uname='linux str'
> config_args='-des -Doptimize=-O2 -march=i386 -mcpu=i686 -g
> -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red
> Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux
> -Dvendorprefix=/usr -Dsiteprefix=/usr
> -Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads
> -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db
> -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
> -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less
> -isr'
> hint=recommended, useposix=true, d_sigaction=define
> usethreads=define use5005threads=undef'
> useithreads=define usemultiplicity=
> useperlio= d_sfio=undef uselargefiles=define usesocks=undef
> use64bitint=undef use64bitall=un uselongdouble=
> usemymalloc=, bincompat5005=undef
> Compiler:
> cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
> -DDEBUGGING -fno-strict-aliasing -I/usr/local/include
> -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
> optimize='',
> cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING
> -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
> ccversion='', gccversion='3.2.2 20030213 (Red Hat Linux 8.0
> 3.2.2-1)', gccosandvers=''
> gccversion='3.2.2 200302'
> intsize=e, longsize= , ptrsize=p, doublesize=8, byteorder=1234
> d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
> ivtype='long'
> k', ivsize=4'
> ivtype='long'
> known_ext, nvtype='double'
> o_nonbl', nvsize=, Off_t='', lseeksize=8
> alignbytes=4, prototype=define
> Linker and Libraries:
> ld='gcc'
> l', ldflags =' -L/usr/local/lib'
> ldf'
> libpth=/usr/local/lib /lib /usr/lib
> libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil
> perllibs=
> libc=/lib/libc-2.3.1.so, so=so, useshrplib=true, libperl=libper
> gnulibc_version='2.3.1'
> Dynamic Linking:
> dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic
> -Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE'
> cccdlflags='-fPIC'
> ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s
> Unicode/Normalize XS/A'
>
>
> Characteristics of this binary (from libperl):
> Compile-time options: DEBUGGING MULTIPLICITY USE_ITHREADS
> USE_LARGE_FILES PERL_IMPLICIT_CONTEXT
> Locally applied patches:
> MAINT18379
> Built under linux
> Compiled at Feb 18 2003 22:19:53
> %ENV:
> PERL5LIB="/usr/eBD/perl"
> @INC:
> /usr/eBD/perl/i386-linux-thread-multi
> /usr/eBD/perl
> /usr/lib/perl5/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/5.8.0
> /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/site_perl/5.8.0
> /usr/lib/perl5/site_perl
> /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/vendor_perl/5.8.0
> /usr/lib/perl5/vendor_perl
> /usr/lib/perl5/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/5.8.0
>
>
>
>
>
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: How to get a core dump
Posted by Glenn Strauss <gs...@gluelogic.com>.
On Fri, Nov 05, 2004 at 01:38:44PM +0100, Marc Gracia wrote:
> So, my question is... There is any way to force apache to dump a
> coredump file? I suppose I'm forgotting something but I really
> desperate...
On Linux, create a directory that is writable by the httpd user
mkdir /var/tmp/apache-cores
chown httpd /var/tmp/apache-cores
chmod 700 /var/tmp/apache-cores
Set the dump directory in httpd.conf:
CoreDumpDirectory /var/tmp/apache-cores
Enable core dumps in your shell and then start Apache
ulimit -c unlimited (for bash; different for csh)
/usr/local/apache/httpd
To test that you can get a dump, start up an httpd process,
send a SIGABRT, SIGBUS, or SIGSEGV to one of the _children_,
and check that it dumped. The error log should indicate
that a possible dump was saved to /var/tmp/apache-cores.
kill -11 <apache_child_pid>
('kill -l' (that's a lowercase letter L) to get a list
of signals and their numbers for your system)
Of course, this will all be a lot more useful to you if you
build Apache with debugging enabled. When building, set
OPTIM=-g before './configure' in the Apache source dir, e.g.
OPTIM=-g ./configure ...
or before 'perl Makefile.PL' in the mod_perl source dir, e.g.
OPTIM=-g perl Makefile.PL ...
depending on how you build Apache and mod_perl.
Cheers,
Glenn
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: How to get a core dump
Posted by Marc Gracia <mg...@oasyssoft.com>.
Hi,
Well, It's only 2 hours of on line testing, but seems that the
recompilation and upgrade to apache-1.3.33 I did with "-g" to get debug
info on coredumps solved the problem....
I used exactly the same configure options on apache,mod_ssl and mod_perl
(I used the same shell script, in fact)
Now I don't know what caused the problem first place, but now seems ok.
I taked in account that first time I compiled all on a shell with
"en_US" locale, and now I used one with "en_US.UTF8".
May the locale used in the shell to compile apache+mod_perl affect the
final executable in some way?
The server has to deal with UTF8 info coming from/going to a SQLServer
backend...
On dv, 2004-11-05 at 17:46, Marc Gracia wrote:
> Many Thanks Stass and Glenn,
> I'll try all this anf will get back..
>
> On dv, 2004-11-05 at 13:38, Marc Gracia wrote:
> > Hi everybody.
> > I have a problem on a production cluster with a somewhat big mod_perl
> > app, and I just cannot get any clue of what is happening.
> >
> > The problem is that the servers just exit with Segmentation fault
> > randomly.
> > The problem is rare, hapens 10/20 times each day in each of the 6
> > frontends, which globaly processes about 1.000.000 daily hits.
> > The global stats show about 30 Internal server errors daily, I don't
> > know if a segfault can cause an Internal Server Error on the client (I
> > suppose not, if the server dies, cannot send the 505), but the numbers
> > don't match anyway.
> >
> > I think a coredump will help me understand why it segfaults, but I don't
> > know how to make apache dump a coredump, I've tried a lot of recipes
> > found on internet with any success. Making things more complicated, this
> > problems only happens on the production systems, (Suppose only on some
> > pages...) so I cannot reproduce it on my test system.
> >
> > So, my question is... There is any way to force apache to dump a
> > coredump file? I suppose I'm forgotting something but I really
> > desperate...
> >
> > A secondary question is, some of the servers transforms all UTF8 strings
> > with "garbage" when some Segfault happens (Seems like double-encoded
> > UTF8, the page shows 3 or 4 chars for every UTF8 char...). The only way
> > to solve this is reboot the machine completely.
> > Is that related to this same problem? Or is an obscure UTF8 perl/Apache
> > problem?
> >
> > Many thanks,
> > I'm using mod_perl 1.29 with apache 1.3.31.
> > My perl conf:
> >
> > Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
> > Platform:
> > osname=linux, osvers=2.4.20-2.48smp,
> > archname=i386-linux-thread-multi
> > uname='linux str'
> > config_args='-des -Doptimize=-O2 -march=i386 -mcpu=i686 -g
> > -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red
> > Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux
> > -Dvendorprefix=/usr -Dsiteprefix=/usr
> > -Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads
> > -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db
> > -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
> > -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less
> > -isr'
> > hint=recommended, useposix=true, d_sigaction=define
> > usethreads=define use5005threads=undef'
> > useithreads=define usemultiplicity=
> > useperlio= d_sfio=undef uselargefiles=define usesocks=undef
> > use64bitint=undef use64bitall=un uselongdouble=
> > usemymalloc=, bincompat5005=undef
> > Compiler:
> > cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
> > -DDEBUGGING -fno-strict-aliasing -I/usr/local/include
> > -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
> > optimize='',
> > cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING
> > -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
> > ccversion='', gccversion='3.2.2 20030213 (Red Hat Linux 8.0
> > 3.2.2-1)', gccosandvers=''
> > gccversion='3.2.2 200302'
> > intsize=e, longsize= , ptrsize=p, doublesize=8, byteorder=1234
> > d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
> > ivtype='long'
> > k', ivsize=4'
> > ivtype='long'
> > known_ext, nvtype='double'
> > o_nonbl', nvsize=, Off_t='', lseeksize=8
> > alignbytes=4, prototype=define
> > Linker and Libraries:
> > ld='gcc'
> > l', ldflags =' -L/usr/local/lib'
> > ldf'
> > libpth=/usr/local/lib /lib /usr/lib
> > libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil
> > perllibs=
> > libc=/lib/libc-2.3.1.so, so=so, useshrplib=true, libperl=libper
> > gnulibc_version='2.3.1'
> > Dynamic Linking:
> > dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic
> > -Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE'
> > cccdlflags='-fPIC'
> > ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s
> > Unicode/Normalize XS/A'
> >
> >
> > Characteristics of this binary (from libperl):
> > Compile-time options: DEBUGGING MULTIPLICITY USE_ITHREADS
> > USE_LARGE_FILES PERL_IMPLICIT_CONTEXT
> > Locally applied patches:
> > MAINT18379
> > Built under linux
> > Compiled at Feb 18 2003 22:19:53
> > %ENV:
> > PERL5LIB="/usr/eBD/perl"
> > @INC:
> > /usr/eBD/perl/i386-linux-thread-multi
> > /usr/eBD/perl
> > /usr/lib/perl5/5.8.0/i386-linux-thread-multi
> > /usr/lib/perl5/5.8.0
> > /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
> > /usr/lib/perl5/site_perl/5.8.0
> > /usr/lib/perl5/site_perl
> > /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
> > /usr/lib/perl5/vendor_perl/5.8.0
> > /usr/lib/perl5/vendor_perl
> > /usr/lib/perl5/5.8.0/i386-linux-thread-multi
> > /usr/lib/perl5/5.8.0
> >
> >
> >
> >
> --
> Marc Gracia <mg...@oasyssoft.com>
> Oasys Soft
>
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: How to get a core dump
Posted by Marc Gracia <mg...@oasyssoft.com>.
Many Thanks Stass and Glenn,
I'll try all this anf will get back..
On dv, 2004-11-05 at 13:38, Marc Gracia wrote:
> Hi everybody.
> I have a problem on a production cluster with a somewhat big mod_perl
> app, and I just cannot get any clue of what is happening.
>
> The problem is that the servers just exit with Segmentation fault
> randomly.
> The problem is rare, hapens 10/20 times each day in each of the 6
> frontends, which globaly processes about 1.000.000 daily hits.
> The global stats show about 30 Internal server errors daily, I don't
> know if a segfault can cause an Internal Server Error on the client (I
> suppose not, if the server dies, cannot send the 505), but the numbers
> don't match anyway.
>
> I think a coredump will help me understand why it segfaults, but I don't
> know how to make apache dump a coredump, I've tried a lot of recipes
> found on internet with any success. Making things more complicated, this
> problems only happens on the production systems, (Suppose only on some
> pages...) so I cannot reproduce it on my test system.
>
> So, my question is... There is any way to force apache to dump a
> coredump file? I suppose I'm forgotting something but I really
> desperate...
>
> A secondary question is, some of the servers transforms all UTF8 strings
> with "garbage" when some Segfault happens (Seems like double-encoded
> UTF8, the page shows 3 or 4 chars for every UTF8 char...). The only way
> to solve this is reboot the machine completely.
> Is that related to this same problem? Or is an obscure UTF8 perl/Apache
> problem?
>
> Many thanks,
> I'm using mod_perl 1.29 with apache 1.3.31.
> My perl conf:
>
> Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
> Platform:
> osname=linux, osvers=2.4.20-2.48smp,
> archname=i386-linux-thread-multi
> uname='linux str'
> config_args='-des -Doptimize=-O2 -march=i386 -mcpu=i686 -g
> -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red
> Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux
> -Dvendorprefix=/usr -Dsiteprefix=/usr
> -Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads
> -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db
> -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
> -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less
> -isr'
> hint=recommended, useposix=true, d_sigaction=define
> usethreads=define use5005threads=undef'
> useithreads=define usemultiplicity=
> useperlio= d_sfio=undef uselargefiles=define usesocks=undef
> use64bitint=undef use64bitall=un uselongdouble=
> usemymalloc=, bincompat5005=undef
> Compiler:
> cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
> -DDEBUGGING -fno-strict-aliasing -I/usr/local/include
> -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
> optimize='',
> cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING
> -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
> ccversion='', gccversion='3.2.2 20030213 (Red Hat Linux 8.0
> 3.2.2-1)', gccosandvers=''
> gccversion='3.2.2 200302'
> intsize=e, longsize= , ptrsize=p, doublesize=8, byteorder=1234
> d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
> ivtype='long'
> k', ivsize=4'
> ivtype='long'
> known_ext, nvtype='double'
> o_nonbl', nvsize=, Off_t='', lseeksize=8
> alignbytes=4, prototype=define
> Linker and Libraries:
> ld='gcc'
> l', ldflags =' -L/usr/local/lib'
> ldf'
> libpth=/usr/local/lib /lib /usr/lib
> libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil
> perllibs=
> libc=/lib/libc-2.3.1.so, so=so, useshrplib=true, libperl=libper
> gnulibc_version='2.3.1'
> Dynamic Linking:
> dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic
> -Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE'
> cccdlflags='-fPIC'
> ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s
> Unicode/Normalize XS/A'
>
>
> Characteristics of this binary (from libperl):
> Compile-time options: DEBUGGING MULTIPLICITY USE_ITHREADS
> USE_LARGE_FILES PERL_IMPLICIT_CONTEXT
> Locally applied patches:
> MAINT18379
> Built under linux
> Compiled at Feb 18 2003 22:19:53
> %ENV:
> PERL5LIB="/usr/eBD/perl"
> @INC:
> /usr/eBD/perl/i386-linux-thread-multi
> /usr/eBD/perl
> /usr/lib/perl5/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/5.8.0
> /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/site_perl/5.8.0
> /usr/lib/perl5/site_perl
> /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/vendor_perl/5.8.0
> /usr/lib/perl5/vendor_perl
> /usr/lib/perl5/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/5.8.0
>
>
>
>
--
Marc Gracia <mg...@oasyssoft.com>
Oasys Soft
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: How to get a core dump
Posted by Marc Gracia <mg...@oasyssoft.com>.
Well at last the core where produced.
When called gdb with the httpd executable and the core file, gdb shows
this:
Core was generated by `/usr/eBD/bin/httpd'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/tls/libpthread.so.0...done.
Loaded symbols for /lib/tls/libpthread.so.0
Reading symbols from /lib/tls/libm.so.6...done.
Loaded symbols for /lib/tls/libm.so.6
..
..
..
Loaded symbols for
/usr/eBD/perl//i386-linux-thread-multi/auto/DBD/ODBC/ODBC.so
Reading symbols from /usr//lib/libodbc.so.1...done.
Loaded symbols for /usr//lib/libodbc.so.1
Reading symbols from /usr/lib/gconv/ISO8859-1.so...done.
Loaded symbols for /usr/lib/gconv/ISO8859-1.so
#0 0x4018de75 in Perl_hv_free_ent (my_perl=0x85ba6d8, hv=0xd103fd0,
entry=0xcfdb698) at hv.c:1592
1592 hv.c: No such file or directory.
in hv.c
(gdb)
This hv.c error is because gdb didn't find this file?
Or this is really the originator of the Segfault?
On dv, 2004-11-05 at 13:38, Marc Gracia wrote:
> Hi everybody.
> I have a problem on a production cluster with a somewhat big mod_perl
> app, and I just cannot get any clue of what is happening.
>
> The problem is that the servers just exit with Segmentation fault
> randomly.
> The problem is rare, hapens 10/20 times each day in each of the 6
> frontends, which globaly processes about 1.000.000 daily hits.
> The global stats show about 30 Internal server errors daily, I don't
> know if a segfault can cause an Internal Server Error on the client (I
> suppose not, if the server dies, cannot send the 505), but the numbers
> don't match anyway.
>
> I think a coredump will help me understand why it segfaults, but I don't
> know how to make apache dump a coredump, I've tried a lot of recipes
> found on internet with any success. Making things more complicated, this
> problems only happens on the production systems, (Suppose only on some
> pages...) so I cannot reproduce it on my test system.
>
> So, my question is... There is any way to force apache to dump a
> coredump file? I suppose I'm forgotting something but I really
> desperate...
>
> A secondary question is, some of the servers transforms all UTF8 strings
> with "garbage" when some Segfault happens (Seems like double-encoded
> UTF8, the page shows 3 or 4 chars for every UTF8 char...). The only way
> to solve this is reboot the machine completely.
> Is that related to this same problem? Or is an obscure UTF8 perl/Apache
> problem?
>
> Many thanks,
> I'm using mod_perl 1.29 with apache 1.3.31.
> My perl conf:
>
> Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
> Platform:
> osname=linux, osvers=2.4.20-2.48smp,
> archname=i386-linux-thread-multi
> uname='linux str'
> config_args='-des -Doptimize=-O2 -march=i386 -mcpu=i686 -g
> -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red
> Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux
> -Dvendorprefix=/usr -Dsiteprefix=/usr
> -Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads
> -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db
> -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
> -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less
> -isr'
> hint=recommended, useposix=true, d_sigaction=define
> usethreads=define use5005threads=undef'
> useithreads=define usemultiplicity=
> useperlio= d_sfio=undef uselargefiles=define usesocks=undef
> use64bitint=undef use64bitall=un uselongdouble=
> usemymalloc=, bincompat5005=undef
> Compiler:
> cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
> -DDEBUGGING -fno-strict-aliasing -I/usr/local/include
> -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
> optimize='',
> cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING
> -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
> ccversion='', gccversion='3.2.2 20030213 (Red Hat Linux 8.0
> 3.2.2-1)', gccosandvers=''
> gccversion='3.2.2 200302'
> intsize=e, longsize= , ptrsize=p, doublesize=8, byteorder=1234
> d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
> ivtype='long'
> k', ivsize=4'
> ivtype='long'
> known_ext, nvtype='double'
> o_nonbl', nvsize=, Off_t='', lseeksize=8
> alignbytes=4, prototype=define
> Linker and Libraries:
> ld='gcc'
> l', ldflags =' -L/usr/local/lib'
> ldf'
> libpth=/usr/local/lib /lib /usr/lib
> libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil
> perllibs=
> libc=/lib/libc-2.3.1.so, so=so, useshrplib=true, libperl=libper
> gnulibc_version='2.3.1'
> Dynamic Linking:
> dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic
> -Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE'
> cccdlflags='-fPIC'
> ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s
> Unicode/Normalize XS/A'
>
>
> Characteristics of this binary (from libperl):
> Compile-time options: DEBUGGING MULTIPLICITY USE_ITHREADS
> USE_LARGE_FILES PERL_IMPLICIT_CONTEXT
> Locally applied patches:
> MAINT18379
> Built under linux
> Compiled at Feb 18 2003 22:19:53
> %ENV:
> PERL5LIB="/usr/eBD/perl"
> @INC:
> /usr/eBD/perl/i386-linux-thread-multi
> /usr/eBD/perl
> /usr/lib/perl5/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/5.8.0
> /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/site_perl/5.8.0
> /usr/lib/perl5/site_perl
> /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/vendor_perl/5.8.0
> /usr/lib/perl5/vendor_perl
> /usr/lib/perl5/5.8.0/i386-linux-thread-multi
> /usr/lib/perl5/5.8.0
>
>
>
>
--
Marc Gracia <mg...@oasyssoft.com>
Oasys Soft
--
Report problems: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html