You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Marcin Kasperski <Ma...@acn.waw.pl> on 2002/11/29 00:20:44 UTC
[mp1] callback called exit (many times)
Recently I happened to get the error log flooded with the message
callback called exit
(repeated a couple of million times, probably by one process or a few
processes).
This was some side-effect of the lack of memory (earlier in the logs I
found messages about allocation failures), which in turn was caused by
some non-apache process allocating too much. Nevertheless, I am trying
to find what happened to my apache/modperl process
(processess?). After all, aborting some processes due to lack of
memory is ok, but filling the whole disk with error log is not.
It is possible that the problem is not caused by the modperl itself
(for instance some of additional modules we implemented in both C and
perl could go wild). But trying to understand the problem I would like
to ask the following questions:
a) Have anyone anywhere observed similar behaviour?
b) When is the message 'callback called exit' printed - and can it be
disabled?
c) Do you have any idea of what should I pay attention to?
--
( Marcin Kasperski | Królik to brzmi dumnie! Wszystko dla szympansa! )
( http://www.mk.w.pl | Chomikiem jestem, wszystko mi wolno! (Mrożek) )
(----------------------------------------------------------------------------)
( Sztuczki i kruczki w C++: http://www.mk.w.pl/porady/porady_cplusplus )
Re: [mp1] callback called exit (many times)
Posted by Marcin Kasperski <Ma...@acn.waw.pl>.
> > Am I right deducing, that I should attempt recompiling perl with
>
> > -DEMERGENCY_SBRK?
>
>
> Try to, but this doesn't solve the original problem. You still have to
> try to narrow down the code that causes the problem. Using ab or a
> similar load tester might help to stress test.
>
>
> Using tools to prevent situations when the server runs out of memory
> is very important. See the online docs for more info. Using these
> tools should probably solve the problem altogether, without using any
> special compilation options.
The memory problem itself is already diagnosed (some non-apache
process allocated more than it was expected, therefore less memory
remained for apache/modperl processes) and I will work on
it. Nevertheless I'd like to improve apache behaviour in lack of
memory conditions - if possible.
Thanks for all the hints.
--
( Marcin Kasperski | In any large change, 1/3 will think it is great, 1/3 )
( http://www.mk.w.pl | will think it is stupid, and 1/3 will wait (Reed) )
(----------------------------------------------------------------------------)
( Porady dla twórców serwisów WWW: http://www.mk.w.pl/porady/porady_www )
Re: [mp1] callback called exit (many times)
Posted by Stas Bekman <st...@stason.org>.
Marcin Kasperski wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>Marcin Kasperski wrote:
>>
>>>Recently I happened to get the error log flooded with the message
>>> callback called exit
>>>(repeated a couple of million times, probably by one process or a few
>>> processes).
>>>This was some side-effect of the lack of memory
>>>(...)
>>
>
>
>>Hope that the following somewhat helps:
>>http://perl.apache.org/search/swish.cgi?query=callback+called+exit&sbm=SecD&submit=search
>
>
> Thank you for the pointer.
>
> Am I right deducing, that I should attempt recompiling perl with
> -DEMERGENCY_SBRK?
Try to, but this doesn't solve the original problem. You still have to
try to narrow down the code that causes the problem. Using ab or a
similar load tester might help to stress test.
Using tools to prevent situations when the server runs out of memory is
very important. See the online docs for more info. Using these tools
should probably solve the problem altogether, without using any special
compilation options.
> The perl I use describes itself as below so probably this option is
> not turned on:
Hmm, it's been a long time since we've last revisited this issue. The
doc says that PERL_EMERGENCY_SBRK it's supposed to be on, but a quick
test on several of my builds shows that it's not true:
#!/usr/bin/perl
use Inline C;
print "PERL_EMERGENCY_SBRK is @{[ mytest() ? '' : 'not']} defined\n";
__END__
__C__
int mytest() {
#ifdef PERL_EMERGENCY_SBRK
return 1;
#else
return 0;
#endif
}
~/perl/5.8.0-ithread/bin/perl test.pl
PERL_EMERGENCY_SBRK is not defined
of course there is the source code, but Inline is nice :)
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [mp1] callback called exit (many times)
Posted by Marcin Kasperski <Ma...@acn.waw.pl>.
Stas Bekman <st...@stason.org> writes:
> Marcin Kasperski wrote:
> > Recently I happened to get the error log flooded with the message
> > callback called exit
> > (repeated a couple of million times, probably by one process or a few
> > processes).
> > This was some side-effect of the lack of memory
> > (...)
> Hope that the following somewhat helps:
> http://perl.apache.org/search/swish.cgi?query=callback+called+exit&sbm=SecD&submit=search
Thank you for the pointer.
Am I right deducing, that I should attempt recompiling perl with
-DEMERGENCY_SBRK?
The perl I use describes itself as below so probably this option is
not turned on:
$ perl -V
Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration:
Platform:
osname=dec_osf, osvers=5.1, archname=alpha-dec_osf
uname='osf1 banach.softax.local v5.1 1885 alpha '
config_args='-des -Dprefix=/opt/igowwwroot/tools_test/perl -Uinstallusrbinperl -Duseshrplib=1 -Dusemymalloc -Ubincompat5005'
hint=previous, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=define use64bitall=define uselongdouble=undef
Compiler:
cc='cc', ccflags ='-std -fprm d -ieee -D_INTRINSICS -I/usr/local/include -DLANGUAGE_C',
optimize='-O4',
cppflags='-std -fprm d -ieee -D_INTRINSICS -I/usr/local/include -DLANGUAGE_C'
ccversion='V6.4-009', gccversion='', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, usemymalloc=y, prototype=define
Linker and Libraries:
ld='ld', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /usr/shlib /usr/ccs/lib /usr/lib/cmplrs/cc /usr/lib /var/shlib
libs=-ldbm -ldb -lm -liconv -lutil
perllibs=-lm -liconv -lutil
libc=/usr/shlib/libc.so, so=so, useshrplib=false, libperl=libperl.a
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags='-shared -expect_unresolved "*" -O4 -msym -std -s -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: USE_64_BIT_INT USE_64_BIT_ALL USE_LARGE_FILES
Built under dec_osf
Compiled at Nov 21 2002 15:05:28
@INC:
/opt/igowwwroot/tools/perl/lib/5.6.1/alpha-dec_osf
/opt/igowwwroot/tools/perl/lib/5.6.1
/opt/igowwwroot/tools/perl/lib/site_perl/5.6.1/alpha-dec_osf
/opt/igowwwroot/tools/perl/lib/site_perl/5.6.1
/opt/igowwwroot/tools/perl/lib/site_perl
.
Re: [mp1] callback called exit (many times)
Posted by Stas Bekman <st...@stason.org>.
Marcin Kasperski wrote:
> Recently I happened to get the error log flooded with the message
> callback called exit
> (repeated a couple of million times, probably by one process or a few
> processes).
>
> This was some side-effect of the lack of memory (earlier in the logs I
> found messages about allocation failures), which in turn was caused by
> some non-apache process allocating too much. Nevertheless, I am trying
> to find what happened to my apache/modperl process
> (processess?). After all, aborting some processes due to lack of
> memory is ok, but filling the whole disk with error log is not.
>
> It is possible that the problem is not caused by the modperl itself
> (for instance some of additional modules we implemented in both C and
> perl could go wild). But trying to understand the problem I would like
> to ask the following questions:
>
> a) Have anyone anywhere observed similar behaviour?
>
> b) When is the message 'callback called exit' printed - and can it be
> disabled?
>
> c) Do you have any idea of what should I pay attention to?
Hope that the following somewhat helps:
http://perl.apache.org/search/swish.cgi?query=callback+called+exit&sbm=SecD&submit=search
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com