You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Tom Lancaster <to...@grubby.net> on 2000/06/06 19:20:02 UTC
Apache hanging when it reaches limit of process limit
Hi,
I've got this strange behavior in a production site that until a recent
upgrade never showed this kind of thing:
I've set maxclients to 50, maxrequestsperchild 1.
What happens is that the server runs fine, is fast, etc, but the number
of apache processes climbs gradually until it reaches ~maxclients, then
will no longer serve pages.
I thought this was due to children not exiting, but some of them
apparently do, as the number of processes rises and falls on its way to
the top.
What is Apache supposed to do when you set maxrequestsperchild to 1?
I'd have thought each process would exit immediatley after serving
something.
I'm inclined to think there's something in my code that's hanging, but I
read that the symptom of that is processes consuming large amounts of
CPU - they're not, and memory usage is normal, too (~10M).
Another symptom: sometimes when the processes are nearing the top,
they'll start haging, but one by one; if I hit stop on a non-responsive
request, then re-try, it serves fine.
I'm using the debian package of modperl 1.21 statically linked with
apache 1.3.9, and HTML::Embperl 1.2.1 ( I think).
Any insight greatly appreciated.
Tom
Re: Apache hanging when it reaches limit of process limit
Posted by Doug MacEachern <do...@covalent.net>.
On Tue, 6 Jun 2000, Tom Lancaster wrote:
> Here's the output of gdb: ( now what does this mean?)
...
> Breakpoint 1, 0x400ddd35 in exit () from /lib/libc.so.6
> (gdb) bt
> #0 0x400ddd35 in exit () from /lib/libc.so.6
> #1 0x80851e1 in ap_start_restart ()
> #2 0x8085454 in ap_start_restart ()
> #3 0x80862d7 in ap_child_terminate ()
> #4 0x8086a4d in main ()
> #5 0x400d5a42 in __libc_start_main () from /lib/libc.so.6
> (gdb)
the exit() makes sense, since you have maxrequestsperchild set to 1, but
the stacktrace should look more like this:
(gdb) where
#0 exit (status=0) at exit.c:40
#1 0x80cf0eb in clean_child_exit (code=0) at http_main.c:493
#2 0x80d1e83 in child_main (child_num_arg=0) at http_main.c:3893
#3 0x80d23dc in make_child (s=0x821dbe4, slot=0, now=960572421)
at http_main.c:4264
#4 0x80d2537 in startup_children (number_to_start=5) at http_main.c:4346
#5 0x80d2b54 in standalone_main (argc=6, argv=0xbffff624) at
http_main.c:4634
#6 0x80d32d3 in main (argc=6, argv=0xbffff624) at http_main.c:4961
are you calling $r->child_terminate somewhere? that's not actually called
anywhere in the apache source, it's just an api for modules.
Re: Apache hanging when it reaches limit of process limit
Posted by Tom Lancaster <to...@grubby.net>.
OK, I followed the directions in SUPPORT, for gdb, and got this :
(gdb) run -X -f /usr/local/apache/conf/httpd.conf.test
Starting program: /usr/local/mod_perl-1.24/../apache_1.3.12/src/httpd -X
-f /usr/local/apache/conf/httpd.conf.test
Subroutine Apache::TIEHANDLE redefined at
/usr/lib/perl5/5.005/i386-linux/mod_perl.pm line 65535.
What's causing this?
Stas Bekman wrote:
>
> On Tue, 6 Jun 2000, Tom Lancaster wrote:
>
> > Here's the output of gdb: ( now what does this mean?)
>
> You should compile mod_perl with PERL_DEBUG=1, see the SUPPORT file.
>
> >
> > Starting program: /usr/sbin/apache -X
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...Breakpoint 1 at 0x400ddd35
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...(no debugging symbols found)...
> > (no debugging symbols found)...
> > Breakpoint 1, 0x400ddd35 in exit () from /lib/libc.so.6
> > (gdb) bt
> > #0 0x400ddd35 in exit () from /lib/libc.so.6
> > #1 0x80851e1 in ap_start_restart ()
> > #2 0x8085454 in ap_start_restart ()
> > #3 0x80862d7 in ap_child_terminate ()
> > #4 0x8086a4d in main ()
> > #5 0x400d5a42 in __libc_start_main () from /lib/libc.so.6
> > (gdb)
> >
> >
> > Stas Bekman wrote:
> > >
> > > On Tue, 6 Jun 2000, Tom Lancaster wrote:
> > >
> > > > Hi,
> > > > I've got this strange behavior in a production site that until a recent
> > > > upgrade never showed this kind of thing:
> > > > I've set maxclients to 50, maxrequestsperchild 1.
> > > > What happens is that the server runs fine, is fast, etc, but the number
> > > > of apache processes climbs gradually until it reaches ~maxclients, then
> > > > will no longer serve pages.
> > > > I thought this was due to children not exiting, but some of them
> > > > apparently do, as the number of processes rises and falls on its way to
> > > > the top.
> > > >
> > > > What is Apache supposed to do when you set maxrequestsperchild to 1?
> > > > I'd have thought each process would exit immediatley after serving
> > > > something.
> > > > I'm inclined to think there's something in my code that's hanging, but I
> > > > read that the symptom of that is processes consuming large amounts of
> > > > CPU - they're not, and memory usage is normal, too (~10M).
> > > >
> > > > Another symptom: sometimes when the processes are nearing the top,
> > > > they'll start haging, but one by one; if I hit stop on a non-responsive
> > > > request, then re-try, it serves fine.
> > > >
> > > > I'm using the debian package of modperl 1.21 statically linked with
> > > > apache 1.3.9, and HTML::Embperl 1.2.1 ( I think).
> > > >
> > > >
> > > > Any insight greatly appreciated.
> > > >
> > > > Tom
> > > >
> > >
> > > try this:
> > >
> > > Date: Thu, 25 May 2000 12:13:20 -0700 (PDT)
> > > From: Doug MacEachern <do...@covalent.net>
> > > To: Roberto Bourgonjen <ro...@Toutatis.COM>
> > > cc: modperl@apache.org
> > > Subject: Re: httpd exits with apache-1.3.12 and mod_perl-1.21 on Redhat
> > > 6.2
> > >
> > > On Wed, 24 May 2000, Roberto Bourgonjen wrote:
> > >
> > > > Hi all,
> > > >
> > > > I am encountering strange problems on redhat 6.2 installations with
> > > > rpm's apache-1.3.12-2 and mod_perl-1.21-10 (the latest available
> > > > versions from redhat. I _have_ to use rpm's). I've tried two machines.
> > > >
> > > > As soon as load certain modules the server exits immediately, without
> > > > producing the slightest error message.
> > >
> > > try this:
> > >
> > > % gdb httpd
> > > (gdb) b exit
> > > (gdb) run -X
> > >
> > > whether it core dumps or exit() is called, post the output of:
> > >
> > > (gdb) bt
> > >
> > > _____________________________________________________________________
> > > Stas Bekman JAm_pH -- Just Another mod_perl Hacker
> > > http://stason.org/ mod_perl Guide http://perl.apache.org/guide
> > > mailto:stas@stason.org http://perl.org http://stason.org/TULARC
> > > http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
> >
>
> _____________________________________________________________________
> Stas Bekman JAm_pH -- Just Another mod_perl Hacker
> http://stason.org/ mod_perl Guide http://perl.apache.org/guide
> mailto:stas@stason.org http://perl.org http://stason.org/TULARC
> http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
Re: Apache hanging when it reaches limit of process limit
Posted by Stas Bekman <sb...@stason.org>.
On Tue, 6 Jun 2000, Tom Lancaster wrote:
> Here's the output of gdb: ( now what does this mean?)
You should compile mod_perl with PERL_DEBUG=1, see the SUPPORT file.
>
> Starting program: /usr/sbin/apache -X
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...Breakpoint 1 at 0x400ddd35
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...(no debugging symbols found)...
> (no debugging symbols found)...
> Breakpoint 1, 0x400ddd35 in exit () from /lib/libc.so.6
> (gdb) bt
> #0 0x400ddd35 in exit () from /lib/libc.so.6
> #1 0x80851e1 in ap_start_restart ()
> #2 0x8085454 in ap_start_restart ()
> #3 0x80862d7 in ap_child_terminate ()
> #4 0x8086a4d in main ()
> #5 0x400d5a42 in __libc_start_main () from /lib/libc.so.6
> (gdb)
>
>
> Stas Bekman wrote:
> >
> > On Tue, 6 Jun 2000, Tom Lancaster wrote:
> >
> > > Hi,
> > > I've got this strange behavior in a production site that until a recent
> > > upgrade never showed this kind of thing:
> > > I've set maxclients to 50, maxrequestsperchild 1.
> > > What happens is that the server runs fine, is fast, etc, but the number
> > > of apache processes climbs gradually until it reaches ~maxclients, then
> > > will no longer serve pages.
> > > I thought this was due to children not exiting, but some of them
> > > apparently do, as the number of processes rises and falls on its way to
> > > the top.
> > >
> > > What is Apache supposed to do when you set maxrequestsperchild to 1?
> > > I'd have thought each process would exit immediatley after serving
> > > something.
> > > I'm inclined to think there's something in my code that's hanging, but I
> > > read that the symptom of that is processes consuming large amounts of
> > > CPU - they're not, and memory usage is normal, too (~10M).
> > >
> > > Another symptom: sometimes when the processes are nearing the top,
> > > they'll start haging, but one by one; if I hit stop on a non-responsive
> > > request, then re-try, it serves fine.
> > >
> > > I'm using the debian package of modperl 1.21 statically linked with
> > > apache 1.3.9, and HTML::Embperl 1.2.1 ( I think).
> > >
> > >
> > > Any insight greatly appreciated.
> > >
> > > Tom
> > >
> >
> > try this:
> >
> > Date: Thu, 25 May 2000 12:13:20 -0700 (PDT)
> > From: Doug MacEachern <do...@covalent.net>
> > To: Roberto Bourgonjen <ro...@Toutatis.COM>
> > cc: modperl@apache.org
> > Subject: Re: httpd exits with apache-1.3.12 and mod_perl-1.21 on Redhat
> > 6.2
> >
> > On Wed, 24 May 2000, Roberto Bourgonjen wrote:
> >
> > > Hi all,
> > >
> > > I am encountering strange problems on redhat 6.2 installations with
> > > rpm's apache-1.3.12-2 and mod_perl-1.21-10 (the latest available
> > > versions from redhat. I _have_ to use rpm's). I've tried two machines.
> > >
> > > As soon as load certain modules the server exits immediately, without
> > > producing the slightest error message.
> >
> > try this:
> >
> > % gdb httpd
> > (gdb) b exit
> > (gdb) run -X
> >
> > whether it core dumps or exit() is called, post the output of:
> >
> > (gdb) bt
> >
> > _____________________________________________________________________
> > Stas Bekman JAm_pH -- Just Another mod_perl Hacker
> > http://stason.org/ mod_perl Guide http://perl.apache.org/guide
> > mailto:stas@stason.org http://perl.org http://stason.org/TULARC
> > http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
>
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://perl.org http://stason.org/TULARC
http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
Re: Apache hanging when it reaches limit of process limit
Posted by Tom Lancaster <to...@grubby.net>.
Here's the output of gdb: ( now what does this mean?)
Starting program: /usr/sbin/apache -X
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...Breakpoint 1 at 0x400ddd35
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...
Breakpoint 1, 0x400ddd35 in exit () from /lib/libc.so.6
(gdb) bt
#0 0x400ddd35 in exit () from /lib/libc.so.6
#1 0x80851e1 in ap_start_restart ()
#2 0x8085454 in ap_start_restart ()
#3 0x80862d7 in ap_child_terminate ()
#4 0x8086a4d in main ()
#5 0x400d5a42 in __libc_start_main () from /lib/libc.so.6
(gdb)
Stas Bekman wrote:
>
> On Tue, 6 Jun 2000, Tom Lancaster wrote:
>
> > Hi,
> > I've got this strange behavior in a production site that until a recent
> > upgrade never showed this kind of thing:
> > I've set maxclients to 50, maxrequestsperchild 1.
> > What happens is that the server runs fine, is fast, etc, but the number
> > of apache processes climbs gradually until it reaches ~maxclients, then
> > will no longer serve pages.
> > I thought this was due to children not exiting, but some of them
> > apparently do, as the number of processes rises and falls on its way to
> > the top.
> >
> > What is Apache supposed to do when you set maxrequestsperchild to 1?
> > I'd have thought each process would exit immediatley after serving
> > something.
> > I'm inclined to think there's something in my code that's hanging, but I
> > read that the symptom of that is processes consuming large amounts of
> > CPU - they're not, and memory usage is normal, too (~10M).
> >
> > Another symptom: sometimes when the processes are nearing the top,
> > they'll start haging, but one by one; if I hit stop on a non-responsive
> > request, then re-try, it serves fine.
> >
> > I'm using the debian package of modperl 1.21 statically linked with
> > apache 1.3.9, and HTML::Embperl 1.2.1 ( I think).
> >
> >
> > Any insight greatly appreciated.
> >
> > Tom
> >
>
> try this:
>
> Date: Thu, 25 May 2000 12:13:20 -0700 (PDT)
> From: Doug MacEachern <do...@covalent.net>
> To: Roberto Bourgonjen <ro...@Toutatis.COM>
> cc: modperl@apache.org
> Subject: Re: httpd exits with apache-1.3.12 and mod_perl-1.21 on Redhat
> 6.2
>
> On Wed, 24 May 2000, Roberto Bourgonjen wrote:
>
> > Hi all,
> >
> > I am encountering strange problems on redhat 6.2 installations with
> > rpm's apache-1.3.12-2 and mod_perl-1.21-10 (the latest available
> > versions from redhat. I _have_ to use rpm's). I've tried two machines.
> >
> > As soon as load certain modules the server exits immediately, without
> > producing the slightest error message.
>
> try this:
>
> % gdb httpd
> (gdb) b exit
> (gdb) run -X
>
> whether it core dumps or exit() is called, post the output of:
>
> (gdb) bt
>
> _____________________________________________________________________
> Stas Bekman JAm_pH -- Just Another mod_perl Hacker
> http://stason.org/ mod_perl Guide http://perl.apache.org/guide
> mailto:stas@stason.org http://perl.org http://stason.org/TULARC
> http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
Re: Apache hanging when it reaches limit of process limit
Posted by Stas Bekman <sb...@stason.org>.
On Tue, 6 Jun 2000, Tom Lancaster wrote:
> Hi,
> I've got this strange behavior in a production site that until a recent
> upgrade never showed this kind of thing:
> I've set maxclients to 50, maxrequestsperchild 1.
> What happens is that the server runs fine, is fast, etc, but the number
> of apache processes climbs gradually until it reaches ~maxclients, then
> will no longer serve pages.
> I thought this was due to children not exiting, but some of them
> apparently do, as the number of processes rises and falls on its way to
> the top.
>
> What is Apache supposed to do when you set maxrequestsperchild to 1?
> I'd have thought each process would exit immediatley after serving
> something.
> I'm inclined to think there's something in my code that's hanging, but I
> read that the symptom of that is processes consuming large amounts of
> CPU - they're not, and memory usage is normal, too (~10M).
>
> Another symptom: sometimes when the processes are nearing the top,
> they'll start haging, but one by one; if I hit stop on a non-responsive
> request, then re-try, it serves fine.
>
> I'm using the debian package of modperl 1.21 statically linked with
> apache 1.3.9, and HTML::Embperl 1.2.1 ( I think).
>
>
> Any insight greatly appreciated.
>
> Tom
>
try this:
Date: Thu, 25 May 2000 12:13:20 -0700 (PDT)
From: Doug MacEachern <do...@covalent.net>
To: Roberto Bourgonjen <ro...@Toutatis.COM>
cc: modperl@apache.org
Subject: Re: httpd exits with apache-1.3.12 and mod_perl-1.21 on Redhat
6.2
On Wed, 24 May 2000, Roberto Bourgonjen wrote:
> Hi all,
>
> I am encountering strange problems on redhat 6.2 installations with
> rpm's apache-1.3.12-2 and mod_perl-1.21-10 (the latest available
> versions from redhat. I _have_ to use rpm's). I've tried two machines.
>
> As soon as load certain modules the server exits immediately, without
> producing the slightest error message.
try this:
% gdb httpd
(gdb) b exit
(gdb) run -X
whether it core dumps or exit() is called, post the output of:
(gdb) bt
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://perl.org http://stason.org/TULARC
http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
Re: Apache hanging when it reaches limit of process limit
Posted by Doug MacEachern <do...@covalent.net>.
On Tue, 6 Jun 2000, Tom Lancaster wrote:
> Hi,
> I've got this strange behavior in a production site that until a recent
> upgrade never showed this kind of thing:
> I've set maxclients to 50, maxrequestsperchild 1.
> What happens is that the server runs fine, is fast, etc, but the number
> of apache processes climbs gradually until it reaches ~maxclients, then
> will no longer serve pages.
> I thought this was due to children not exiting, but some of them
> apparently do, as the number of processes rises and falls on its way to
> the top.
>
> What is Apache supposed to do when you set maxrequestsperchild to 1?
> I'd have thought each process would exit immediatley after serving
> something.
> I'm inclined to think there's something in my code that's hanging, but I
> read that the symptom of that is processes consuming large amounts of
> CPU - they're not, and memory usage is normal, too (~10M).
>
> Another symptom: sometimes when the processes are nearing the top,
> they'll start haging, but one by one; if I hit stop on a non-responsive
> request, then re-try, it serves fine.
>
> I'm using the debian package of modperl 1.21 statically linked with
> apache 1.3.9, and HTML::Embperl 1.2.1 ( I think).
i suggest upgrading to 1.24. i also suggest setting maxrequestsperchild
higher than 1. if you do have httpds hanging, this might shed some light:
% gdb httpd <pid of spinning process>
(gdb) where
(gdb) source mod_perl-x.xx/.gdbinit
(gdb) curinfo