You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Stephany Burge <st...@ofoto.com> on 2005/01/31 20:45:38 UTC
[users@httpd] graceful restarts and child procs that never die
Hello -
I am having a problem with Apache 2.0.52, graceful restarts, and
mod_proxy. While serving long lasting mod_proxy requests, I do a
graceful restart. Some httpd child processes will never die, nor
answer new incoming requests, and the # of threads will eventually
build up to the max and the whole thing will start hanging. This
occurs on Linux Fedora Core 1 and Solaris 8.
To reproduce the situation, I have set up http_load to hit the
webserver with numerous requests which will be handled by mod_proxy.
All of the requests will eventually time out. (ie. the destination
server does not exist). This is just a more extreme version of what
we see in production.
Here's the process:
1. Start Apache. 20 child threads.
2. Begin load test. MaxClients reached (250 httpd threads).
Requests begin timing out.
3. Do an "apachectl graceful" restart
4. Load test finishes (ran for 2 minutes) (250 threads when
concluded)
5. Wait 30+ minutes. Some threads die out, but 188 httpd threads
still live.
I've attached the server-status ouput in html format (for readability's
sake). The threads are stuck in the L state. After additional
"apachectl graceful" restarts, the threads will be stuck in G state,
indefinitely. I've let the server sit idle for a week and not seen
any more threads die. "apachectl stop" does stop all the threads.
Here's the error log output:
[Fri Jan 28 10:01:07 2005] [notice] Apache configured -- resuming
normal operations
[Fri Jan 28 10:06:29 2005] [error] server reached MaxClients setting,
consider raising the MaxClients setting
[Fri Jan 28 10:07:04 2005] [notice] Graceful restart requested, doing
restart
[Fri Jan 28 10:07:10 2005] [notice] Apache configured -- resuming
normal operations
[Fri Jan 28 10:07:10 2005] [warn] long lost child came home! (pid 1364)
Here's my relevant httpd configs:
<IfModule prefork.c>
StartServers 50
MinSpareServers 10
MaxSpareServers 20
MaxClients 250
MaxRequestsPerChild 100
</IfModule>
ProxyTimeout 300
ProxyPass /photos200 http://111.111.111.111/photos200
#ScoreBoardFile logs/httpd.scoreboard
Is this a known issue? Should I submit a bug report? Or is there a
configuration option I am missing?
Thank you,
Stephany Burge
Re: [users@httpd] graceful restarts and child procs that never die
Posted by Stephany Burge <st...@ofoto.com>.
I found the corresponding bug. It was an issue between cronolog and
apache.
Here are the relevant reports:
http://cronolog.org/mailing-list/msg00258.html
http://www.mail-archive.com/dev%40httpd.apache.org/msg19357.html
http://issues.apache.org/bugzilla/show_bug.cgi?id=26467
-Stephany
On Jan 31, 2005, at 11:45 AM, Stephany Burge wrote:
> Hello -
>
> I am having a problem with Apache 2.0.52, graceful restarts, and
> mod_proxy. While serving long lasting mod_proxy requests, I do a
> graceful restart. Some httpd child processes will never die, nor
> answer new incoming requests, and the # of threads will eventually
> build up to the max and the whole thing will start hanging. This
> occurs on Linux Fedora Core 1 and Solaris 8.
>
> To reproduce the situation, I have set up http_load to hit the
> webserver with numerous requests which will be handled by mod_proxy.
> All of the requests will eventually time out. (ie. the destination
> server does not exist). This is just a more extreme version of what
> we see in production.
>
> Here's the process:
> 1. Start Apache. 20 child threads.
> 2. Begin load test. MaxClients reached (250 httpd threads).
> Requests begin timing out.
> 3. Do an "apachectl graceful" restart
> 4. Load test finishes (ran for 2 minutes) (250 threads when
> concluded)
> 5. Wait 30+ minutes. Some threads die out, but 188 httpd threads
> still live.
>
> I've attached the server-status ouput in html format (for
> readability's sake). The threads are stuck in the L state. After
> additional "apachectl graceful" restarts, the threads will be stuck in
> G state, indefinitely. I've let the server sit idle for a week and
> not seen any more threads die. "apachectl stop" does stop all the
> threads.
>
> Here's the error log output:
> [Fri Jan 28 10:01:07 2005] [notice] Apache configured -- resuming
> normal operations
> [Fri Jan 28 10:06:29 2005] [error] server reached MaxClients setting,
> consider raising the MaxClients setting
> [Fri Jan 28 10:07:04 2005] [notice] Graceful restart requested, doing
> restart
> [Fri Jan 28 10:07:10 2005] [notice] Apache configured -- resuming
> normal operations
> [Fri Jan 28 10:07:10 2005] [warn] long lost child came home! (pid 1364)
>
> Here's my relevant httpd configs:
> <IfModule prefork.c>
> StartServers 50
> MinSpareServers 10
> MaxSpareServers 20
> MaxClients 250
> MaxRequestsPerChild 100
> </IfModule>
> ProxyTimeout 300
> ProxyPass /photos200 http://111.111.111.111/photos200
> #ScoreBoardFile logs/httpd.scoreboard
>
>
> Is this a known issue? Should I submit a bug report? Or is there
> a configuration option I am missing?
>
>
> Thank you,
>
> Stephany Burge
> <graceful-
> server.html>-----------------------------------------------------------
> ----------
> The official User-To-User support forum of the Apache HTTP Server
> Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org