You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by "Jeremy R." <je...@novawave.net> on 2010/01/23 01:59:46 UTC
[users@httpd] Apache hangs on reload
Hello:
I'm running PHP inside suexec with mod_fcgid and a couple wrapper scripts, and though it works properly, Apache now hangs whenever someone issues /etc/init.d/apache2 reload (which happens fairly frequently). Killing off all FCGI processes and the www-data Apache processes seems to let it continue working, but this shouldn't be happening.
Here's the setup for one of my vhosts (one other is set up similarly, and suexec is otherwise not used at all - mod_suexec is not enabled):
dr-xr-xr-x 2 www-webmail www-webmail 4096 2010-01-20 21:57 /var/lib/roundcube/fcgi-wrapper
-r-xr-xr-x 1 www-webmail www-webmail 95 2010-01-20 21:57 php-cgi
#!/bin/sh
export PHP_FCGI_CHILDREN=0
export PHP_FCGI_MAX_REQUESTS=4000
exec /usr/bin/php5-cgi
-rwxr-xr-x 1 root root 71 2010-01-20 21:34 php-cgi-suexec
#!/bin/sh
exec /usr/lib/apache2/suexec www-webmail www-webmail php-cgi
And here is the relevant line in the vhost configuration:
FCGIWrapper "/var/lib/roundcube/fcgi-wrapper/php-cgi-suexec" .php
Any thoughts or tips?
--
Jeremy R.
Owner, NovaWave Solutions
http://www.novawave.net/
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache hangs on reload
Posted by "Jeremy R." <je...@novawave.net>.
On 2010-01-23, at 4:42 PM, Jeff Trawick wrote:
> mod_fcgid tries to exit the httpd child process right after issuing
> that message. I think it encounters the error because it is running
> in a child process from the old generation (before restart) and the
> lock was removed in the parent process.
>
> Did you have to forcefully kill those same processes that said "can't
> get lock"? Next time can you attach with gdb and get a backtrace?
>
> Do something like this:
>
> gdb /path/to/httpd 22501
> ...
> (gdb) where
> ...
> (gdb) quit
Doesn't seem to work (even when run as root):
# gdb /usr/sbin/apache2 14604
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu"...
(no debugging symbols found)
Attaching to program: /usr/sbin/apache2, process 14604
ptrace: Operation not permitted.
/root/14604: No such file or directory.
Also, for some reason the message shows up in the main Apache log, and sometimes in the vhost's error log. If that helps.
--
Jeremy R.
Owner, NovaWave Solutions
http://www.novawave.net/
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache hangs on reload
Posted by Jeff Trawick <tr...@gmail.com>.
On Sat, Jan 23, 2010 at 3:08 PM, Jeremy R. <je...@novawave.net> wrote:
> Okay, so here are the salient lines from the error logs (I have two vhosts, webmail and wiki, running through suexec – if I put them through the regular FCGI process without suexec, I don't have this problem). Also, I should not that this problem does not occur on my testing VM, only on our production server (so I'm not quite sure what the problem is).
>
> webmail-error.log:
>
> [Wed Jan 20 23:38:17 2010] [emerg] mod_fcgid: server is restarted, 32760 must exit
>
> wiki-error.log
>
> [Sat Jan 23 15:03:25 2010] [emerg] mod_fcgid: server is restarted, 22502 must exit
> [Sat Jan 23 15:03:25 2010] [emerg] (22)Invalid argument: mod_fcgid: can't get lock, pid: 22502
> [Sat Jan 23 15:03:25 2010] [emerg] mod_fcgid: server is restarted, 16264 must exit
> [Sat Jan 23 15:03:25 2010] [emerg] (22)Invalid argument: mod_fcgid: can't get lock, pid: 16264
> [Sat Jan 23 15:03:25 2010] [emerg] mod_fcgid: server is restarted, 22501 must exit
> [Sat Jan 23 15:03:25 2010] [emerg] (22)Invalid argument: mod_fcgid: can't get lock, pid: 22501
>
> It looks like the emergency messages about being unable to get a lock on these processes is what's causing Apache to hang until I forcefully kill them.
mod_fcgid tries to exit the httpd child process right after issuing
that message. I think it encounters the error because it is running
in a child process from the old generation (before restart) and the
lock was removed in the parent process.
Did you have to forcefully kill those same processes that said "can't
get lock"? Next time can you attach with gdb and get a backtrace?
Do something like this:
gdb /path/to/httpd 22501
...
(gdb) where
...
(gdb) quit
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache hangs on reload
Posted by "Jeremy R." <je...@novawave.net>.
Okay, so here are the salient lines from the error logs (I have two vhosts, webmail and wiki, running through suexec – if I put them through the regular FCGI process without suexec, I don't have this problem). Also, I should not that this problem does not occur on my testing VM, only on our production server (so I'm not quite sure what the problem is).
webmail-error.log:
[Wed Jan 20 23:38:17 2010] [emerg] mod_fcgid: server is restarted, 32760 must exit
wiki-error.log
[Sat Jan 23 15:03:25 2010] [emerg] mod_fcgid: server is restarted, 22502 must exit
[Sat Jan 23 15:03:25 2010] [emerg] (22)Invalid argument: mod_fcgid: can't get lock, pid: 22502
[Sat Jan 23 15:03:25 2010] [emerg] mod_fcgid: server is restarted, 16264 must exit
[Sat Jan 23 15:03:25 2010] [emerg] (22)Invalid argument: mod_fcgid: can't get lock, pid: 16264
[Sat Jan 23 15:03:25 2010] [emerg] mod_fcgid: server is restarted, 22501 must exit
[Sat Jan 23 15:03:25 2010] [emerg] (22)Invalid argument: mod_fcgid: can't get lock, pid: 22501
error.log
[Sat Jan 23 15:03:27 2010] [notice] mod_fcgid: process /var/lib/mediawiki/index.php(22557) exit(shutting down), terminated by calling exit(), return code: 0
[Sat Jan 23 15:03:27 2010] [notice] mod_fcgid: process /var/lib/mediawiki/index.php(22561) exit(shutting down), terminated by calling exit(), return code: 0
[Sat Jan 23 15:03:27 2010] [notice] mod_fcgid: process /var/lib/mediawiki/index.php(22562) exit(shutting down), terminated by calling exit(), return code: 0
[Sat Jan 23 15:03:27 2010] [notice] mod_fcgid: process /var/lib/roundcube/index.php(2380) exit(shutting down), terminated by calling exit(), return code: 0
[Sat Jan 23 15:03:27 2010] [notice] mod_fcgid: process /var/lib/roundcube/index.php(5653) exit(shutting down), terminated by calling exit(), return code: 0
[Sat Jan 23 15:03:27 2010] [notice] mod_fcgid: process /var/lib/roundcube/index.php(6212) exit(shutting down), terminated by calling exit(), return code: 0
[Sat Jan 23 15:03:27 2010] [notice] mod_webauth: initialized (3.6.0) (Built by buildd@excelsior on 2008-05-02 22:03:49 UTC)
[Sat Jan 23 15:03:27 2010] [notice] Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g WebAuth/3.6.0 configured -- resuming normal operations
[Sat Jan 23 15:03:27 2010] [notice] mod_fcgid: call /var/lib/mediawiki/index.php with wrapper /var/lib/mediawiki/fcgi-wrapper/php-cgi-suexec
[Sat Jan 23 15:03:28 2010] [notice] mod_fcgid: call /var/lib/mediawiki/index.php with wrapper /var/lib/mediawiki/fcgi-wrapper/php-cgi-suexec
[Sat Jan 23 15:03:28 2010] [notice] mod_fcgid: call /var/lib/mediawiki/index.php with wrapper /var/lib/mediawiki/fcgi-wrapper/php-cgi-suexec
It looks like the emergency messages about being unable to get a lock on these processes is what's causing Apache to hang until I forcefully kill them.
On 2010-01-22, at 9:24 PM, Daniel Reinhardt wrote:
>
> --------------------------------------------------
> From: "Jeremy R." <je...@novawave.net>
> Sent: 23 January, 2010 0:59
> To: <us...@httpd.apache.org>
> Subject: [users@httpd] Apache hangs on reload
>
>> Hello:
>>
>> I'm running PHP inside suexec with mod_fcgid and a couple wrapper scripts, and though it works properly, Apache now hangs whenever someone issues /etc/init.d/apache2 reload (which happens fairly frequently). Killing off all FCGI processes and the www-data Apache processes seems to let it continue working, but this shouldn't be happening.
>>
>> Here's the setup for one of my vhosts (one other is set up similarly, and suexec is otherwise not used at all - mod_suexec is not enabled):
>>
>> dr-xr-xr-x 2 www-webmail www-webmail 4096 2010-01-20 21:57 /var/lib/roundcube/fcgi-wrapper
>>
>> -r-xr-xr-x 1 www-webmail www-webmail 95 2010-01-20 21:57 php-cgi
>> #!/bin/sh
>> export PHP_FCGI_CHILDREN=0
>> export PHP_FCGI_MAX_REQUESTS=4000
>> exec /usr/bin/php5-cgi
>>
>> -rwxr-xr-x 1 root root 71 2010-01-20 21:34 php-cgi-suexec
>> #!/bin/sh
>> exec /usr/lib/apache2/suexec www-webmail www-webmail php-cgi
>>
>> And here is the relevant line in the vhost configuration:
>>
>> FCGIWrapper "/var/lib/roundcube/fcgi-wrapper/php-cgi-suexec" .php
>>
>> Any thoughts or tips?
>>
>> --
>> Jeremy R.
>> Owner, NovaWave Solutions
>> http://www.novawave.net/
>>
>>
>> ---------------------------------------------------------------------
>> The official User-To-User support forum of the Apache HTTP Server Project.
>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> " from the digest: users-digest-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
>
> Jeremy,
>
> Check your error logs.
>
> Thanks,
> Daniel
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
--
Jeremy R.
Owner, NovaWave Solutions
http://www.novawave.net/
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache hangs on reload
Posted by Daniel Reinhardt <cr...@cryptodan.net>.
--------------------------------------------------
From: "Jeremy R." <je...@novawave.net>
Sent: 23 January, 2010 0:59
To: <us...@httpd.apache.org>
Subject: [users@httpd] Apache hangs on reload
> Hello:
>
> I'm running PHP inside suexec with mod_fcgid and a couple wrapper scripts, and
> though it works properly, Apache now hangs whenever someone issues
> /etc/init.d/apache2 reload (which happens fairly frequently). Killing off all
> FCGI processes and the www-data Apache processes seems to let it continue
> working, but this shouldn't be happening.
>
> Here's the setup for one of my vhosts (one other is set up similarly, and
> suexec is otherwise not used at all - mod_suexec is not enabled):
>
> dr-xr-xr-x 2 www-webmail www-webmail 4096 2010-01-20 21:57
> /var/lib/roundcube/fcgi-wrapper
>
> -r-xr-xr-x 1 www-webmail www-webmail 95 2010-01-20 21:57 php-cgi
> #!/bin/sh
> export PHP_FCGI_CHILDREN=0
> export PHP_FCGI_MAX_REQUESTS=4000
> exec /usr/bin/php5-cgi
>
> -rwxr-xr-x 1 root root 71 2010-01-20 21:34 php-cgi-suexec
> #!/bin/sh
> exec /usr/lib/apache2/suexec www-webmail www-webmail php-cgi
>
> And here is the relevant line in the vhost configuration:
>
> FCGIWrapper "/var/lib/roundcube/fcgi-wrapper/php-cgi-suexec" .php
>
> Any thoughts or tips?
>
> --
> Jeremy R.
> Owner, NovaWave Solutions
> http://www.novawave.net/
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
Jeremy,
Check your error logs.
Thanks,
Daniel
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org