You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by roberto blanko <ro...@gmail.com> on 2011/03/01 16:10:40 UTC
[users@httpd] FastCGI causes website to hang
Hello fellow Apache community,
we just switched from our old hosting platform (which used to run PHP as an
Apache module) to Plesk which (in our configuration) runs PHP through
FastCGI. In general we prefer the principle and security features of
FastCGI, so we want to stick to it.
However, since we started migrating some bigger websites (bigger means more
hits a day) we encounter regular stability problems. After some time (can be
some minutes to some hours) some websites just don't react anymore. They
keep loading and loading until Apache quits with a timeout.
The only way to solve this issue, is to restart apache. When the server
hangs `ps aux | grep php` shows the following output:
user2 9708 0.0 1.1 263772 37544 ? S Feb21 0:04
/usr/bin/php-cgi -c /var/www/vhosts/user2.domain.tld/conf/php.ini
user2 9836 0.0 1.2 264028 37812 ? S Feb21 0:03
/usr/bin/php-cgi -c /var/www/vhosts/user2.domain.tld/conf/php.ini
user1 15775 0.1 1.4 266704 45476 ? S 07:36 0:24
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user1 15780 0.0 1.4 266704 45408 ? S 07:36 0:14
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user1 15790 0.1 1.4 266588 45244 ? S 07:36 0:25
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user3 23580 0.0 1.0 261880 34020 ? S 08:17 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user3.domain.tld/conf/php.ini
user1 23806 0.2 1.1 264128 37644 ? S 11:40 0:01
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user3 24230 0.0 0.8 259016 26112 ? S 04:53 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user3.domain.tld/conf/php.ini
user1 27796 0.4 1.0 259772 33556 ? S 11:50 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user1 27831 0.2 1.1 264124 37616 ? S 11:51 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user1 27857 0.2 1.0 259772 33292 ? S 11:52 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user1 27963 0.2 1.1 262480 36284 ? S 11:52 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user1 28001 0.2 0.9 261580 28856 ? S 11:52 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user1 28012 0.1 0.9 261596 28860 ? S 11:52 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user1 28037 0.3 0.8 260548 27712 ? S 11:53 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user1 28042 1.3 1.1 263600 36044 ? S 11:53 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
user1 28058 0.7 1.1 263852 36124 ? S 11:53 0:00
/usr/bin/php-cgi -c /var/www/vhosts/user1.domain.tld/conf/php.ini
If I keep waiting for some time the PHP processes change slightly and are
being replaced over time by processes having 0:00 all the time.
We already tried to increase some numbers in the FastCGI-settings, but it
didn't help. No matter how high we set FcgidMaxProcesses and
FcgidMaxProcessesPerClass: At the time the server crashes, we only have
about 10 to 20 running processes - never more.
This is how our current config looks like:
<IfModule mod_fcgid.c>
FcgidIPCDir /var/run/mod_fcgid/sock
FcgidProcessTableFile /var/run/mod_fcgid/fcgid_shm
FcgidIdleTimeout 40
FcgidProcessLifeTime 30
FcgidMaxProcesses 1000
FcgidMaxProcessesPerClass 100
FcgidMinProcessesPerClass 25
FcgidConnectTimeout 30
FcgidIOTimeout 130
# PHP's max_execution time is 120, so give FCGI 10 secs extra
FcgidInitialEnv RAILS_ENV production
FcgidIdleScanInterval 10
</IfModule>
The error_log doesn't show anything suspicious besides some lines from time
to time:
mod_fcgid: process 23786 graceful kill fail, sending SIGKILL
Do you have any ideas what the problem could be? Any help would be very
appreciated. If you need more specific information, just ask for it and I'll
take a look at the server. But since I have not even the slightest idea,
what the cause could be, I'm not sure what to post here.
Regards
roberto
Re: [users@httpd] FastCGI causes website to hang
Posted by Björn Zettergren <bj...@basefarm.se>.
On 03/02/2011 09:17 AM, roberto blanko wrote:
> I'm not the real freak when it comes to semaphores and shared memory
> usage. Can you see something suspicious there?
Hm, sorry to say i can't really.
> Another notice: After reading your reply I deactivated eAccellerator,
> ionCube-Loader and Zend-Optimizer, so that PHP runs without any
> additional caches or optimzers, Unfortunately this didn't help and
> apache stopped working again.
Good start, try stripping down as much as possible, and try to find a
scenario where you can trigger the fault manually. That will surely help
you to find the problem. If you feel confident that you'll be able to
trigger the error quickly, you could do a "strace" on your
FCGI-processes, and see what it is they're actually doing when they're
not working (or rather, what they did just before :-) )
/Björn
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] FastCGI causes website to hang
Posted by roberto blanko <ro...@gmail.com>.
>
> Try restarting your apache, then list your semaphores with the command
> "ipcs". Wait until apache stops working in the described manner, then list
> them again and check if there are any semaphores missing.
>
Hi Björn,
thanks for your support. I monitored the semaphores and shared memory the
way you described. With the following results:
Right after Apache restart:
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x0115859c 26116096 root 600 682600 5
------ Semaphore Arrays --------
key semid owner perms nsems
0x00000000 2359296 apache 600 1
0x00000000 2392065 apache 600 1
0x00000000 2424834 apache 600 1
0x00000000 2457603 apache 600 1
0x00000000 2490372 apache 600 1
0x00000000 2523141 apache 600 1
0x00000000 2555910 apache 600 1
0x00000000 2588679 apache 600 1
0x00000000 2621448 apache 600 1
After apache stopped working:
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x0115859c 0 root 600 682600 12
------ Semaphore Arrays --------
key semid owner perms nsems
0x00000000 0 apache 600 1
0x00000000 32769 apache 600 1
0x00000000 65538 apache 600 1
0x00000000 98307 apache 600 1
0x00000000 131076 apache 600 1
0x00000000 163845 apache 600 1
0x00000000 196614 apache 600 1
0x00000000 229383 apache 600 1
0x00000000 262152 apache 600 1
I'm not the real freak when it comes to semaphores and shared memory usage.
Can you see something suspicious there?
Another notice: After reading your reply I deactivated eAccellerator,
ionCube-Loader and Zend-Optimizer, so that PHP runs without any additional
caches or optimzers, Unfortunately this didn't help and apache stopped
working again.
Re: [users@httpd] FastCGI causes website to hang
Posted by Björn Zettergren <bj...@basefarm.se>.
Hi,
Your configuration does not really match mine, but i've had a similar
issue with sort of the same behaviour.
I was using mod_fastcgi (not mod_fcgid), and PHP used the APC (opcode
cache for php). The source of my problem was that a PHP-CGI-child would
also remove the semaphore that its parent created when the childs
maxrequests-setting was reached.
Try restarting your apache, then list your semaphores with the command
"ipcs". Wait until apache stops working in the described manner, then
list them again and check if there are any semaphores missing.
But then again, this might not be relevant at all.
/Björn
On 03/01/2011 04:10 PM, roberto blanko wrote:
> Hello fellow Apache community,
>
> we just switched from our old hosting platform (which used to run PHP as
> an Apache module) to Plesk which (in our configuration) runs PHP through
> FastCGI. In general we prefer the principle and security features of
> FastCGI, so we want to stick to it.
>
> However, since we started migrating some bigger websites (bigger means
> more hits a day) we encounter regular stability problems. After some
> time (can be some minutes to some hours) some websites just don't react
> anymore. They keep loading and loading until Apache quits with a timeout.
>
> The only way to solve this issue, is to restart apache. When the server
> hangs `ps aux | grep php` shows the following output:
>
> user2 9708 0.0 1.1 263772 37544 ? S Feb21 0:04 /usr/bin/php-cgi -c
> /var/www/vhosts/user2.domain.tld/conf/php.ini
> user2 9836 0.0 1.2 264028 37812 ? S Feb21 0:03 /usr/bin/php-cgi -c
> /var/www/vhosts/user2.domain.tld/conf/php.ini
> user1 15775 0.1 1.4 266704 45476 ? S 07:36 0:24 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user1 15780 0.0 1.4 266704 45408 ? S 07:36 0:14 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user1 15790 0.1 1.4 266588 45244 ? S 07:36 0:25 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user3 23580 0.0 1.0 261880 34020 ? S 08:17 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user3.domain.tld/conf/php.ini
> user1 23806 0.2 1.1 264128 37644 ? S 11:40 0:01 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user3 24230 0.0 0.8 259016 26112 ? S 04:53 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user3.domain.tld/conf/php.ini
> user1 27796 0.4 1.0 259772 33556 ? S 11:50 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user1 27831 0.2 1.1 264124 37616 ? S 11:51 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user1 27857 0.2 1.0 259772 33292 ? S 11:52 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user1 27963 0.2 1.1 262480 36284 ? S 11:52 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user1 28001 0.2 0.9 261580 28856 ? S 11:52 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user1 28012 0.1 0.9 261596 28860 ? S 11:52 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user1 28037 0.3 0.8 260548 27712 ? S 11:53 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user1 28042 1.3 1.1 263600 36044 ? S 11:53 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
> user1 28058 0.7 1.1 263852 36124 ? S 11:53 0:00 /usr/bin/php-cgi -c
> /var/www/vhosts/user1.domain.tld/conf/php.ini
>
> If I keep waiting for some time the PHP processes change slightly and
> are being replaced over time by processes having 0:00 all the time.
>
> We already tried to increase some numbers in the FastCGI-settings, but
> it didn't help. No matter how high we set FcgidMaxProcesses and
> FcgidMaxProcessesPerClass: At the time the server crashes, we only have
> about 10 to 20 running processes - never more.
>
> This is how our current config looks like:
>
> <IfModule mod_fcgid.c>
> FcgidIPCDir /var/run/mod_fcgid/sock
> FcgidProcessTableFile /var/run/mod_fcgid/fcgid_shm
>
> FcgidIdleTimeout 40
> FcgidProcessLifeTime 30
> FcgidMaxProcesses 1000
> FcgidMaxProcessesPerClass 100
> FcgidMinProcessesPerClass 25
> FcgidConnectTimeout 30
> FcgidIOTimeout 130
> # PHP's max_execution time is 120, so give FCGI 10 secs extra
> FcgidInitialEnv RAILS_ENV production
> FcgidIdleScanInterval 10
> </IfModule>
>
> The error_log doesn't show anything suspicious besides some lines from
> time to time:
> mod_fcgid: process 23786 graceful kill fail, sending SIGKILL
>
> Do you have any ideas what the problem could be? Any help would be very
> appreciated. If you need more specific information, just ask for it and
> I'll take a look at the server. But since I have not even the slightest
> idea, what the cause could be, I'm not sure what to post here.
>
> Regards
> roberto
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org