You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Simone Frattegiani <si...@gmail.com> on 2012/03/09 09:07:34 UTC
[users@httpd] suddendly high cpu load because of googlebot
Hello,
i suddendly started having CPU load issues, like this:
top - 03:52:04 up 3 days, 8:52, 0 users, load average: 74.35, 73.66, 71.41
Tasks: 304 total, 75 running, 229 sleeping, 0 stopped, 0 zombie
Cpu(s): 72.1%us, 2.0%sy, 0.1%ni, 23.9%id, 1.2%wa, 0.4%hi, 0.4%si, 0.0%st
Mem: 3115464k total, 2840484k used, 274980k free, 197472k buffers
Swap: 5177336k total, 76k used, 5177260k free, 1702148k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19392 apache 25 0 34096 15m 3548 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
start -DSSL
19480 apache 24 0 34172 15m 3788 R 19.6 0.5 0:04.79 /usr/sbin/httpd -k
start -DSSL
19481 apache 25 0 34048 14m 3436 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
start -DSSL
19482 apache 25 0 34180 14m 3500 R 19.6 0.5 0:06.63 /usr/sbin/httpd -k
start -DSSL
19881 apache 25 0 34048 14m 3436 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
start -DSSL
20262 root 20 0 2424 1032 696 R 2.0 0.0 0:00.01 /usr/bin/top -c -b -n 1
1 root 15 0 2160 596 516 S 0.0 0.0 0:00.62 init [3]
2 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [migration/0]
3 root 34 19 0 0 0 R 0.0 0.0 0:00.00 [ksoftirqd/0]
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
[...]
I can also see from my panel that apache spawned like 50 children,
which is very unusual.
Server-status shows this:
Total Accesses: 292068
Total kBytes: 5540956
CPULoad: .0884054
Uptime: 45823
ReqPerSec: 6.37383
BytesPerSec: 123823
BytesPerReq: 19426.8
BusyWorkers: 99
IdleWorkers: 10
Scoreboard: WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWWWWWWWWWWWWWWWWWWWWWWKWWWWWCWWWWWWWWWKWWWWWWWWWWKWWW_W_CWWW__WS__K_WWK__K_S........
........................................................
...................................................
With hundreds of lines like this (66.249.66.44 is googlebot):
24-0 - 0/0/8 . 0.02 661 34 0.0 0.00 0.00 66.249.66.44
www.mydomain.com GET /mypage
So, please correct me if i'm wrong, that 661 in the SS column means
that 661 seconds have passed since the last time Googlebot used that
slot to make a request... so basically the server is keeping alive a
connection that has been idle for like 11 minutes!!
Did i understand correctly? Is it normal? Is it what is causing the problem?
The commands "netstat -a | grep 66.249" shows 20 connections from
googlebot ip, 1 in ESTABILISHEd status, the others in TIME WAIT.
If i restart apache, everything gets back to normal.
Any suggestions?
THanks!
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] suddendly high cpu load because of googlebot
Posted by Igor Cicimov <ic...@gmail.com>.
You didnt tell us what OS is the apache running on? If Unix/Linux you might
check the TCP kernel parameters like tcp_keepalive and the socket timeouts.
On Mar 9, 2012 9:34 PM, "Simone Frattegiani" <si...@gmail.com>
wrote:
> Hello, thanks for your reply.
>
> Here are my settings (never changed them):
>
> Timeout 60
> KeepAlive On
> MaxKeepAliveRequests 100
> KeepAliveTimeout 1
>
>
> On Fri, Mar 9, 2012 at 11:11 AM, Igor Cicimov <ic...@gmail.com> wrote:
> > What are your TimeOut and KeepAliveTimeout values?
> >
> > On Mar 9, 2012 7:08 PM, "Simone Frattegiani" <
> simone.frattegiani@gmail.com>
> > wrote:
> >>
> >> Hello,
> >>
> >> i suddendly started having CPU load issues, like this:
> >>
> >> top - 03:52:04 up 3 days, 8:52, 0 users, load average: 74.35, 73.66,
> 71.41
> >> Tasks: 304 total, 75 running, 229 sleeping, 0 stopped, 0 zombie
> >> Cpu(s): 72.1%us, 2.0%sy, 0.1%ni, 23.9%id, 1.2%wa, 0.4%hi, 0.4%si, 0.0%st
> >> Mem: 3115464k total, 2840484k used, 274980k free, 197472k buffers
> >> Swap: 5177336k total, 76k used, 5177260k free, 1702148k cached
> >>
> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> >> 19392 apache 25 0 34096 15m 3548 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
> >> start -DSSL
> >> 19480 apache 24 0 34172 15m 3788 R 19.6 0.5 0:04.79 /usr/sbin/httpd -k
> >> start -DSSL
> >> 19481 apache 25 0 34048 14m 3436 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
> >> start -DSSL
> >> 19482 apache 25 0 34180 14m 3500 R 19.6 0.5 0:06.63 /usr/sbin/httpd -k
> >> start -DSSL
> >> 19881 apache 25 0 34048 14m 3436 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
> >> start -DSSL
> >> 20262 root 20 0 2424 1032 696 R 2.0 0.0 0:00.01 /usr/bin/top -c -b -n 1
> >> 1 root 15 0 2160 596 516 S 0.0 0.0 0:00.62 init [3]
> >> 2 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [migration/0]
> >> 3 root 34 19 0 0 0 R 0.0 0.0 0:00.00 [ksoftirqd/0]
> >> 4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
> >>
> >> [...]
> >>
> >>
> >> I can also see from my panel that apache spawned like 50 children,
> >> which is very unusual.
> >>
> >>
> >> Server-status shows this:
> >>
> >> Total Accesses: 292068
> >> Total kBytes: 5540956
> >> CPULoad: .0884054
> >> Uptime: 45823
> >> ReqPerSec: 6.37383
> >> BytesPerSec: 123823
> >> BytesPerReq: 19426.8
> >> BusyWorkers: 99
> >> IdleWorkers: 10
> >> Scoreboard: WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
> >>
> >>
> WWWWWWWWWWWWWWWWWWWWWWWWWWKWWWWWCWWWWWWWWWKWWWWWWWWWWKWWW_W_CWWW__WS__K_WWK__K_S........
> >> ........................................................
> >> ...................................................
> >>
> >> With hundreds of lines like this (66.249.66.44 is googlebot):
> >>
> >> 24-0 - 0/0/8 . 0.02 661 34 0.0 0.00 0.00 66.249.66.44
> >> www.mydomain.com GET /mypage
> >>
> >> So, please correct me if i'm wrong, that 661 in the SS column means
> >> that 661 seconds have passed since the last time Googlebot used that
> >> slot to make a request... so basically the server is keeping alive a
> >> connection that has been idle for like 11 minutes!!
> >> Did i understand correctly? Is it normal? Is it what is causing the
> >> problem?
> >>
> >> The commands "netstat -a | grep 66.249" shows 20 connections from
> >> googlebot ip, 1 in ESTABILISHEd status, the others in TIME WAIT.
> >>
> >> If i restart apache, everything gets back to normal.
> >>
> >> Any suggestions?
> >> THanks!
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> >> For additional commands, e-mail: users-help@httpd.apache.org
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
Re: [users@httpd] suddendly high cpu load because of googlebot
Posted by Simone Frattegiani <si...@gmail.com>.
Hello, thanks for your reply.
Here are my settings (never changed them):
Timeout 60
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 1
On Fri, Mar 9, 2012 at 11:11 AM, Igor Cicimov <ic...@gmail.com> wrote:
> What are your TimeOut and KeepAliveTimeout values?
>
> On Mar 9, 2012 7:08 PM, "Simone Frattegiani" <si...@gmail.com>
> wrote:
>>
>> Hello,
>>
>> i suddendly started having CPU load issues, like this:
>>
>> top - 03:52:04 up 3 days, 8:52, 0 users, load average: 74.35, 73.66, 71.41
>> Tasks: 304 total, 75 running, 229 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 72.1%us, 2.0%sy, 0.1%ni, 23.9%id, 1.2%wa, 0.4%hi, 0.4%si, 0.0%st
>> Mem: 3115464k total, 2840484k used, 274980k free, 197472k buffers
>> Swap: 5177336k total, 76k used, 5177260k free, 1702148k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 19392 apache 25 0 34096 15m 3548 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
>> start -DSSL
>> 19480 apache 24 0 34172 15m 3788 R 19.6 0.5 0:04.79 /usr/sbin/httpd -k
>> start -DSSL
>> 19481 apache 25 0 34048 14m 3436 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
>> start -DSSL
>> 19482 apache 25 0 34180 14m 3500 R 19.6 0.5 0:06.63 /usr/sbin/httpd -k
>> start -DSSL
>> 19881 apache 25 0 34048 14m 3436 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
>> start -DSSL
>> 20262 root 20 0 2424 1032 696 R 2.0 0.0 0:00.01 /usr/bin/top -c -b -n 1
>> 1 root 15 0 2160 596 516 S 0.0 0.0 0:00.62 init [3]
>> 2 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [migration/0]
>> 3 root 34 19 0 0 0 R 0.0 0.0 0:00.00 [ksoftirqd/0]
>> 4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
>>
>> [...]
>>
>>
>> I can also see from my panel that apache spawned like 50 children,
>> which is very unusual.
>>
>>
>> Server-status shows this:
>>
>> Total Accesses: 292068
>> Total kBytes: 5540956
>> CPULoad: .0884054
>> Uptime: 45823
>> ReqPerSec: 6.37383
>> BytesPerSec: 123823
>> BytesPerReq: 19426.8
>> BusyWorkers: 99
>> IdleWorkers: 10
>> Scoreboard: WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
>>
>> WWWWWWWWWWWWWWWWWWWWWWWWWWKWWWWWCWWWWWWWWWKWWWWWWWWWWKWWW_W_CWWW__WS__K_WWK__K_S........
>> ........................................................
>> ...................................................
>>
>> With hundreds of lines like this (66.249.66.44 is googlebot):
>>
>> 24-0 - 0/0/8 . 0.02 661 34 0.0 0.00 0.00 66.249.66.44
>> www.mydomain.com GET /mypage
>>
>> So, please correct me if i'm wrong, that 661 in the SS column means
>> that 661 seconds have passed since the last time Googlebot used that
>> slot to make a request... so basically the server is keeping alive a
>> connection that has been idle for like 11 minutes!!
>> Did i understand correctly? Is it normal? Is it what is causing the
>> problem?
>>
>> The commands "netstat -a | grep 66.249" shows 20 connections from
>> googlebot ip, 1 in ESTABILISHEd status, the others in TIME WAIT.
>>
>> If i restart apache, everything gets back to normal.
>>
>> Any suggestions?
>> THanks!
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] suddendly high cpu load because of googlebot
Posted by Igor Cicimov <ic...@gmail.com>.
What are your TimeOut and KeepAliveTimeout values?
On Mar 9, 2012 7:08 PM, "Simone Frattegiani" <si...@gmail.com>
wrote:
> Hello,
>
> i suddendly started having CPU load issues, like this:
>
> top - 03:52:04 up 3 days, 8:52, 0 users, load average: 74.35, 73.66, 71.41
> Tasks: 304 total, 75 running, 229 sleeping, 0 stopped, 0 zombie
> Cpu(s): 72.1%us, 2.0%sy, 0.1%ni, 23.9%id, 1.2%wa, 0.4%hi, 0.4%si, 0.0%st
> Mem: 3115464k total, 2840484k used, 274980k free, 197472k buffers
> Swap: 5177336k total, 76k used, 5177260k free, 1702148k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 19392 apache 25 0 34096 15m 3548 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
> start -DSSL
> 19480 apache 24 0 34172 15m 3788 R 19.6 0.5 0:04.79 /usr/sbin/httpd -k
> start -DSSL
> 19481 apache 25 0 34048 14m 3436 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
> start -DSSL
> 19482 apache 25 0 34180 14m 3500 R 19.6 0.5 0:06.63 /usr/sbin/httpd -k
> start -DSSL
> 19881 apache 25 0 34048 14m 3436 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
> start -DSSL
> 20262 root 20 0 2424 1032 696 R 2.0 0.0 0:00.01 /usr/bin/top -c -b -n 1
> 1 root 15 0 2160 596 516 S 0.0 0.0 0:00.62 init [3]
> 2 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [migration/0]
> 3 root 34 19 0 0 0 R 0.0 0.0 0:00.00 [ksoftirqd/0]
> 4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
>
> [...]
>
>
> I can also see from my panel that apache spawned like 50 children,
> which is very unusual.
>
>
> Server-status shows this:
>
> Total Accesses: 292068
> Total kBytes: 5540956
> CPULoad: .0884054
> Uptime: 45823
> ReqPerSec: 6.37383
> BytesPerSec: 123823
> BytesPerReq: 19426.8
> BusyWorkers: 99
> IdleWorkers: 10
> Scoreboard: WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
>
> WWWWWWWWWWWWWWWWWWWWWWWWWWKWWWWWCWWWWWWWWWKWWWWWWWWWWKWWW_W_CWWW__WS__K_WWK__K_S........
> ........................................................
> ...................................................
>
> With hundreds of lines like this (66.249.66.44 is googlebot):
>
> 24-0 - 0/0/8 . 0.02 661 34 0.0 0.00 0.00 66.249.66.44
> www.mydomain.com GET /mypage
>
> So, please correct me if i'm wrong, that 661 in the SS column means
> that 661 seconds have passed since the last time Googlebot used that
> slot to make a request... so basically the server is keeping alive a
> connection that has been idle for like 11 minutes!!
> Did i understand correctly? Is it normal? Is it what is causing the
> problem?
>
> The commands "netstat -a | grep 66.249" shows 20 connections from
> googlebot ip, 1 in ESTABILISHEd status, the others in TIME WAIT.
>
> If i restart apache, everything gets back to normal.
>
> Any suggestions?
> THanks!
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
Re: [users@httpd] suddendly high cpu load because of googlebot
Posted by Simone Frattegiani <si...@gmail.com>.
Ok, i've started studying a few things, especially fastcgi, because
after reading your messages, i'm starting to think that one of the
scripts on a website is causing all this... anyway, in the meanwhile,
it happen again... so i hope to give you a clearer view on this
matter.
There are about 50 pids for apache, and 50 for mysql.
Top gives this:
top - 09:01:05 up 14 days, 14:01, 0 users, load average: 36.45, 36.33, 36.48
Tasks: 232 total, 36 running, 196 sleeping, 0 stopped, 0 zombie
Cpu(s): 52.1%us, 1.9%sy, 0.2%ni, 43.4%id, 1.6%wa, 0.4%hi, 0.4%si, 0.0%st
Mem: 3115464k total, 2993804k used, 121660k free, 207764k buffers
Swap: 5177336k total, 76k used, 5177260k free, 2039608k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6343 apache 25 0 34148 15m 3540 R 19.6 0.5 0:05.49 /usr/sbin/httpd -k
start -DSSL
27394 apache 25 0 41588 23m 4328 R 19.6 0.8 30:35.21 /usr/sbin/httpd
-k start -DSSL
28396 apache 25 0 44904 26m 3692 R 19.6 0.9 24:19.87 /usr/sbin/httpd
-k start -DSSL
28607 apache 25 0 34248 15m 3616 R 19.6 0.5 21:07.42 /usr/sbin/httpd
-k start -DSSL
27235 apache 25 0 40412 22m 3868 R 9.8 0.7 19:15.02 /usr/sbin/httpd -k
start -DSSL
1602 apache 25 0 34284 15m 3620 R 5.9 0.5 3:28.19 /usr/sbin/httpd -k start -DSSL
6322 apache 18 0 41236 22m 3676 S 2.0 0.8 0:00.78 /usr/sbin/httpd -k start -DSSL
1 root 15 0 2160 596 516 S 0.0 0.0 0:01.24 init [3]
2 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [migration/0]
[...]
And here is a screenshot of server-status: www . supermariolovers.com/status.png
These googlebot connections are not in time wait, and they're taking
up a slot! So i guess it's entirely the script's fault... am i right?
Cheers!
On Fri, Mar 9, 2012 at 8:07 PM, Simone Frattegiani
<si...@gmail.com> wrote:
> Thank you so much for all your helpful advice!
>
> I'll take a few days to review all your suggestions and anylize the
> problem, and i will let you know as soon as i'll have news!
>
> Cheers :)
>
> On Fri, Mar 9, 2012 at 2:01 PM, Tom Evans <te...@googlemail.com> wrote:
>> On Fri, Mar 9, 2012 at 8:07 AM, Simone Frattegiani
>> <si...@gmail.com> wrote:
>>> Hello,
>>>
>>> i suddendly started having CPU load issues, like this:
>>
>> A couple of things:
>>
>>> Server-status shows this:
>>>
>>> […]
>>> ReqPerSec: 6.37383
>>> BusyWorkers: 99
>>> IdleWorkers: 10
>>> Scoreboard: WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
>>>
>>
>> This is what worries me. You are handling 6.3 requests per second, but
>> have 100 active workers handling requests. This means your requests
>> are really, really slow. When requests are really slow, and the server
>> gets a little busy, two things happen:
>>
>> 1) Requests take even longer to serve, as the resource contention
>> (probably database) increases
>> 2) Apache has to serve more simultaneous requests, and so has to have
>> more children, which uses more CPU to start and run the processes, and
>> more RAM is used on the new process.
>>
>> When this reaches the tipping point, which can be due to not enough
>> RAM, not enough CPU, not enough IOPs, Apache has to start processes
>> faster than it can serve requests, and load will metaphorically
>> explode.
>>
>>> […]
>>> The commands "netstat -a | grep 66.249" shows 20 connections from
>>> googlebot ip, 1 in ESTABILISHEd status, the others in TIME WAIT.
>>>
>>
>> So this may be largely irrelevant. If they are all in TIME_WAIT, then
>> that is a keep alive connection. A keep alive connection isn't using
>> your CPU, so it isn't slowing down your webserver. What it is doing is
>> taking up a slot, which isn't ideal.
>>
>>> If i restart apache, everything gets back to normal.
>>>
>>> Any suggestions?
>>> THanks!
>>>
>>
>> I have a couple!
>>
>> First of all, you are using prefork MPM. This means to get an extra
>> slot, apache has to fork and start up a new child. This is not
>> efficient! If you use the worker or event MPM, then each child has
>> multiple threads, and so you do not require to start, or keep
>> starting, so many children. You will also have less CPU/RAM costs per
>> slot.
>>
>> This will stop CPU usage exploding when a lot of requests come in, and
>> hopefully you can serve requests faster.
>>
>> Finally, if you are worried about keep alive connections from google
>> bot, you should seriously consider event MPM. This will use a single
>> thread to handle ALL keep alive connections, waiting for a new request
>> on it, and handing the request off to a different thread once there is
>> data. This means you don't use multiple slots to handle keep alive
>> connections.
>>
>> Finally, you don't mention it, but I assume you are running some web
>> application, like PHP? I would recommend using fastcgi to host the
>> application, divorce it entirely from apache.
>> This will give you a clear idea of how much of your resources are
>> being devoted to PHP, and will actually reduce the amount of resources
>> over using mod_php. mod_php adds a PHP interpreter in every apache
>> child, whether it is serving a PHP request, a static file from disk,
>> or monitoring a keep alive connection.
>>
>> You can also strictly control how many PHP processes are used when
>> running with fastcgi.
>>
>> Cheers
>>
>> Tom
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] suddendly high cpu load because of googlebot
Posted by Simone Frattegiani <si...@gmail.com>.
Thank you so much for all your helpful advice!
I'll take a few days to review all your suggestions and anylize the
problem, and i will let you know as soon as i'll have news!
Cheers :)
On Fri, Mar 9, 2012 at 2:01 PM, Tom Evans <te...@googlemail.com> wrote:
> On Fri, Mar 9, 2012 at 8:07 AM, Simone Frattegiani
> <si...@gmail.com> wrote:
>> Hello,
>>
>> i suddendly started having CPU load issues, like this:
>
> A couple of things:
>
>> Server-status shows this:
>>
>> […]
>> ReqPerSec: 6.37383
>> BusyWorkers: 99
>> IdleWorkers: 10
>> Scoreboard: WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
>>
>
> This is what worries me. You are handling 6.3 requests per second, but
> have 100 active workers handling requests. This means your requests
> are really, really slow. When requests are really slow, and the server
> gets a little busy, two things happen:
>
> 1) Requests take even longer to serve, as the resource contention
> (probably database) increases
> 2) Apache has to serve more simultaneous requests, and so has to have
> more children, which uses more CPU to start and run the processes, and
> more RAM is used on the new process.
>
> When this reaches the tipping point, which can be due to not enough
> RAM, not enough CPU, not enough IOPs, Apache has to start processes
> faster than it can serve requests, and load will metaphorically
> explode.
>
>> […]
>> The commands "netstat -a | grep 66.249" shows 20 connections from
>> googlebot ip, 1 in ESTABILISHEd status, the others in TIME WAIT.
>>
>
> So this may be largely irrelevant. If they are all in TIME_WAIT, then
> that is a keep alive connection. A keep alive connection isn't using
> your CPU, so it isn't slowing down your webserver. What it is doing is
> taking up a slot, which isn't ideal.
>
>> If i restart apache, everything gets back to normal.
>>
>> Any suggestions?
>> THanks!
>>
>
> I have a couple!
>
> First of all, you are using prefork MPM. This means to get an extra
> slot, apache has to fork and start up a new child. This is not
> efficient! If you use the worker or event MPM, then each child has
> multiple threads, and so you do not require to start, or keep
> starting, so many children. You will also have less CPU/RAM costs per
> slot.
>
> This will stop CPU usage exploding when a lot of requests come in, and
> hopefully you can serve requests faster.
>
> Finally, if you are worried about keep alive connections from google
> bot, you should seriously consider event MPM. This will use a single
> thread to handle ALL keep alive connections, waiting for a new request
> on it, and handing the request off to a different thread once there is
> data. This means you don't use multiple slots to handle keep alive
> connections.
>
> Finally, you don't mention it, but I assume you are running some web
> application, like PHP? I would recommend using fastcgi to host the
> application, divorce it entirely from apache.
> This will give you a clear idea of how much of your resources are
> being devoted to PHP, and will actually reduce the amount of resources
> over using mod_php. mod_php adds a PHP interpreter in every apache
> child, whether it is serving a PHP request, a static file from disk,
> or monitoring a keep alive connection.
>
> You can also strictly control how many PHP processes are used when
> running with fastcgi.
>
> Cheers
>
> Tom
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] suddendly high cpu load because of googlebot
Posted by Tom Evans <te...@googlemail.com>.
On Fri, Mar 9, 2012 at 8:07 AM, Simone Frattegiani
<si...@gmail.com> wrote:
> Hello,
>
> i suddendly started having CPU load issues, like this:
A couple of things:
> Server-status shows this:
>
> […]
> ReqPerSec: 6.37383
> BusyWorkers: 99
> IdleWorkers: 10
> Scoreboard: WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
>
This is what worries me. You are handling 6.3 requests per second, but
have 100 active workers handling requests. This means your requests
are really, really slow. When requests are really slow, and the server
gets a little busy, two things happen:
1) Requests take even longer to serve, as the resource contention
(probably database) increases
2) Apache has to serve more simultaneous requests, and so has to have
more children, which uses more CPU to start and run the processes, and
more RAM is used on the new process.
When this reaches the tipping point, which can be due to not enough
RAM, not enough CPU, not enough IOPs, Apache has to start processes
faster than it can serve requests, and load will metaphorically
explode.
> […]
> The commands "netstat -a | grep 66.249" shows 20 connections from
> googlebot ip, 1 in ESTABILISHEd status, the others in TIME WAIT.
>
So this may be largely irrelevant. If they are all in TIME_WAIT, then
that is a keep alive connection. A keep alive connection isn't using
your CPU, so it isn't slowing down your webserver. What it is doing is
taking up a slot, which isn't ideal.
> If i restart apache, everything gets back to normal.
>
> Any suggestions?
> THanks!
>
I have a couple!
First of all, you are using prefork MPM. This means to get an extra
slot, apache has to fork and start up a new child. This is not
efficient! If you use the worker or event MPM, then each child has
multiple threads, and so you do not require to start, or keep
starting, so many children. You will also have less CPU/RAM costs per
slot.
This will stop CPU usage exploding when a lot of requests come in, and
hopefully you can serve requests faster.
Finally, if you are worried about keep alive connections from google
bot, you should seriously consider event MPM. This will use a single
thread to handle ALL keep alive connections, waiting for a new request
on it, and handing the request off to a different thread once there is
data. This means you don't use multiple slots to handle keep alive
connections.
Finally, you don't mention it, but I assume you are running some web
application, like PHP? I would recommend using fastcgi to host the
application, divorce it entirely from apache.
This will give you a clear idea of how much of your resources are
being devoted to PHP, and will actually reduce the amount of resources
over using mod_php. mod_php adds a PHP interpreter in every apache
child, whether it is serving a PHP request, a static file from disk,
or monitoring a keep alive connection.
You can also strictly control how many PHP processes are used when
running with fastcgi.
Cheers
Tom
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] suddendly high cpu load because of googlebot
Posted by Pete Houston <ph...@openstrike.co.uk>.
On Fri, Mar 09, 2012 at 09:07:34AM +0100, Simone Frattegiani wrote:
>
> Server-status shows this:
>
> Total Accesses: 292068
> Total kBytes: 5540956
> CPULoad: .0884054
> Uptime: 45823
> ReqPerSec: 6.37383
> BytesPerSec: 123823
> BytesPerReq: 19426.8
> BusyWorkers: 99
> IdleWorkers: 10
> Scoreboard: WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
> WWWWWWWWWWWWWWWWWWWWWWWWWWKWWWWWCWWWWWWWWWKWWWWWWWWWWKWWW_W_CWWW__WS__K_WWK__K_S........
> ........................................................
> ...................................................
>
> With hundreds of lines like this (66.249.66.44 is googlebot):
>
> 24-0 - 0/0/8 . 0.02 661 34 0.0 0.00 0.00 66.249.66.44
> www.mydomain.com GET /mypage
>
> So, please correct me if i'm wrong, that 661 in the SS column means
> that 661 seconds have passed since the last time Googlebot used that
> slot to make a request...
That is correct.
> so basically the server is keeping alive a
> connection that has been idle for like 11 minutes!!
That is incorrect. The hyphen in the second field indicates that there
is no running process associated with this slot. Furthermore the dot in
the status collumn indicates that it is an empty slot (look at the
Scoreboard Key for meanings of characters in the status column).
> Did i understand correctly? Is it normal? Is it what is causing the problem?
It is normal, so that's not anything to worry about. The problem is
more likely to be connected with the surprisingly large proportion of
your slots in the W status. When the problem occurs, check to see what
those W status processes are doing (also check you error log, of course).
HTH,
Pete
--
Openstrike - improving business through open source
http://www.openstrike.co.uk/ or call 01722 770036 / 07092 020107