You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by G VM <dr...@hotmail.com> on 2012/05/14 17:41:09 UTC
[users@httpd] IO load due to apache? Normal httpd behavior or script related?
Hi all,
Recently I found out one server generating alot of load on our storage.
I did a few steps to narrow down what is the cause and what might be a solution.
As first I used iostat to get what device is using most IO:
Linux 2.6.18-274.7.1.el5 (hostname) 05/14/2012
avg-cpu: %user %nice %system %iowait %steal %idle
11.99 0.00 8.53 2.85 0.00 76.63
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 952.75 5.54 14090.70 42708891 108632689706
sda1 0.00 0.00 0.00 6988 1802
sda2 0.51 0.17 8.29 1322786 63885168
sda3 952.24 5.37 14082.42 41378693 108568802736
sdb 8.29 1.01 739.31 7791930 5699710024
dm-0 2.14 1.48 17.03 11382146 131278120
dm-1 0.76 0.78 5.47 6030578 42202752
dm-2 8.69 3.08 68.72 23737626 529796056
dm-3 1748.90 0.03 13991.19 226754 107865527240
dm-4 0.00 0.00 0.00 1192 216
dm-5 92.49 1.01 739.31 7790146 5699710024
Using ls -lh /dev/mapper I found out that dm-3 corresponds with the mountpoint for /tmp (note that the system is using LVM).
One thing that seems strange to me is that there is a huge load of blocks written however almost none get red.
Then I used lsof | grep tmp:
httpd 25142 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25386 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25429 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25431 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25454 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25622 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25792 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25794 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25846 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25849 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
...
The only other is for sshd and another for vmware tools.
If I try to do ls -lh /tmp/apc.Fm8SAS I get no such file or directory. Using the access time I would see if this file is accessed over and over again.
Am I missing something here? Is this apache doing a fstat() call for example?
Or is this file created and deleted over and over again at a very fast pace that I can't access it (this seems less likely to me).
Has anyone seen such behaviour in httpd? Or does someone knows a little more of the inner workings of httpd?
The name does me think that APC (sort of php module iirc) is having something to do with this:
apc.ini:apc.mmap_file_mask=/tmp/apc.XXXXXX
The config file seems to confirm this.
Anyone has an idea how to fix this or to go further investigate?
Thanks alot in advance!
RE: [users@httpd] IO load due to apache? Normal httpd behavior or
script related?
Posted by G VM <dr...@hotmail.com>.
So I did the change, restarted apache and... Fixed!
So changing to shm did fixed the issue with APC for PHP.
Greetings
> Date: Wed, 16 May 2012 16:26:42 +0100
> From: tevans.uk@googlemail.com
> To: users@httpd.apache.org
> Subject: Re: [users@httpd] IO load due to apache? Normal httpd behavior or script related?
>
> On Wed, May 16, 2012 at 2:57 PM, G VM <dr...@hotmail.com> wrote:
> >> Likely a script problem, httpd wont cause harm
> > Did some further investigation and it seems APC (php opcode cache) generates
> > this IO(or so it seems).
> > http://serverfault.com/questions/361032/high-disk-i-o-when-cache-is-used
> >
> >> that is pretty ancient
> > RHEL. Not that ancient.
> >
> >
> >> ahhhhhhhhhhhhhhh so this is a VM
> > Yes :p any further point on this?
> >
> >
> >> If you are not sure what APC is then you cant need it and should disable
> >> it
> > Although I might not have a full understanding of what it does the customer
> > is using it (passively). So disabling is not an option.
> > Sometimes disabling stuff from which you don't know what it does backfires
> > ;)
> >
> > One of these days I will be able to change the setting and verify that this
> > fixed my problem.
> >
> > Thanks for your response anyway:)
> >
> > Grtz
> >
>
> APC stand for Alternative PHP Cache. It aims to speed up your Apache
> webserver by caching the parsed PHP object code, in a manner I don't
> understand because I use neither PHP nor APC.
>
> However, there are two takeaways from those two sentences. The first
> is that it is a cache, the site should function without it. The second
> is that it is meant to speed up your webserver. If it is not doing so,
> and is instead causing load issues, it seems counter productive to use
> it.
>
> Cheers
>
> Tom
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
RE: [users@httpd] IO load due to apache? Normal httpd behavior or
script related?
Posted by G VM <dr...@hotmail.com>.
Hi Tom,
You absolutely have a point there.
The site functions without it, however APC improves the sites performance(it should as there is almost 100% cache hit, however I didn't and can't do decent investigation into that).
Note that the customer configures all application parts(php, apache, mysql, ...).
So if the customer writes bad code and the site doesn't perform, that is not a problem.
However load on the VM is kind of ok. It is the IO load to the san that I want to get down :)
Grtz
>
> APC stand for Alternative PHP Cache. It aims to speed up your Apache
> webserver by caching the parsed PHP object code, in a manner I don't
> understand because I use neither PHP nor APC.
>
> However, there are two takeaways from those two sentences. The first
> is that it is a cache, the site should function without it. The second
> is that it is meant to speed up your webserver. If it is not doing so,
> and is instead causing load issues, it seems counter productive to use
> it.
>
> Cheers
>
> Tom
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
Re: [users@httpd] IO load due to apache? Normal httpd behavior or
script related?
Posted by Tom Evans <te...@googlemail.com>.
On Wed, May 16, 2012 at 2:57 PM, G VM <dr...@hotmail.com> wrote:
>> Likely a script problem, httpd wont cause harm
> Did some further investigation and it seems APC (php opcode cache) generates
> this IO(or so it seems).
> http://serverfault.com/questions/361032/high-disk-i-o-when-cache-is-used
>
>> that is pretty ancient
> RHEL. Not that ancient.
>
>
>> ahhhhhhhhhhhhhhh so this is a VM
> Yes :p any further point on this?
>
>
>> If you are not sure what APC is then you cant need it and should disable
>> it
> Although I might not have a full understanding of what it does the customer
> is using it (passively). So disabling is not an option.
> Sometimes disabling stuff from which you don't know what it does backfires
> ;)
>
> One of these days I will be able to change the setting and verify that this
> fixed my problem.
>
> Thanks for your response anyway:)
>
> Grtz
>
APC stand for Alternative PHP Cache. It aims to speed up your Apache
webserver by caching the parsed PHP object code, in a manner I don't
understand because I use neither PHP nor APC.
However, there are two takeaways from those two sentences. The first
is that it is a cache, the site should function without it. The second
is that it is meant to speed up your webserver. If it is not doing so,
and is instead causing load issues, it seems counter productive to use
it.
Cheers
Tom
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] IO load due to apache? Normal httpd behavior or
script related?
Posted by G VM <dr...@hotmail.com>.
> Likely a script problem, httpd wont cause harm
Did some further investigation and it seems APC (php opcode cache) generates this IO(or so it seems).
http://serverfault.com/questions/361032/high-disk-i-o-when-cache-is-used
> that is pretty ancient
RHEL. Not that ancient.
> ahhhhhhhhhhhhhhh so this is a VM
Yes :p any further point on this?
> If you are not sure what APC is then you cant need it and should disable it
Although I might not have a full understanding of what it does the customer is using it (passively). So disabling is not an option.
Sometimes disabling stuff from which you don't know what it does backfires ;)
One of these days I will be able to change the setting and verify that this fixed my problem.
Thanks for your response anyway:)
Grtz
From: noel.butler@ausics.net
To: users@httpd.apache.org
Date: Wed, 16 May 2012 10:12:47 +1000
Subject: Re: [users@httpd] IO load due to apache? Normal httpd behavior or script related?
On Mon, 2012-05-14 at 17:41 +0200, G VM wrote:
Hi all,
Recently I found out one server generating alot of load on our storage.
Likely a script problem, httpd wont cause harm
Linux 2.6.18-274.7.1.el5
that is pretty ancient
httpd 25431 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25454 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25622 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25792 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25794 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25846 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
httpd 25849 apache DEL REG 253,3 13 /tmp/apc.Fm8SAS
...
The only other is for sshd and another for vmware tools.
ahhhhhhhhhhhhhhh so this is a VM
If I try to do ls -lh /tmp/apc.Fm8SAS I get no such file or directory. Using the access time I would see if this file is accessed over and over again.
Am I missing something here? Is this apache doing a fstat() call for example?
Or is this file created and deleted over and over again at a very fast pace that I can't access it (this seems less likely to me).
Has anyone seen such behaviour in httpd? Or does someone knows a little more of the inner workings of httpd?
The name does me think that APC (sort of php module iirc) is having something to do with this:
apc.ini:apc.mmap_file_mask=/tmp/apc.XXXXXX
The config file seems to confirm this.
If you are not sure what APC is then you cant need it and should disable it
Re: [users@httpd] IO load due to apache? Normal httpd behavior or
script related?
Posted by Noel Butler <no...@ausics.net>.
On Mon, 2012-05-14 at 17:41 +0200, G VM wrote:
> Hi all,
>
> Recently I found out one server generating alot of load on our
> storage.
>
Likely a script problem, httpd wont cause harm
> Linux 2.6.18-274.7.1.el5
>
that is pretty ancient
> httpd 25431 apache DEL REG 253,3
> 13 /tmp/apc.Fm8SAS
> httpd 25454 apache DEL REG 253,3
> 13 /tmp/apc.Fm8SAS
> httpd 25622 apache DEL REG 253,3
> 13 /tmp/apc.Fm8SAS
> httpd 25792 apache DEL REG 253,3
> 13 /tmp/apc.Fm8SAS
> httpd 25794 apache DEL REG 253,3
> 13 /tmp/apc.Fm8SAS
> httpd 25846 apache DEL REG 253,3
> 13 /tmp/apc.Fm8SAS
> httpd 25849 apache DEL REG 253,3
> 13 /tmp/apc.Fm8SAS
> ...
>
> The only other is for sshd and another for vmware tools.
>
ahhhhhhhhhhhhhhh so this is a VM
> If I try to do ls -lh /tmp/apc.Fm8SAS I get no such file or directory.
> Using the access time I would see if this file is accessed over and
> over again.
> Am I missing something here? Is this apache doing a fstat() call for
> example?
> Or is this file created and deleted over and over again at a very fast
> pace that I can't access it (this seems less likely to me).
>
> Has anyone seen such behaviour in httpd? Or does someone knows a
> little more of the inner workings of httpd?
>
> The name does me think that APC (sort of php module iirc) is having
> something to do with this:
> apc.ini:apc.mmap_file_mask=/tmp/apc.XXXXXX
>
> The config file seems to confirm this.
>
If you are not sure what APC is then you cant need it and should disable
it