You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by oh...@cox.net on 2012/07/16 16:47:54 UTC
[users@httpd] Apache taking (exactly) 30 seconds to serve static images
Hi,
We are using Apache proxying and also host some static content (images, under htdocs) on the Apache, and we're finding that, intermittently (we'll have a page, and say, 2 images will take 30 seconds), it is taking a long time to serve some images, almost exactly 30 seconds. The Apache is running on Redhat EL5, I think.
We normally run several instances behind a load balancer (Cisco ACE) and use SSL, but we're seeing this even if we shutdown all Apache instances except for 1, so this doesn't seem to be an SSL session stickiness problem.
Has anyone encountered something like this. What timeouts might be set to 30 seconds in Apache that might be causing this kind of behavior?
Thanks,
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Solved was Re: [users@httpd] Apache taking (exactly) 30 seconds to
serve static images
Posted by oh...@cox.net.
---- "Macks wrote:
> Can you run either strace (or dtrace depending on platform) against the running process and see what it is doing during the request?
>
> A
> --
> Aaron Macks
> Sr. Unix Systems Engineer
>
> Harvard Business Publishing
> 300 North Beacon St. | Watertown, MA 02472
> (617) 783-7461 | Fax: (617) 783-7467
> www.harvardbusiness.org | Cell:(978) 317-3614
>
> On Jul 16, 2012, at 6:30 PM, <oh...@cox.net>
> wrote:
>
> >
> > ---- Vivek Nambiar <vi...@gmail.com> wrote:
> >> May be you can add the hostname and ipaddress in the linux hosts
> >> file,restart apache and see if that makes any difference??
> >>
> >> Thanks
> >> VIVEK
> >>
> >
> > Hi Vivek,
> >
> > I made suer that there was an entry in /etc/hosts, with both the FQDN and short hostname and bounced Apache, and still have the same problem.
> >
> > Again, I'm wondering what might be the difference re. why does it work fine when I run Apache in single process mode, but then it has this stalling problem when I run it normally?
> >
> > Jim
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> > For additional commands, e-mail: users-help@httpd.apache.org
> >
>
Hi,
For the record, I think that we were able to eliminate these problems. There were a number of things:
- We had to add in the httpd-ssl.conf:
KeepAlive Off
- Also, I noted that when I ran "mount" it was showing that the SSLSessionCache and the semaphone/mutex directives in the httpd-ssl.conf were pointing to a mount point on which a mapped volume was mounted, rather than a physical drive. Also, the mount point didn't have "nosuid". So, I changed both directives to point to /tmp
After we bounced the Apache, it worked, both in single-process mode and in normal mode.
The original hint was that it would run in single-process mode, so I guessed that the problem when in non-single process mode was related to some mechanism that the processes were using to coordinate, which lead me to the SSLSessionCache and semaphore/mutex directives.
Thanks,
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache taking (exactly) 30 seconds to serve
static images
Posted by "Macks, Aaron" <am...@harvardbusiness.org>.
Can you run either strace (or dtrace depending on platform) against the running process and see what it is doing during the request?
A
--
Aaron Macks
Sr. Unix Systems Engineer
Harvard Business Publishing
300 North Beacon St. | Watertown, MA 02472
(617) 783-7461 | Fax: (617) 783-7467
www.harvardbusiness.org | Cell:(978) 317-3614
On Jul 16, 2012, at 6:30 PM, <oh...@cox.net>
wrote:
>
> ---- Vivek Nambiar <vi...@gmail.com> wrote:
>> May be you can add the hostname and ipaddress in the linux hosts
>> file,restart apache and see if that makes any difference??
>>
>> Thanks
>> VIVEK
>>
>
> Hi Vivek,
>
> I made suer that there was an entry in /etc/hosts, with both the FQDN and short hostname and bounced Apache, and still have the same problem.
>
> Again, I'm wondering what might be the difference re. why does it work fine when I run Apache in single process mode, but then it has this stalling problem when I run it normally?
>
> Jim
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache taking (exactly) 30 seconds to serve
static images
Posted by oh...@cox.net.
---- Vivek Nambiar <vi...@gmail.com> wrote:
> May be you can add the hostname and ipaddress in the linux hosts
> file,restart apache and see if that makes any difference??
>
> Thanks
> VIVEK
>
Hi Vivek,
I made suer that there was an entry in /etc/hosts, with both the FQDN and short hostname and bounced Apache, and still have the same problem.
Again, I'm wondering what might be the difference re. why does it work fine when I run Apache in single process mode, but then it has this stalling problem when I run it normally?
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache taking (exactly) 30 seconds to serve static images
Posted by Vivek Nambiar <vi...@gmail.com>.
May be you can add the hostname and ipaddress in the linux hosts
file,restart apache and see if that makes any difference??
Thanks
VIVEK
On Tue, Jul 17, 2012 at 2:26 AM, David Hubbard <
dhubbard@dino.hostasaurus.com> wrote:
> From: Garrison, Jim (ETW) [mailto:Jim.Garrison@nike.com]
> >
> > > -----Original Message-----
> > > Subject: Re: [users@httpd] Apache taking (exactly) 30
> > seconds to serve
> > > static images
> >
> > Just a wild guess, but is it possible Apache is configured to resolve
> > Hostnames and rDNS is not set up correctly?
>
> Additionaly, same issue can occur if a deny statement using a hostname
> is put in a .htaccess file.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
RE: [users@httpd] Apache taking (exactly) 30 seconds to serve
static images
Posted by oh...@cox.net.
---- David Hubbard <dh...@dino.hostasaurus.com> wrote:
> From: Garrison, Jim (ETW) [mailto:Jim.Garrison@nike.com]
> >
> > > -----Original Message-----
> > > Subject: Re: [users@httpd] Apache taking (exactly) 30
> > seconds to serve
> > > static images
> >
> > Just a wild guess, but is it possible Apache is configured to resolve
> > Hostnames and rDNS is not set up correctly?
>
> Additionaly, same issue can occur if a deny statement using a hostname
> is put in a .htaccess file.
>
David,
We don't have / use any .htaccess stuff.
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] Apache taking (exactly) 30 seconds to serve static images
Posted by David Hubbard <dh...@dino.hostasaurus.com>.
From: Garrison, Jim (ETW) [mailto:Jim.Garrison@nike.com]
>
> > -----Original Message-----
> > Subject: Re: [users@httpd] Apache taking (exactly) 30
> seconds to serve
> > static images
>
> Just a wild guess, but is it possible Apache is configured to resolve
> Hostnames and rDNS is not set up correctly?
Additionaly, same issue can occur if a deny statement using a hostname
is put in a .htaccess file.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] Apache taking (exactly) 30 seconds to serve
static images
Posted by oh...@cox.net.
---- "Garrison wrote:
> > -----Original Message-----
> > Subject: Re: [users@httpd] Apache taking (exactly) 30 seconds to serve
> > static images
>
> Just a wild guess, but is it possible Apache is configured to resolve
> Hostnames and rDNS is not set up correctly?
>
Jim,
Thanks for the suggestion, but "HostnameLookups" is not in the httpd.conf or other conf files. I think that is "Off" by default.
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] Apache taking (exactly) 30 seconds to serve
static images
Posted by "Garrison, Jim (ETW)" <Ji...@nike.com>.
> -----Original Message-----
> Subject: Re: [users@httpd] Apache taking (exactly) 30 seconds to serve
> static images
Just a wild guess, but is it possible Apache is configured to resolve
Hostnames and rDNS is not set up correctly?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache taking (exactly) 30 seconds to serve
static images
Posted by oh...@cox.net.
---- Nick Kew <ni...@webthing.com> wrote:
> On Mon, 16 Jul 2012 10:47:54 -0400
> <oh...@cox.net> wrote:
>
> > Has anyone encountered something like this. What timeouts might be set to 30 seconds in Apache that might be causing this kind of behavior?
>
> Check your keepalive.
>
> If your browser, or some agent on the network, is getting its knickers
> in a twist about making lots of requests, it might be waiting for one
> keepalive connection to close before opening another through which
> it's scheduled to fetch those images.
>
> If you have any kind of rate-limiting or DoS-protection module in Apache,
> that could also be delaying a connection if the client is trying to
> open more than your server defences permit.
>
> --
> Nick Kew
Hi Nick,
This is a really strange one. The page is an HTML login page, with not much too it, and has some links for background images (maybe 4-5 of them). Not much to it. The images aren't huge either, max ~70KB.
- Apache is 2.2.22, which we built, on Redhat 64bit.
- The problem only happens if we enable SSL on Apache, either 1-way or 2-way SSL have same problem.
- Sometimes the page comes up. Sometimes it comes up partially, and then sits there for awhile. Firefox Firebug is showing "aborted" (in red) for the image GETs in the latter case.
- There are no firewalls or DOS-type software. I'm going directly from the browser (Firefox and IE) to Apache.
Thus far what we've tried:
- Changed SSLRandomKey to point to /dev/urandom ==> no change in problem
- Tried adding to <VirtualHost> in ssl.conf:
KeepAlive off
KeepAliveTimeout 2
==> no change in problem.
- Apache error_log doesn't show anything weird/errors.
- As I mentioned, the problem can happen with either Firefox or IE
- Apache is built with MPM pre-fork, here's the "httpd -V":
Server version: Apache/2.2.22 (Unix)
Server built: Jun 20 2012 12:11:29
Server's Module Magic Number: 20051115:30
Server loaded: APR 1.4.5, APR-Util 1.4.1
Compiled using: APR 1.4.5, APR-Util 1.4.1
Architecture: 64-bit
Server MPM: Prefork
threaded: no
forked: yes (variable process count)
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=128
-D HTTPD_ROOT="/apps/oracle/apache"
-D SUEXEC_BIN="/apps/oracle/apache/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="logs/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
And, here's the weirdest part: I *think* that if I run Apache in single-process mode (-k start -X), the problem doesn't happen. I've tried a bunch of tests when Apache is in single-process mode, and haven't been able to get the problem yet. Assuming that I continue to not see the problem when in single-process mode, does that point to something as the problem?
Thanks,
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache taking (exactly) 30 seconds to serve
static images
Posted by Nick Kew <ni...@webthing.com>.
On Mon, 16 Jul 2012 10:47:54 -0400
<oh...@cox.net> wrote:
> Has anyone encountered something like this. What timeouts might be set to 30 seconds in Apache that might be causing this kind of behavior?
Check your keepalive.
If your browser, or some agent on the network, is getting its knickers
in a twist about making lots of requests, it might be waiting for one
keepalive connection to close before opening another through which
it's scheduled to fetch those images.
If you have any kind of rate-limiting or DoS-protection module in Apache,
that could also be delaying a connection if the client is trying to
open more than your server defences permit.
--
Nick Kew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org