You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by jome <jo...@emoj.net> on 2006/06/21 03:07:16 UTC
[users@httpd] Thousands of VirtualHosts slowing down start/stop/graceful
Hello!
I'm running Apache 2.2.2[1] on a Mac OS X Server 10.3 box, and I'm
having some trouble which I'm pretty sure is related to having large
amounts of VirtualHosts (or large amounts of Directory). There's
currently about 2200 of them and most, if not all, follow a specific
scheme[2].
The issue is basically that when I try to run apachectl restart or
graceful the execution will take two to three minutes, and right after
it will take rougly the same time before the new httpd processes are
spawned and during much of this time the server is left unreachable.
I have tried to locate the error but all I can come up with is that it
is related to the large configuration (roughly 700k) file or the
contents of it. An idea might be that the DocumentRoot checks are taking
a lot of time, but I'm not familiar enough with neither Apache nor Mac
OS X to determine if that's the case. Another idea is that the delay is
caused by the fact that a lot more memory[3] has to be freed/allocated
when Apache is stopped and started with a large configuration file.
I am aware that this might be possible to avoid using the "mass"
instructions for VirtualHosts but if there's a way to stick with what I
have today I'd prefer that.
Any help or hints is greatly appreciated!
Thanks,
jome
1:
Output of httpd -V:
Server version: Apache/2.2.2
Server built: Jun 9 2006 16:41:37
Server's Module Magic Number: 20051115:2
Server loaded: APR 1.2.7, APR-Util 1.2.7
Compiled using: APR 1.2.7, APR-Util 1.2.7
Architecture: 32-bit
Server MPM: Prefork
threaded: no
forked: yes (variable process count)
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=128
-D HTTPD_ROOT="/usr"
-D SUEXEC_BIN="/usr/bin/suexec"
-D DEFAULT_PIDLOG="/var/run/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="/var/run/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types"
-D SERVER_CONFIG_FILE="/etc/apache2/httpd.conf"
2:
<VirtualHost ip (same on all vhosts)>
ServerName domain
DocumentRoot /path
DirectoryIndex index.html
<Directory "/path">
Options All +MultiViews -ExecCGI -Indexes
AllowOverride None
</Directory>
</VirtualHost>
3:
With small config:
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE
VSIZE
9804 httpd 0.0% 0:00.00 1 10 37 80K 21.3M 480K
47.1M
With large config:
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE
VSIZE
9685 httpd 0.0% 0:00.10 1 12 39 344K 35.1M 11.4M
55.1M
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Thousands of VirtualHosts slowing down start/stop/graceful
Posted by jome <jo...@emoj.net>.
Matus UHLAR - fantomas wrote:
> On 21.06.06 03:07, jome wrote:
>> Subject: [users@httpd] Thousands of VirtualHosts slowing down start/stop/graceful
>
>> I'm running Apache 2.2.2[1] on a Mac OS X Server 10.3 box, and I'm
>> having some trouble which I'm pretty sure is related to having large
>> amounts of VirtualHosts (or large amounts of Directory). There's
>> currently about 2200 of them and most, if not all, follow a specific
>> scheme[2].
>>
>> The issue is basically that when I try to run apachectl restart or
>> graceful the execution will take two to three minutes, and right after
>> it will take rougly the same time before the new httpd processes are
>> spawned and during much of this time the server is left unreachable.
>
> so you use different log files for virtual servers?
> If so, here is your problem - reopening thousands of files takes ages.
>
Good idea, but no, I'm using a single log file.
Thanks,
jome
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Thousands of VirtualHosts slowing down start/stop/graceful
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 21.06.06 03:07, jome wrote:
> Subject: [users@httpd] Thousands of VirtualHosts slowing down start/stop/graceful
> I'm running Apache 2.2.2[1] on a Mac OS X Server 10.3 box, and I'm
> having some trouble which I'm pretty sure is related to having large
> amounts of VirtualHosts (or large amounts of Directory). There's
> currently about 2200 of them and most, if not all, follow a specific
> scheme[2].
>
> The issue is basically that when I try to run apachectl restart or
> graceful the execution will take two to three minutes, and right after
> it will take rougly the same time before the new httpd processes are
> spawned and during much of this time the server is left unreachable.
so you use different log files for virtual servers?
If so, here is your problem - reopening thousands of files takes ages.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Fighting for peace is like fucking for virginity...
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Thousands of VirtualHosts slowing down start/stop/graceful
Posted by Joshua Slive <jo...@slive.ca>.
On 6/21/06, jome <jo...@emoj.net> wrote:
> Joshua Slive wrote:
> > On 6/20/06, jome <jo...@emoj.net> wrote:
> >
> >> I have tried to locate the error but all I can come up with is that it
> >> is related to the large configuration (roughly 700k) file or the
> >> contents of it. An idea might be that the DocumentRoot checks are taking
> >> a lot of time, but I'm not familiar enough with neither Apache nor Mac
> >> OS X to determine if that's the case. Another idea is that the delay is
> >> caused by the fact that a lot more memory[3] has to be freed/allocated
> >> when Apache is stopped and started with a large configuration file.
> >>
> >> I am aware that this might be possible to avoid using the "mass"
> >> instructions for VirtualHosts but if there's a way to stick with what I
> >> have today I'd prefer that.
> >
> > I'm not sure exactly what you want to know. Yes, this problem is
> > almost certainly due to needing to read and process a huge config file
> > and check all the document roots. Yes, your config looks like it is a
> > perfect use case for mod_vhost_alias.
>
> I forgot to say that in a previous install on the same machine with
> Apache 1.3.x, the webserver restarted in a matter of seconds and
> rendered no downtime (and that was with roughly 2800 VirtualHosts). I'm
> still puzzled that there's downtime with graceful as it is my
> understanding that new processes are supposedly spawned as the existing
> ones are dying.
>
> I guess basically my question is, can I optimize this somehow (remove
> unnecessary checks etc)? I personally can't see how processing a 800k
> file - even with checking 2000 paths - can be taking so long on this
> machine.
My understanding (which may not be prefectly accurate) is that in a
graceful restart, the old processes immediately stop accepting new
connections and the new processes begin to start up. Normally, they
would start up fast enough that new connections would be held in the
listen queue and then answered. But if the restart takes long enough,
I suppose connections could time out in the listen queue.
If restarts are taking this long, I assume that initial startup also
takes a long time?
If so, run an "apachectl -X" under strace (errr... I guess "ktrace" on
osx) and see what syscalls apache is doing that are causing the delay.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Thousands of VirtualHosts slowing down start/stop/graceful
Posted by jome <jo...@emoj.net>.
Joshua Slive wrote:
> On 6/20/06, jome <jo...@emoj.net> wrote:
>
>> I have tried to locate the error but all I can come up with is that it
>> is related to the large configuration (roughly 700k) file or the
>> contents of it. An idea might be that the DocumentRoot checks are taking
>> a lot of time, but I'm not familiar enough with neither Apache nor Mac
>> OS X to determine if that's the case. Another idea is that the delay is
>> caused by the fact that a lot more memory[3] has to be freed/allocated
>> when Apache is stopped and started with a large configuration file.
>>
>> I am aware that this might be possible to avoid using the "mass"
>> instructions for VirtualHosts but if there's a way to stick with what I
>> have today I'd prefer that.
>
> I'm not sure exactly what you want to know. Yes, this problem is
> almost certainly due to needing to read and process a huge config file
> and check all the document roots. Yes, your config looks like it is a
> perfect use case for mod_vhost_alias.
I forgot to say that in a previous install on the same machine with
Apache 1.3.x, the webserver restarted in a matter of seconds and
rendered no downtime (and that was with roughly 2800 VirtualHosts). I'm
still puzzled that there's downtime with graceful as it is my
understanding that new processes are supposedly spawned as the existing
ones are dying.
I guess basically my question is, can I optimize this somehow (remove
unnecessary checks etc)? I personally can't see how processing a 800k
file - even with checking 2000 paths - can be taking so long on this
machine.
Thanks,
jome
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Thousands of VirtualHosts slowing down start/stop/graceful
Posted by Joshua Slive <jo...@slive.ca>.
On 6/20/06, jome <jo...@emoj.net> wrote:
> I have tried to locate the error but all I can come up with is that it
> is related to the large configuration (roughly 700k) file or the
> contents of it. An idea might be that the DocumentRoot checks are taking
> a lot of time, but I'm not familiar enough with neither Apache nor Mac
> OS X to determine if that's the case. Another idea is that the delay is
> caused by the fact that a lot more memory[3] has to be freed/allocated
> when Apache is stopped and started with a large configuration file.
>
> I am aware that this might be possible to avoid using the "mass"
> instructions for VirtualHosts but if there's a way to stick with what I
> have today I'd prefer that.
I'm not sure exactly what you want to know. Yes, this problem is
almost certainly due to needing to read and process a huge config file
and check all the document roots. Yes, your config looks like it is a
perfect use case for mod_vhost_alias.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org