You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Tony Finch <do...@dotat.at> on 1999/01/04 17:27:35 UTC
mass vhosting, was Re: [STATUS] (apache-1.3) Sun Jan 3 23:45:22 EST 1999
> 1.3 STATUS:
> Last modified at [$Date: 1999/01/03 13:46:16 $]
>
>Documentation that needs writing:
>
> * Need a document explaining mod_rewrite/"UseCanonicalName off" based
> virtualhosting. (If it exists already I can't find it easily.)
> => It still doesn't exists but I've already assembled the relevant
> information and config snippets. We just have to write a
> vhost-xxx.html document out of it. -- rse
I have a text file (below) which I'll HTMLise if you like it. A couple
of the examples are derived from http://www.engelschall.com/pw/apache/rewriteguide/
Would it be sensible to force hostnames and ServerNames to be all
lower case? It would make handling this sort of thing slightly simpler
:-)
Tony.
--
f.a.n.finch dot@dotat.at fanf@demon.net
This document describes various techniques for efficiently serving an
arbitrary number of virtual hosts with Apache.
Motivation
==========
These techniques are useful if your httpd.conf contains hundreds of
<VirtualHost> sections that are substantially the same, for example:
#### extract from httpd.conf
NameVirtualHost 111.22.33.44
<VirtualHost 111.22.33.44>
ServerName www.customer-1.com
DocumentRoot /www/hosts/www.customer-1.com/docs
ScriptAlias /cgi-bin/ /www/hosts/www.customer-1.com/cgi-bin
</VirtualHost>
<VirtualHost 111.22.33.44>
ServerName www.customer-2.com
DocumentRoot /www/hosts/www.customer-2.com/docs
ScriptAlias /cgi-bin/ /www/hosts/www.customer-2.com/cgi-bin
</VirtualHost>
# blah blah blah
<VirtualHost 111.22.33.44>
ServerName www.customer-N.com
DocumentRoot /www/hosts/www.customer-N.com/docs
ScriptAlias /cgi-bin/ /www/hosts/www.customer-N.com/cgi-bin
</VirtualHost>
#### end
The basic idea is to replace all of the static <VirtualHost>
configuration with a mechanism that works it out dynamically. This has
a number of advantages:
(1) Your configuration file is smaller so Apache starts faster.
(2) Adding virtual hosts is simply a matter of creating the
appropriate directories in the filesystem and entries in the DNS --
you don't need to reconfigure or restart Apache.
The main disadvantage is that you cannot have different log files for
each server; however if you have very many virtual hosts then doing
this is dubious anyway because it eats file descriptors. It's better
to log to a pipe or a fifo and arrange for the process at the other
end to distribute the logs (and perhaps accumulate statistics, etc.).
A LogFormat directive that includes %v for the virtual host makes it
easy to do this.
Overview of the technique
=========================
All of the dynamic virtual hosts will either be configured as part of
the main server configuration, or within a <VirtualHost> section. For
a simple (very uniform) setup, <VirtualHost> sections aren't needed at
all.
A couple of things need to be `faked' to make the dynamic virtual host
look like a normal one. The most important is the ServerName, and the
way it is determined is controlled by the UseCanonicalName directive.
With UseCanonicalName off the ServerName comes from the contents of
the Host: header in the request. If there is no Host: header then the
configured ServerName is used instead; this can be used to provide a
default virtual host which might allow backwardly compatible access to
the dynamic virtual hosts.
The other one is the DocumentRoot. This is used by the core module
when mapping URIs to filenames, but in this context its value only
matters if any CGIs or SSI files make use of the DOCUMENT_ROOT
environment variable. This is an Apache (NCSA?) extension to the CGI
spec and as such shouldn't really be relied upon, especially because
this technique breaks it: there isn't currently a fix.
The meat of the mechanism works via Apache's URI-to-filename
translation API phase. This is used by a number of modules:
mod_rewite, mod_alias, mod_userdir, and the core module. In the
default configuration these modules are called in that order and given
a chance to say that they know what the filename is. Most of these
modules do it in a fairly simple fashion except for mod_rewrite,
which provides enough functionality to do all sorts of sick and
twisted things (like dynamic virtual hosting).
The dynamic virtual hosting idea is very simple: use the ServerName as
well as the URI to determine the corresponding filename.
Name-based dynamic virtual hosts with mod_rewrite
=================================================
This configuration snippet implements the virtual host arrangement
outlined in the Motivation section above, but in a generic fashion.
#### extract from httpd.conf
# dynamic ServerName
UseCanonicalName Off
# splittable logs
LogFormat "%v %h %l %u %t \"%r\" %s %b" vcommon
CustomLog logs/access_log vcommon
<Directory /www/hosts>
# ExecCGI is needed here because we can't force
# CGI execution in the way that ScriptAlias does
Options FollowSymLinks ExecCGI
</Directory>
## now for the hard bit
RewriteEngine On
# a ServerName derived from a Host: header may be any case at all
RewriteMap lowercase int:tolower
## deal with normal documents first:
# allow Alias /icons/ to work -- repeat for other aliases
RewriteCond %{REQUEST_URI} !^/icons/
# allow CGIs to work
RewriteCond %{REQUEST_URI} !^/cgi-bin/
# do the magic
RewriteRule ^/(.*)$ /www/hosts/${lowercase:%{SERVER_NAME}}/docs/$1
## and now deal with CGIs -- we have to force a MIME type
RewriteCond %{REQUEST_URI} ^/cgi-bin/
RewriteRule ^/(.*)$ /www/hosts/${lowercase:%{SERVER_NAME}}/cgi-bin/$1 [T=application/x-httpd-cgi]
# that's it!
#### end
A virtually hosted homepages system
===================================
based on <http://www.engelschall.com/pw/apache/rewriteguide/#virtuserhost>
This is an adjustment of the above system tailored for a homepages
server. Using slightly more complicated rewriting rules we can select
substrings of the hostname to use in the filename so that e.g. the
documents for www.user.isp.com are found in /home/user/.
#### extract from httpd.conf
RewriteEngine on
RewriteMap lowercase int:tolower
# allow CGIs to work -- we'll only allow standard server-wide CGIs
RewriteCond %{REQUEST_URI} !^/cgi-bin/
# check the hostname is right so that the RewriteRule works
RewriteCond ${lowercase:%{HTTP_HOST}} ^www\.[a-z-]+\.isp\.com$
# concatenate the virtual host name onto the start of the URI
# the [C] means do the next rewrite on the result of this one
RewriteRule ^(.+) ${lowercase:%{HTTP_HOST}}$1 [C]
# now create the real file name
RewriteRule ^www\.([a-z-]+)\.isp\.com/(.*) /home/$1/$2
# define the global CGI directory
ScriptAlias /cgi-bin/ /www/std-cgi/
#### end
Using a separate virtual host configuration file
================================================
based on <http://www.engelschall.com/pw/apache/rewriteguide/#massvhost>
This arrangement uses a separate configuration file to make the
translation from virtual host to document root completely configurable
again.
#### extract from vhost.map
www.customer-1.com /www/customers/1
www.customer-2.com /www/customers/2
# ...
www.customer-N.com /www/customers/N
#### end
#### extract from httpd.conf
# as above, then:
RewriteEngine on
RewriteMap lowercase int:tolower
# define the map file
RewriteMap vhost txt:/www/conf/vhost.map
# deal with aliases as above
RewriteCond %{REQUEST_URI} !^/icons/
RewriteCond %{REQUEST_URI} !^/cgi-bin/
RewriteCond ${lowercase:%{SERVER_NAME}} ^(.+)$
# this does the file-based remap
RewriteCond ${vhost:%1} ^(/.*)$
RewriteRule ^/(.*)$ %1/docs/$1
RewriteCond %{REQUEST_URI} ^/cgi-bin/
RewriteCond ${lowercase:%{SERVER_NAME}} ^(.+)$
RewriteCond ${vhost:%1} ^(/.*)$
RewriteRule ^/(.*)$ %1/cgi-bin/$1
#### end
Using more than one virtual hosting system on the same server
=============================================================
With more complicated setups, you can use Apache's normal
<VirtualHost> directives to control the scope of all the rewrite
stuff. For example, you could have one IP address for homepages
customers and another for commercial customers with the following
setup. This can of course be combined with convential <VirtualHost>
sections.
#### extract from httpd.conf
UseCanonicalName Off
LogFormat "%v %h %l %u %t \"%r\" %s %b" vcommon
CustomLog logs/access_log vcommon
<Directory /www/commercial>
Options FollowSymLinks ExecCGI
AllowOverride All
</Directory>
<Directory /www/homepages>
Options FollowSymLinks
AllowOverride None
</Directory>
<VirtualHost 111.22.33.44>
ServerName www.commercial.isp.com
RewriteEngine On
RewriteMap lowercase int:tolower
RewriteCond %{REQUEST_URI} !^/icons/
RewriteCond %{REQUEST_URI} !^/cgi-bin/
RewriteRule ^/(.*)$ /www/commercial/${lowercase:%{SERVER_NAME}}/docs/$1
RewriteCond %{REQUEST_URI} ^/cgi-bin/
RewriteRule ^/(.*)$ /www/commercial/${lowercase:%{SERVER_NAME}}/cgi-bin/$1 [T=application/x-httpd-cgi]
</VirtualHost>
<VirtualHost 111.22.33.45>
ServerName www.homepages.isp.com
RewriteEngine on
RewriteMap lowercase int:tolower
RewriteCond %{REQUEST_URI} !^/cgi-bin/
RewriteCond ${lowercase:%{HTTP_HOST}} ^www\.[a-z-]+\.isp\.com$
RewriteRule ^(.+) ${lowercase:%{HTTP_HOST}}$1 [C]
RewriteRule ^www\.([a-z-]+)\.isp\.com/(.*) /www/homepages/$1/$2
ScriptAlias /cgi-bin/ /www/std-cgi/
</VirtualHost>
#### end
Re: mass vhosting, was Re: [STATUS] (apache-1.3) Sun Jan 3
23:45:22 EST 1999
Posted by Brian Behlendorf <br...@hyperreal.org>.
At 11:24 AM 1/5/99 +0000, Tony Finch wrote:
>Brian Behlendorf <br...@hyperreal.org> wrote:
>>This is really good; if you HTMLize it I'll commit it next to the other
>>vhost documentation.
>
>Thanks. Shall I send it to the list or to you directly?
Just send it directly, once it's in CVS people will still have a chance to
comment and change things.
>>I'm sure the #1 question people will have after they implement it is, "I
>>want to set 'Redirect /blah /blah' for a certain vhost. How do I do that
>>under this system?" which I guess the answer to is "two ways: do an
>>explicit <VirtualHost> section, or use a .htaccess in that vhost's dir."
>
>I'll include something like this too -- I think I need to make it
>clearer that mod_rewrite overrides mod_alias etc. so care needs to be
>taken with using features provided by it.
That would be great. Maybe we could even distribute a separate, modified
httpd.conf-dist, that shows what can & can't be done, a la
highperformance.conf-dist.
Brian
--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
History is made at night; brian@hyperreal.org
character is what you are in the dark.
Re: mass vhosting, was Re: [STATUS] (apache-1.3) Sun Jan 3
23:45:22 EST 1999
Posted by Tony Finch <do...@dotat.at>.
Brian Behlendorf <br...@hyperreal.org> wrote:
>Tony Finch wrote:
>>
>>I have a text file (below) which I'll HTMLise if you like it. A couple
>>of the examples are derived from
>>http://www.engelschall.com/pw/apache/rewriteguide/
>
>This is really good; if you HTMLize it I'll commit it next to the other
>vhost documentation.
Thanks. Shall I send it to the list or to you directly?
>>If there is no Host: header then the configured ServerName is used
>>instead; this can be used to provide a default virtual host which
>>might allow backwardly compatible access to the dynamic virtual
>>hosts.
>
>There are lots of hidden problems with trying to do that though;
>forcing non-'/'-rooted relative pathnames in all your HREF links is
>one, but I'm sure there are other more subtle ones, and ServerPath
>does a good job at addressing it, but still I worry about "new
>feature pressure" that such a suggestion might entail.
Yes, that should be mentioned because it's easily overlooked. I have a
sneaking suspicion that mod_rewrite could be persuaded to do something
like a dynamic ServerPath, but there are all sorts of reasons why this
isn't worth it: it requires co-operation from customers to work and
therefore it frequently won't; it causes cache pollution by causing
multiple URLs to refer to the same resource; etc.
>I don't think it'd be unreasonable to suggest instead that
>non-Host:-header browsers are just out of luck. I mean, what are we
>talking about here? netscape pre-2.0, msie pre-3.0, lynx pre-2.4,
>what else?
Broken robots, but we hardly care about them.
>I'm sure the #1 question people will have after they implement it is, "I
>want to set 'Redirect /blah /blah' for a certain vhost. How do I do that
>under this system?" which I guess the answer to is "two ways: do an
>explicit <VirtualHost> section, or use a .htaccess in that vhost's dir."
I'll include something like this too -- I think I need to make it
clearer that mod_rewrite overrides mod_alias etc. so care needs to be
taken with using features provided by it.
Tony.
--
f.a.n.finch dot@dotat.at fanf@demon.net
Re: mass vhosting, was Re: [STATUS] (apache-1.3) Sun Jan 3 23:45:22 EST 1999
Posted by Dean Gaudet <dg...@arctic.org>.
On Wed, 6 Jan 1999, Martin Pool wrote:
> Why is it necessary to copy the Directory structures for each vhost?
> Couldn't global <Directory> configs just be shared across all of them?
It isn't necessary -- and they aren't copied... or at least they're not
supposed to be that I can remember. Looking at the code it doesn't look
like it copies them, it copies pointers to the directory structures, but
doesn't copy the structures themselves.
There's essentially no "copy this dir/vhost config" primitive in the
module interface, so it'd be like totally impossible to copy things.
There's only "create a new dir/vhost config" and "merge these two
dir/vhost configs". And no, it doesn't create a new and merge to do a
"copy".
But if you use the srm.conf or access.conf files, there is some crap that
happens -- I just suggest not using them. It rereads those files for each
vhost or some other such silliness. So if the dir sections were in one of
those files, then yeah it'd be duplicated.
Dean
Re: mass vhosting, was Re: [STATUS] (apache-1.3) Sun Jan 3 23:45:22 EST 1999
Posted by Martin Pool <mb...@wistful.humbug.org.au>.
On Mon, Jan 04, 1999 at 07:05:16PM -0800, Brian Behlendorf wrote:
> At 04:27 PM 1/4/99 +0000, Tony Finch wrote:
> >I have a text file (below) which I'll HTMLise if you like it. A couple
> >of the examples are derived from
> >http://www.engelschall.com/pw/apache/rewriteguide/
If I understand correctly, the problem with configurations like this:
<Directory /home/mbp>
...
</Directory>
<VirtualHost 1.2.3.4:80>
ServerName mbp.victim.com
ServerRoot /home/mbp
...
</VirtualHost>
<Directory /home/cjp>
...
</Directory>
<VirtualHost 1.2.3.4:80>
ServerName cjp.victim.com
ServerRoot /home/cjp
...
</VirtualHost>
[repeated many times]
is that memory usage goes up as the product of the number of
<Directory> and <VirtualHost> configurations, because Apache copies
the Directory entries for each VirtualHost. (I remember reading a PR
about this where the answer was basically "don't do that then", plus a
suggestion to use rewrites instead.)
Why is it necessary to copy the Directory structures for each vhost?
Couldn't global <Directory> configs just be shared across all of them?
--
Martin Pool
Re: mass vhosting, was Re: [STATUS] (apache-1.3) Sun Jan 3
23:45:22 EST 1999
Posted by Brian Behlendorf <br...@hyperreal.org>.
At 04:27 PM 1/4/99 +0000, Tony Finch wrote:
>I have a text file (below) which I'll HTMLise if you like it. A couple
>of the examples are derived from
>http://www.engelschall.com/pw/apache/rewriteguide/
This is really good; if you HTMLize it I'll commit it next to the other
vhost documentation.
>Would it be sensible to force hostnames and ServerNames to be all
>lower case? It would make handling this sort of thing slightly simpler
>:-)
Yep, sounds good to me. Some comments...
>A couple of things need to be `faked' to make the dynamic virtual host
>look like a normal one. The most important is the ServerName, and the
>way it is determined is controlled by the UseCanonicalName directive.
>With UseCanonicalName off the ServerName comes from the contents of
>the Host: header in the request. If there is no Host: header then the
>configured ServerName is used instead; this can be used to provide a
>default virtual host which might allow backwardly compatible access to
>the dynamic virtual hosts.
There are lots of hidden problems with trying to do that though; forcing
non-'/'-rooted relative pathnames in all your HREF links is one, but I'm
sure there are other more subtle ones, and ServerPath does a good job at
addressing it, but still I worry about "new feature pressure" that such a
suggestion might entail. I don't think it'd be unreasonable to suggest
instead that non-Host:-header browsers are just out of luck. I mean, what
are we talking about here? netscape pre-2.0, msie pre-3.0, lynx pre-2.4,
what else? I can't imagine it's more than 2% of traffic this days. All
I'm addressing here is the recommendation that would be in this doc.
I'm sure the #1 question people will have after they implement it is, "I
want to set 'Redirect /blah /blah' for a certain vhost. How do I do that
under this system?" which I guess the answer to is "two ways: do an
explicit <VirtualHost> section, or use a .htaccess in that vhost's dir."
Brian
--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
History is made at night; brian@hyperreal.org
character is what you are in the dark.