You are viewing a plain text version of this content. The canonical link for it is here.
Posted to cvs@httpd.apache.org by rb...@apache.org on 2002/06/15 22:09:08 UTC
cvs commit: httpd-2.0/docs/manual/vhosts fd-limits.html.en
rbowen 2002/06/15 13:09:08
Modified: docs/manual/vhosts fd-limits.html.en
Log:
Added configuration example, code example, and a little explanation, to
facilitate logging all of your virtual hosts to a single file, and then
splitting them back up after. Note that if Apache has problems with this
many file handles, Perl might also. I'm not sure.
Revision Changes Path
1.5 +78 -7 httpd-2.0/docs/manual/vhosts/fd-limits.html.en
Index: fd-limits.html.en
===================================================================
RCS file: /home/cvs/httpd-2.0/docs/manual/vhosts/fd-limits.html.en,v
retrieving revision 1.4
retrieving revision 1.5
diff -u -r1.4 -r1.5
--- fd-limits.html.en 22 Sep 2001 19:39:26 -0000 1.4
+++ fd-limits.html.en 15 Jun 2002 20:09:08 -0000 1.5
@@ -17,7 +17,7 @@
<p>When using a large number of Virtual Hosts, Apache may run
out of available file descriptors (sometimes called <cite>file
- handles</cite> if each Virtual Host specifies different log
+ handles</cite>) if each Virtual Host specifies different log
files. The total number of file descriptors used by Apache is
one for each distinct error log file, one for every other log
file directive, plus 10-20 for internal use. Unix operating
@@ -29,11 +29,11 @@
this may not work if:</p>
<ol>
- <li>Your system does not provide the setrlimit() system
- call.</li>
+ <li>Your system does not provide the <code>setrlimit()</code>
+ system call.</li>
- <li>The setrlimit(RLIMIT_NOFILE) call does not function on
- your system (such as Solaris 2.3)</li>
+ <li>The <code>setrlimit(RLIMIT_NOFILE)</code> call does not
+ function on your system (such as Solaris 2.3)</li>
<li>The number of file descriptors required exceeds the hard
limit.</li>
@@ -46,8 +46,11 @@
<ul>
<li>Reduce the number of log files; don't specify log files
- in the VirtualHost sections, but only log to the main log
- files.</li>
+ in the <code><a
+ href="../mod/core.html#virtualhost"><VirtualHost></a></code>
+ sections, but only log to the main log files. (See <a
+ href="#splitlogs">Splitting up your log files</a>, below, for more
+ information on doing this.)</li>
<li>
If you system falls into 1 or 2 (above), then increase the
@@ -67,6 +70,74 @@
document containing further details about file descriptor
problems and how they can be solved on your operating
system.</p>
+
+<h2><a name="splitlogs">Splitting up your log files</a></h2>
+
+<p>If you want to log multiple virtual hosts to the same log file, you
+may want to split up the log files afterwards in order to run
+statistical analysis of the various virtual hosts. This can be
+accomplished in the following manner.</p>
+
+<p>First, you will need to add the virtual host information to the log
+entries. This can be done using the <code><a
+href="../mod/mod_log_config.html#logformat">LogFormat</a></code>
+directive, and the <code>%v</code> variable. Add this to the beginning
+of your log format string:</p>
+
+<blockquote><table cellpadding="10"><tr><td bgcolor="#eeeeee"><code>
+ LogFormat "%v %h %l %u %t \"%r\" %>s %b" vhost<br>
+ CustomLog logs/multiple_vhost_log vhost
+</code></td></tr></table></blockquote>
+
+<p>This will create a log file in the common log format, but with the
+canonical virtual host (whatever appears in the
+../mod/core.html#servername">ServerName</a> directive) prepended to
+each line. (See <a
+href="../mod/mod_log_config.html#formats">Custom Log Formats</a> for
+more about customizing your log files.)</p>
+
+<p>When you wish to split your log file into its component parts (one
+file per virtual host) you can use a program like the following to
+accomplish this:</p>
+
+<blockquote><table cellpadding="10"><tr><td bgcolor="#eeeeee"><pre>
+#!/usr/bin/perl
+# Filename: split_log
+# Usage: split_log multiple_vhost_log
+# Creates one log file per virtual host
+
+use strict;
+my $file = $ARGV[0]; # Name of the log file
+my %fh; # File handles
+
+# Read the log file, one line at a time
+open LOG, $file;
+foreach my $line (<LOG>) {
+ $line =~ s/^(.*?) //;
+
+ # Do we already have a file handle for this vhost?
+ unless ($fh{$1}) {
+ my $handle;
+ open ($handle, '>' , $1 . '_split');
+ $fh{$1} = $handle;
+ }
+
+ # Write out the log entry
+ select ($fh{$1});
+ print $line;
+}
+close LOG;
+
+# Close all the open file handles
+foreach my $h ( keys %fh ) {
+ close $h;
+}
+</pre></td></tr></table></blockquote>
+
+<p>This program, when run with the name of your vhost log file, will
+generate one file for each virtual host that appears in your log file.
+Each file will be called <code>hostname_split</code>.</p>
+
<!--#include virtual="footer.html" -->
</body>
</html>
Re: cvs commit: httpd-2.0/docs/manual/vhosts fd-limits.html.en
Posted by Tony Finch <do...@dotat.at>.
On Sun, Jun 16, 2002 at 06:55:45PM -0400, Joshua Slive wrote:
> Tony Finch wrote:
> > On Sat, Jun 15, 2002 at 10:15:17PM -0400, Joshua Slive wrote:
>
> >>I guess you can put pretty much whatever you like in the Host: header.
> >>It is not a major security whole, in my opinion, but it is better not
> >>allowed.
> >
> > Before this hole was fixed in 1.3 it exposed the password file etc.
>
> I don't believe so. You could only write to files with the .log extension.
I was thinking of users of mod_vhost_alias -- perhaps I should have
checked what started this thread :-)
Tony.
--
f.a.n.finch <do...@dotat.at> http://dotat.at/
SOUTHEAST TRAFALGAR: NORTHERLY 3 OR 4. MAINLY FAIR. GOOD.
---------------------------------------------------------------------
To unsubscribe, e-mail: docs-unsubscribe@httpd.apache.org
For additional commands, e-mail: docs-help@httpd.apache.org
Re: cvs commit: httpd-2.0/docs/manual/vhosts fd-limits.html.en
Posted by Joshua Slive <jo...@slive.ca>.
Tony Finch wrote:
> On Sat, Jun 15, 2002 at 10:15:17PM -0400, Joshua Slive wrote:
>>I guess you can put pretty much whatever you like in the Host: header.
>>It is not a major security whole, in my opinion, but it is better not
>>allowed. Cliff just checked in a fix to get rid of the problem in
>>httpd-2.0.
>
>
> Before this hole was fixed in 1.3 it exposed the password file etc.
I don't believe so. You could only write to files with the .log extension.
Joshua.
---------------------------------------------------------------------
To unsubscribe, e-mail: docs-unsubscribe@httpd.apache.org
For additional commands, e-mail: docs-help@httpd.apache.org
Re: cvs commit: httpd-2.0/docs/manual/vhosts fd-limits.html.en
Posted by Tony Finch <do...@dotat.at>.
On Sat, Jun 15, 2002 at 10:15:17PM -0400, Joshua Slive wrote:
> Rich Bowen wrote:
> >
> > Can you elaborate on that? Why would the vhost name ever have a slash in
> > it? I can see that it could be a security problem, but how would one
> > ever get in there?
Script kiddies.
> I guess you can put pretty much whatever you like in the Host: header.
> It is not a major security whole, in my opinion, but it is better not
> allowed. Cliff just checked in a fix to get rid of the problem in
> httpd-2.0.
Before this hole was fixed in 1.3 it exposed the password file etc.
Tony.
--
f.a.n.finch <do...@dotat.at> http://dotat.at/
IRISH SEA: SOUTHERLY 5 TO 7, OCCASIONALLY GALE 8. RAIN THEN FAIR. MODERATE
WITH FOG PATCHES BECOMING GOOD.
---------------------------------------------------------------------
To unsubscribe, e-mail: docs-unsubscribe@httpd.apache.org
For additional commands, e-mail: docs-help@httpd.apache.org
Re: cvs commit: httpd-2.0/docs/manual/vhosts fd-limits.html.en
Posted by Joshua Slive <jo...@slive.ca>.
Rich Bowen wrote:
> On Sat, 15 Jun 2002, Joshua Slive wrote:
>
>>It is fairly similar to yours, but not quite the same. Both your script
>>and the one in httpd-2.0 are missing a necessary security fix from the
>>1.3 version (strip slashes from the vhost name).
>
>
> Can you elaborate on that? Why would the vhost name ever have a slash in
> it? I can see that it could be a security problem, but how would one
> ever get in there?
I guess you can put pretty much whatever you like in the Host: header.
It is not a major security whole, in my opinion, but it is better not
allowed. Cliff just checked in a fix to get rid of the problem in
httpd-2.0.
Joshua.
---------------------------------------------------------------------
To unsubscribe, e-mail: docs-unsubscribe@httpd.apache.org
For additional commands, e-mail: docs-help@httpd.apache.org
Re: cvs commit: httpd-2.0/docs/manual/vhosts fd-limits.html.en
Posted by Rich Bowen <rb...@rcbowen.com>.
On Sat, 15 Jun 2002, Joshua Slive wrote:
> rbowen@apache.org wrote:
> > rbowen 2002/06/15 13:09:08
> >
> > Modified: docs/manual/vhosts fd-limits.html.en
> > Log:
> > Added configuration example, code example, and a little explanation, to
> > facilitate logging all of your virtual hosts to a single file, and then
> > splitting them back up after. Note that if Apache has problems with this
> > many file handles, Perl might also. I'm not sure.
>
> Note that there is a support script included with apache that does this:
> http://cvs.apache.org/viewcvs.cgi/*checkout*/httpd-2.0/support/split-logfile.in?rev=1.2
> or
> http://httpd.apache.org/docs-2.0/programs/other.html
I have *never* noticed that. And I was going to recommend that this
script be included in the bin directory so that people could just look
there, rather than having a full listing in the docs. I guess that's not
necessary. Recommendations? Should I just direct people to that instead?
> It is fairly similar to yours, but not quite the same. Both your script
> and the one in httpd-2.0 are missing a necessary security fix from the
> 1.3 version (strip slashes from the vhost name).
Can you elaborate on that? Why would the vhost name ever have a slash in
it? I can see that it could be a security problem, but how would one
ever get in there?
--
Pilgrim, how you journey on the road you chose
To find out where the winds die and where the stories go
--Pilgrim (Enya - A Day Without Rain)
---------------------------------------------------------------------
To unsubscribe, e-mail: docs-unsubscribe@httpd.apache.org
For additional commands, e-mail: docs-help@httpd.apache.org
Re: cvs commit: httpd-2.0/docs/manual/vhosts fd-limits.html.en
Posted by Joshua Slive <jo...@slive.ca>.
rbowen@apache.org wrote:
> rbowen 2002/06/15 13:09:08
>
> Modified: docs/manual/vhosts fd-limits.html.en
> Log:
> Added configuration example, code example, and a little explanation, to
> facilitate logging all of your virtual hosts to a single file, and then
> splitting them back up after. Note that if Apache has problems with this
> many file handles, Perl might also. I'm not sure.
Note that there is a support script included with apache that does this:
http://cvs.apache.org/viewcvs.cgi/*checkout*/httpd-2.0/support/split-logfile.in?rev=1.2
or
http://httpd.apache.org/docs-2.0/programs/other.html
It is fairly similar to yours, but not quite the same. Both your script
and the one in httpd-2.0 are missing a necessary security fix from the
1.3 version (strip slashes from the vhost name).
Joshua.
---------------------------------------------------------------------
To unsubscribe, e-mail: docs-unsubscribe@httpd.apache.org
For additional commands, e-mail: docs-help@httpd.apache.org