DO NOT REPLY [Bug 29190] New: - Error logging per vhost consuming all available file descriptors

[bu...@apache.org]

DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG 
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=29190>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND 
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=29190

Error logging per vhost consuming all available file descriptors

           Summary: Error logging per vhost consuming all available file
                    descriptors
           Product: Apache httpd-2.0
           Version: 2.0.49
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: Enhancement
          Priority: Other
         Component: Core
        AssignedTo: bugs@httpd.apache.org
        ReportedBy: kieran@digital-crocus.com


This is more of a feature request than a bug report but let me first explain 
the problem:

When hosting large numbers of virtual hosts with Apache, having an ErrorLog 
directive for each vhost quickly consumes all available system FDs. Increasing 
operating system limits is a *workaround*, not a solution. With per-vhost 
access logs it's possible to specify %V in a global LogFormat directive and 
then log all requests to one file (or pipe) for later separation. This is not 
possible with ErrorLogs because they do not come under the control of 
mod_log_config.

Running a web hosting provider on Apache, I have the problem that I want to 
allow my users to view error logs for script debugging purposes, but I do not 
want them to be able to see eachother's error lines for privacy and security 
reasons. The solution to this problem would be to enter an ErrorLog directive 
for each vhost as mentioned, except that this consumes all my system's 
available file desciptors. 

One solution would be to add a config option that would tell Apache to close 
each error log when it's done writing a line to it, and re-open it next time 
it needs to do a write. This may be expensive for access logs which typically 
receive hundreds of new lines a second, but less so for error logs because 
errors should only occur in exceptional circumstances. However, this solution 
is a bit of a kludge and (?) might make the server vulnerable to DoS attacks 
by clients sending hundreds of requests that produce an error.

A better solution, and the solution I've hacked into my own web server is to 
somehow allow prepending of vhost information to each error log line. This 
would allow one global error log for all vhosts to be used which could then be 
easily separated out at a later date (or via a logging pipe). 

I've hacked server/log.c in the Apache source to prepend vhost names (where 
available) to all error log lines (this is a quick fix rather than a 
solution). Anyway, below is a patch that implements this behaviour incase 
anyone else will find it useful and also to help illustrate the problem. 

As a sidenote, since implementing this I've noticed that not all error log 
lines are added via log_error_core(), in particular messages like the 
following are not logged via this function, but still appear in the globally 
defined ErrorLog file: "piped log 
program '/usr/local/httpd/vhostlogs/caching_logger' failed unexpectedly". I'm 
not sure if this is a bug or it's intentional, but I mention it here just 
incase.

Anyway, here's the patch:

--- log.c.orig  Mon May 24 21:32:24 2004
+++ log.c       Mon May 24 22:31:05 2004
@@ -417,15 +417,32 @@
         }
     }

-    if (logf && ((level & APLOG_STARTUP) != APLOG_STARTUP)) {
-        errstr[0] = '[';
-        ap_recent_ctime(errstr + 1, apr_time_now());
-        errstr[1 + APR_CTIME_LEN - 1] = ']';
-        errstr[1 + APR_CTIME_LEN    ] = ' ';
-        len = 1 + APR_CTIME_LEN + 1;
+    /* DC patch */
+
+
+    /* if we've got a s struct containing the vhost name */
+    if (s != NULL) {
+       /* add it to the beginning of the error string */
+       len = apr_snprintf(errstr, MAX_STRING_LEN, "%s ",
s->server_hostname);
     } else {
-        len = 0;
+       /* else, prepend -servererror- so that at least our piped logger
will
+          have a filename to use rather than just whatever the first field
+          happens to be */
+       len = apr_snprintf(errstr, MAX_STRING_LEN, "-servererror- ");
+    }
+
+    /* changed this code too so that it doesn't assume it's starting at the
+       beginning of the string (basically use pointer arithmatic to offset
+       the entire thing by len number of characters) */
+    if (logf && ((level & APLOG_STARTUP) != APLOG_STARTUP)) {
+        errstr[len++] = '[';
+        ap_recent_ctime(errstr + len, apr_time_now());
+        errstr[len + APR_CTIME_LEN - 1] = ']';
+        errstr[len + APR_CTIME_LEN    ] = ' ';
+        len = len + APR_CTIME_LEN + 1;
     }
+
+    /* end DC Patch */

     if ((level & APLOG_STARTUP) != APLOG_STARTUP) {
         len += apr_snprintf(errstr + len, MAX_STRING_LEN - len,

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org