You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by "Ralf S. Engelschall" <rs...@engelschall.com> on 1998/03/05 16:00:29 UTC
[PATCH] Network Socket Support for Logfiles
Network Socket Support for Logfiles
-----------------------------------
Background:
In July 1997 Stanley Gambarin <st...@cs.bu.edu> posted a short patch for
the http_log.c file for hacking in the ability to use the syntax
hostname:port for the filename argument of ErrorLog which leads to the open
of a socket to this target. This is a very useful variant of writing out log
information, especially for ISPs or large website providers who use a
cluster of webservers. Here all Apache servers just log their information to
a master machine, providing you the ability to monitor all websites at a
single point.
Problem of the past:
Although this variant of logging is very useful for a lot of people, we
didn't incorporated it, because the patch was not very complete. The
original patch from Stanley directly changes the open_error_log() function
in src/main/http_log.c which leads to the fact that network sockets could
only be used for the ErrorLog directive. And it didn't documented the new
possible variant.
Solution:
I've started from scratch today and first wrote a open_socket_log()
function in addition to the already existing open_piped_log() function.
Then I've just _minimally_ patched mod_log_config.c and the open_error_log()
function in http_log.c to use this new open_socket_log() function. Finally
the corresponding documentation changes are provided to document this
variant of writing logfiles.
Effect:
For instance when you now have a tool like "socket" or "netcat" installed
(FreeBSD and Linux hackers usually have out-of-the-box), you just use
$ socket -f -l -s 8000 >>/path/to/master/access_log &
$ socket -f -l -s 8001 >>/path/to/master/error_log &
on the monitoring site `mastersite' and
TransferLog @mastersite:8000
ErrorLog @mastersite:8001
on all Apache's on your cluster and then magically all log informations of
the whole cluster is collected at the master site.
BTW: This is also very useful when using Apache in Reverse Proxy context.
Both for writing the Reverse Proxy delegation logfile to the monitoring
site via
CustomLog @master:8000 "%{%v/%T}t %h -> %{SERVER}e URL: %U"
and to collect the logfiles of the various back-end servers as
described above.
Ralf S. Engelschall
rse@engelschall.com
www.engelschall.com
Index: src/CHANGES
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/src/CHANGES,v
retrieving revision 1.687
diff -u -r1.687 CHANGES
--- CHANGES 1998/03/04 14:28:25 1.687
+++ CHANGES 1998/03/05 14:36:10
@@ -1,5 +1,12 @@
Changes with Apache 1.3b6
+ *) Add the ability to write logfile entries to network sockets in addition
+ to local files and pipes for ErrorLog (core) and TransferLog/CustomLog
+ (mod_log_config). This is intended to be used either for clusters of
+ webservers where one wants to store all error and transfer log
+ information on a single master site for monitoring or just to on-the-fly
+ transfer away log information. [Ralf S. Engelschall]
+
*) Fix mod_rewrite for the ugly API case where <VirtualHost> sections exist
but without any RewriteXXXXX directives. Here mod_rewrite is given no
chance by the API to initialise its per-server configuration and thus
Index: src/include/http_log.h
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/src/include/http_log.h,v
retrieving revision 1.25
diff -u -r1.25 http_log.h
--- http_log.h 1998/01/21 19:17:38 1.25
+++ http_log.h 1998/03/05 13:50:43
@@ -132,4 +132,6 @@
#define piped_log_write_fd(pl) (fileno((pl)->write_f))
#endif
+API_EXPORT(int) open_socket_log(char *target);
+
#endif /* !APACHE_HTTP_LOG_H */
Index: src/main/http_log.c
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/src/main/http_log.c,v
retrieving revision 1.48
diff -u -r1.48 http_log.c
--- http_log.c 1998/01/07 16:46:05 1.48
+++ http_log.c 1998/03/05 14:37:07
@@ -213,6 +213,11 @@
s->error_log = NULL;
}
#endif
+
+ else if (*s->error_fname == '@' && strchr(s->error_fname, ':') != NULL) {
+ s->error_log = fdopen(open_socket_log(s->error_fname+1), "w");
+ }
+
else {
fname = server_root_relative (p, s->error_fname);
if(!(s->error_log = pfopen(p, fname, "a"))) {
@@ -638,3 +643,56 @@
pfclose (pl->p, pl->write_f);
}
#endif
+
+
+API_EXPORT(int) open_socket_log(char *target)
+{
+ char host[MAX_STRING_LEN];
+ char *port;
+ struct hostent *he;
+ struct protoent *pe;
+ struct sockaddr_in sar;
+ FILE *fp;
+ int s;
+
+ /* parse host:port pair */
+ ap_cpystrn(host, target, sizeof(host));
+ if ((port = strchr(host, ':')) == NULL) {
+ fprintf(stderr, "httpd: Invalid socket target `%s'\n", target);
+ exit(1);
+ }
+ *port++ = '\0';
+
+ /* create socket address structure */
+ memset((char *)&sar, 0, sizeof(sar));
+ sar.sin_family = AF_INET;
+ sar.sin_port = htons(atoi(port));
+ if ((sar.sin_addr.s_addr = inet_addr(host)) == INADDR_NONE) {
+ if ((he = gethostbyname(host)) == NULL) {
+ perror("gethostbyname");
+ fprintf(stderr, "httpd: Couldn't resolve hostname `%s'\n", host);
+ exit(1);
+ }
+ sar.sin_addr.s_addr = *((u_long *)(he->h_addr_list[0]));
+ }
+
+ /* create the socket and connect to it */
+ if ((pe = getprotobyname("tcp")) == NULL) {
+ perror("getprotobyname");
+ fprintf(stderr, "httpd: Couldn't resolve TCP protocol info\n");
+ exit(1);
+ }
+ if ((s = socket(AF_INET, SOCK_STREAM, pe->p_proto)) < 0) {
+ perror("socket");
+ fprintf(stderr, "httpd: Couldn't create socket\n");
+ exit(1);
+ }
+ if (connect(s, (struct sockaddr *)&sar, sizeof(sar)) < 0) {
+ perror("connect");
+ fprintf(stderr, "httpd: Couldn't connect to %s\n", target);
+ exit(1);
+ }
+
+ return s;
+}
+
Index: src/modules/standard/mod_log_config.c
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/src/modules/standard/mod_log_config.c,v
retrieving revision 1.47
diff -u -r1.47 mod_log_config.c
--- mod_log_config.c 1998/03/03 08:31:28 1.47
+++ mod_log_config.c 1998/03/05 14:21:35
@@ -912,6 +912,9 @@
}
cls->log_fd = piped_log_write_fd(pl);
}
+ else if (*cls->fname == '@' && strchr(cls->fname, ':') != NULL) {
+ cls->log_fd = open_socket_log(cls->fname+1);
+ }
else {
char *fname = server_root_relative(p, cls->fname);
if ((cls->log_fd = popenf(p, fname, xfer_flags, xfer_mode)) < 0) {
Index: htdocs/manual/mod/core.html
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/htdocs/manual/mod/core.html,v
retrieving revision 1.103
diff -u -r1.103 core.html
--- core.html 1998/02/20 06:52:03 1.103
+++ core.html 1998/03/05 14:27:26
@@ -775,6 +775,8 @@
then it is assumed to be relative to the <A HREF="#serverroot">ServerRoot</A>.
If the filename begins with a pipe (|) then it is assumed to be a command to
spawn to handle the error log.
+If the filename begins with an at-sign (@) then it is assumed to be followed
+by a <CODE>hostname:port</CODE> pair for opening a socket.
<p>
Example:
<BLOCKQUOTE><CODE>ErrorLog /dev/null</CODE></BLOCKQUOTE>
Index: htdocs/manual/mod/mod_log_config.html
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/htdocs/manual/mod/mod_log_config.html,v
retrieving revision 1.26
diff -u -r1.26 mod_log_config.html
--- mod_log_config.html 1998/03/03 08:38:26 1.26
+++ mod_log_config.html 1998/03/05 14:26:24
@@ -365,6 +365,8 @@
<DD>A program to receive the agent log information on its standard input.
Note the a new program will not be started for a VirtualHost if it inherits
the TransferLog from the main server.
+<DT> `@' followed by a hostname:port pair
+<DD>A socket to receive the log information.
</DL>
<STRONG>Security:</STRONG> if a program is used, then it will be
run under the user who started httpd. This will be root if the server
Re: [PATCH] Network Socket Support for Logfiles
Posted by Dean Gaudet <dg...@arctic.org>.
FEATURITIS! Piped logs let you do all of this already. This is the same
complaint I had about adding syslog support.
-1 until you prove why using "socket" or "netcat" is insufficient.
Dean