You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Rodent of Unusual Size <CO...@PROCESS.COM> on 1997/11/03 15:48:00 UTC
Aha! The conditional logging beast rears its head again
There was a discussion a while ago (July/August, I think) about
adding conditionalisation to mod_log_config, so the configuration
could direct which requests should be logged and which not (a la
mod_log_referer). The discussion was happening just when 1.3
feature-freeze was starting to settle in. The basic disagreement,
as I remember, was that some people (those with big systems and big
disks, I think) felt it's more appropriate for such filtering to be
an after-process, or done by some inline filter (now that reliable
piped logs are here). Personally, I still prefer to have it in the
server itself and not waste the cycles nor bytes trying to winnow
through a "log-of-everything" file. It's less of an issue with a
piped filter, but I'd *still* prefer to see it in the server -
no-one says people *have* to use it.
PR #1351 is asking for this, so be warned - I may open the
discussion once again.. ;->
#ken P-)}
Re: Aha! The conditional logging beast rears its head again
Posted by Marc Slemko <ma...@worldgate.com>.
There is already (at least) one PR already open on this before today. eg.
PR#548.
On Mon, 3 Nov 1997, Rodent of Unusual Size wrote:
> There was a discussion a while ago (July/August, I think) about
> adding conditionalisation to mod_log_config, so the configuration
> could direct which requests should be logged and which not (a la
> mod_log_referer). The discussion was happening just when 1.3
> feature-freeze was starting to settle in. The basic disagreement,
> as I remember, was that some people (those with big systems and big
> disks, I think) felt it's more appropriate for such filtering to be
> an after-process, or done by some inline filter (now that reliable
> piped logs are here). Personally, I still prefer to have it in the
> server itself and not waste the cycles nor bytes trying to winnow
> through a "log-of-everything" file. It's less of an issue with a
> piped filter, but I'd *still* prefer to see it in the server -
> no-one says people *have* to use it.
>
> PR #1351 is asking for this, so be warned - I may open the
> discussion once again.. ;->
>
> #ken P-)}
>
Re: Aha! The conditional logging beast rears its head again
Posted by Brian Behlendorf <br...@organic.com>.
I'll reiterate my opposition, for what it's worth. This is taking us in an
opposite direction from where I think we should be headed. If Dean's
example of piping through grep and >> is not fine for NT, then a program
which accepts input on stdin could easily be written. The example of a
module setting a flag to prevent logging could be handled by stuffing a
marker in r->notes and "logging" notes, with the script looking for that
marker. No reason we can't include that logger program in /support or
/contrib. No reason we can't let it work for Apache too. However, the
comment "you don't have to use it" is false; if it's integrated into the
core or the standard mod_log_config, we do have to use the code, even if we
don't use that functionality.
I'm really really not trying to be an asshole about this, honest; I am just
very weary of complexity and featuritis.
Brian
At 10:48 AM 11/3/97 -0400, Rodent of Unusual Size wrote:
> There was a discussion a while ago (July/August, I think) about
> adding conditionalisation to mod_log_config, so the configuration
> could direct which requests should be logged and which not (a la
> mod_log_referer). The discussion was happening just when 1.3
> feature-freeze was starting to settle in. The basic disagreement,
> as I remember, was that some people (those with big systems and big
> disks, I think) felt it's more appropriate for such filtering to be
> an after-process, or done by some inline filter (now that reliable
> piped logs are here). Personally, I still prefer to have it in the
> server itself and not waste the cycles nor bytes trying to winnow
> through a "log-of-everything" file. It's less of an issue with a
> piped filter, but I'd *still* prefer to see it in the server -
> no-one says people *have* to use it.
>
> PR #1351 is asking for this, so be warned - I may open the
> discussion once again.. ;->
>
> #ken P-)}
>
--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
"it's a big world, with lots of records to play." - sig brian@organic.com
Re: Aha! The conditional logging beast rears its head again
Posted by Elizabeth Mattijsen <li...@xxLINK.nl>.
At 13:56 3-11-97 -0700, Marc Slemko wrote:
>I still like the idea of having an variable that modules could set to stop
>mod_log_config from logging an entry. You could then do an awful lot with
>mod_rewrite. Is there anything unworkable in that idea?
Actually, that would be nice. Or to be able to log directly from
mod_rewrite. That would be nicer. But mod_perl would probably be a better
solution for that kind of thing.
(no, I'm not voting ;-)
Elizabeth Mattijsen
xxLINK Internet Services
Re: Aha! The conditional logging beast rears its head again
Posted by Dean Gaudet <dg...@arctic.org>.
On Mon, 3 Nov 1997, Ben Laurie wrote:
> Dean Gaudet wrote:
> >
> > Just because users ask for a feature doesn't mean we have to implement it
> > the way they ask for it. You know my opinion on this topic. For example,
> > here we go, here's conditional logging for 1.3:
> >
> > TransferLog "| pidthing /path/to/pidfile grep -v 'do your best' >>/path/to/access_log"
> >
> > And here's an easier way to rotate logs for 1.3 without doing a full
> > server restar:
> >
> > TransferLog "| pidthing /path/to/pidfile cat >>/path/to/access_log"
> >
> > Where pidthing is the following sh script:
> >
> > #!/bin/sh
> > if [ $# -lt 2 ]; then
> > echo "usage: $0 pidfile_path program_to_exec [args ...]" 1>&2
> > exit 1
> > fi
> > if ! echo $$ > $1; then
> > echo "can't write pidfile" 1>&2
> > exit 1
> > fi
> > shift
> > exec ${1+"$@"}
> > echo "exec failed" 1>&2
> > exit 1
> >
> > Impressive, no? The unix philosophy at its best.
>
> So impressive I don't understand it :-)
>
> Actually, I presume the idea is you kill the cat from time to time?
> Perhaps the script should be called "schroedinger"? :-)
Yup, the pidthing writes the pid of the cat/grep/whatever to a file, and
you can do the usual "kill `cat logger.pid`" to rotate logs.
Dean
Re: Aha! The conditional logging beast rears its head again
Posted by Ben Laurie <be...@algroup.co.uk>.
Dean Gaudet wrote:
>
> Just because users ask for a feature doesn't mean we have to implement it
> the way they ask for it. You know my opinion on this topic. For example,
> here we go, here's conditional logging for 1.3:
>
> TransferLog "| pidthing /path/to/pidfile grep -v 'do your best' >>/path/to/access_log"
>
> And here's an easier way to rotate logs for 1.3 without doing a full
> server restar:
>
> TransferLog "| pidthing /path/to/pidfile cat >>/path/to/access_log"
>
> Where pidthing is the following sh script:
>
> #!/bin/sh
> if [ $# -lt 2 ]; then
> echo "usage: $0 pidfile_path program_to_exec [args ...]" 1>&2
> exit 1
> fi
> if ! echo $$ > $1; then
> echo "can't write pidfile" 1>&2
> exit 1
> fi
> shift
> exec ${1+"$@"}
> echo "exec failed" 1>&2
> exit 1
>
> Impressive, no? The unix philosophy at its best.
So impressive I don't understand it :-)
Actually, I presume the idea is you kill the cat from time to time?
Perhaps the script should be called "schroedinger"? :-)
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686|Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org
and Technical Director|Email: ben@algroup.co.uk |Apache-SSL author
A.L. Digital Ltd, |http://www.algroup.co.uk/Apache-SSL
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache
Re: Aha! The conditional logging beast rears its head again
Posted by Marc Slemko <ma...@worldgate.com>.
On Mon, 3 Nov 1997, Dean Gaudet wrote:
> Yeah whatever, I'm not going to veto it since you all seem to want it. I
> just think that down this path lies feature bloat, and is more of a
> "windows" philosophy than a unix philosophy. What if one of our modules
So, do piped logs work reliably on NT? <g> You also have to consider
that the average NT installation does things the Windows way, so the user
doesn't have a choice of just piping it to grep. It is either have it
built into the server, or download twelve zillion little things and try to
make them work.
I agree that for many things they are better, however many people just
don't want that.
Re: Aha! The conditional logging beast rears its head again
Posted by Dean Gaudet <dg...@arctic.org>.
Yeah whatever, I'm not going to veto it since you all seem to want it. I
just think that down this path lies feature bloat, and is more of a
"windows" philosophy than a unix philosophy. What if one of our modules
isn't powerful enough to set this variable in some case? Folks will still
be asking us for more features for it. I also question folks ability to
understand mod_rewrite ;)
BTW, if you're considering an env variable for this, which is the natural
thing to use, it's debatable how much faster it'll be... remember env vars
are looked up using a linear lookup, and linear string comparisons.
(Something I meant to profile the effect of but never got around to.)
Piped logs solve the log rotation problem much better than USR1 or HUP.
Piped logs solve the HostnameLookups problem much better than Apache can
do on its own.
Piped logs solve the N-zillion filehandles problem for vhost access_logs
much better than we can in Apache itself. Note that error_log remains
unfixed by this, something that I would like to see fixed, but which
requires some sophisticated changes to how CGIs are run, we can't just
give them a stderr attached to an error_log. Erm. Actually we could let
suexec handle stderr for CGIs... hrm. Maybe this is easier than I
thought.
The only thing missing is a featureful logger.
Dean
On Mon, 3 Nov 1997, Marc Slemko wrote:
> I still like the idea of having an variable that modules could set to stop
> mod_log_config from logging an entry. You could then do an awful lot with
> mod_rewrite. Is there anything unworkable in that idea?
>
> It is simple, extensible (ie. other modules can set that variable however
> the heck they want), and far nicer for simple things than logging to a
> process.
>
> On Mon, 3 Nov 1997, Dean Gaudet wrote:
>
> > Just because users ask for a feature doesn't mean we have to implement it
> > the way they ask for it. You know my opinion on this topic. For example,
> > here we go, here's conditional logging for 1.3:
> >
> > TransferLog "| pidthing /path/to/pidfile grep -v 'do your best' >>/path/to/access_log"
> >
> > And here's an easier way to rotate logs for 1.3 without doing a full
> > server restar:
> >
> > TransferLog "| pidthing /path/to/pidfile cat >>/path/to/access_log"
> >
> > Where pidthing is the following sh script:
> >
> > #!/bin/sh
> > if [ $# -lt 2 ]; then
> > echo "usage: $0 pidfile_path program_to_exec [args ...]" 1>&2
> > exit 1
> > fi
> > if ! echo $$ > $1; then
> > echo "can't write pidfile" 1>&2
> > exit 1
> > fi
> > shift
> > exec ${1+"$@"}
> > echo "exec failed" 1>&2
> > exit 1
> >
> > Impressive, no? The unix philosophy at its best.
> >
> > Dean
> >
> > On Mon, 3 Nov 1997, Rodent of Unusual Size wrote:
> >
> > > There was a discussion a while ago (July/August, I think) about
> > > adding conditionalisation to mod_log_config, so the configuration
> > > could direct which requests should be logged and which not (a la
> > > mod_log_referer). The discussion was happening just when 1.3
> > > feature-freeze was starting to settle in. The basic disagreement,
> > > as I remember, was that some people (those with big systems and big
> > > disks, I think) felt it's more appropriate for such filtering to be
> > > an after-process, or done by some inline filter (now that reliable
> > > piped logs are here). Personally, I still prefer to have it in the
> > > server itself and not waste the cycles nor bytes trying to winnow
> > > through a "log-of-everything" file. It's less of an issue with a
> > > piped filter, but I'd *still* prefer to see it in the server -
> > > no-one says people *have* to use it.
> > >
> > > PR #1351 is asking for this, so be warned - I may open the
> > > discussion once again.. ;->
> > >
> > > #ken P-)}
> > >
> >
>
>
Re: Aha! The conditional logging beast rears its head again
Posted by Marc Slemko <ma...@worldgate.com>.
I still like the idea of having an variable that modules could set to stop
mod_log_config from logging an entry. You could then do an awful lot with
mod_rewrite. Is there anything unworkable in that idea?
It is simple, extensible (ie. other modules can set that variable however
the heck they want), and far nicer for simple things than logging to a
process.
On Mon, 3 Nov 1997, Dean Gaudet wrote:
> Just because users ask for a feature doesn't mean we have to implement it
> the way they ask for it. You know my opinion on this topic. For example,
> here we go, here's conditional logging for 1.3:
>
> TransferLog "| pidthing /path/to/pidfile grep -v 'do your best' >>/path/to/access_log"
>
> And here's an easier way to rotate logs for 1.3 without doing a full
> server restar:
>
> TransferLog "| pidthing /path/to/pidfile cat >>/path/to/access_log"
>
> Where pidthing is the following sh script:
>
> #!/bin/sh
> if [ $# -lt 2 ]; then
> echo "usage: $0 pidfile_path program_to_exec [args ...]" 1>&2
> exit 1
> fi
> if ! echo $$ > $1; then
> echo "can't write pidfile" 1>&2
> exit 1
> fi
> shift
> exec ${1+"$@"}
> echo "exec failed" 1>&2
> exit 1
>
> Impressive, no? The unix philosophy at its best.
>
> Dean
>
> On Mon, 3 Nov 1997, Rodent of Unusual Size wrote:
>
> > There was a discussion a while ago (July/August, I think) about
> > adding conditionalisation to mod_log_config, so the configuration
> > could direct which requests should be logged and which not (a la
> > mod_log_referer). The discussion was happening just when 1.3
> > feature-freeze was starting to settle in. The basic disagreement,
> > as I remember, was that some people (those with big systems and big
> > disks, I think) felt it's more appropriate for such filtering to be
> > an after-process, or done by some inline filter (now that reliable
> > piped logs are here). Personally, I still prefer to have it in the
> > server itself and not waste the cycles nor bytes trying to winnow
> > through a "log-of-everything" file. It's less of an issue with a
> > piped filter, but I'd *still* prefer to see it in the server -
> > no-one says people *have* to use it.
> >
> > PR #1351 is asking for this, so be warned - I may open the
> > discussion once again.. ;->
> >
> > #ken P-)}
> >
>
Re: Aha! The conditional logging beast rears its head again
Posted by Dean Gaudet <dg...@arctic.org>.
Just because users ask for a feature doesn't mean we have to implement it
the way they ask for it. You know my opinion on this topic. For example,
here we go, here's conditional logging for 1.3:
TransferLog "| pidthing /path/to/pidfile grep -v 'do your best' >>/path/to/access_log"
And here's an easier way to rotate logs for 1.3 without doing a full
server restar:
TransferLog "| pidthing /path/to/pidfile cat >>/path/to/access_log"
Where pidthing is the following sh script:
#!/bin/sh
if [ $# -lt 2 ]; then
echo "usage: $0 pidfile_path program_to_exec [args ...]" 1>&2
exit 1
fi
if ! echo $$ > $1; then
echo "can't write pidfile" 1>&2
exit 1
fi
shift
exec ${1+"$@"}
echo "exec failed" 1>&2
exit 1
Impressive, no? The unix philosophy at its best.
Dean
On Mon, 3 Nov 1997, Rodent of Unusual Size wrote:
> There was a discussion a while ago (July/August, I think) about
> adding conditionalisation to mod_log_config, so the configuration
> could direct which requests should be logged and which not (a la
> mod_log_referer). The discussion was happening just when 1.3
> feature-freeze was starting to settle in. The basic disagreement,
> as I remember, was that some people (those with big systems and big
> disks, I think) felt it's more appropriate for such filtering to be
> an after-process, or done by some inline filter (now that reliable
> piped logs are here). Personally, I still prefer to have it in the
> server itself and not waste the cycles nor bytes trying to winnow
> through a "log-of-everything" file. It's less of an issue with a
> piped filter, but I'd *still* prefer to see it in the server -
> no-one says people *have* to use it.
>
> PR #1351 is asking for this, so be warned - I may open the
> discussion once again.. ;->
>
> #ken P-)}
>