You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Corp <ec...@corp.electracide.net> on 2003/01/09 22:49:24 UTC
[users@httpd] Question on bin/rotatelogs
Hi all,
I need some help on rotatelogs(or maybe of some other solution
others may have?). I want to be able to log each days hits into a seprate
log file. Right now rotatelogs only logs a 24hour period into a log(and
not as daily).
I have:
CustomLog "|/path/to/rotatelogs /path/to/$log 86400" common
So, this will log a 24hour period into that file. What I want to be able
to do, if possible, is only log one days hits into one log file(example,
only 09/Jan/2003. The on midnight start to log to a new file for
10/Jan/2003).
Is this doable some how with rotatelogs?
Thanks!
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] cronolog is my solution! Re: [users@httpd] Question on bin/rotatelogs
Posted by Corp <ec...@corp.electracide.net>.
Just wanted to let others who might have the same requirements as me that
cronolog works perfectly!
CustomLog "|/path/to/cronolog /path/to/$log.%Y%m%d" custom
And it creates a new log file daily(I had to mess with dates on my test
boxes as I couldnt wait another 5 hours for it to hit midnight :) ).
Awesome.
On Thu, 9 Jan 2003, Corp wrote:
> On Thu, 9 Jan 2003, Jacob Coby wrote:
>
> > > I should have specified this was needed for hundreds of servers.
> >
> > Umm, yeah :-)
> >
> > Are these webservers clustered or can you combine all of their logs into one
> > big log and get meaningful data? If so, mod_log_spread might do what you
> > want.
> >
> > http://www.backhand.org/mod_log_spread/
>
> These arent clustered. Some are behind Foundry SI's, others are directly
> on the 'net. And we can't combime the logs due to the the requirements on
> some of the servers(the logging is different based on the type of
> server; Plus we only log hits to our mod_perl server, and not the proxy
> redirects(each server runs a proxy on port80, and a modperl server on the
> back end)).
>
> That module does look very interesting, but wont work for me.
>
> >
> > > So, what I was hopefully looking for was something I can pipe the log into
> > > so it can log a days hits, then automagically log the next days hits into
> > > another log file. This will do away with my custom rotate script, and the
> > > random sleep script, and a script which grabs a days hits.
> >
> > If the above doesn't help..
> >
> > I don't know of anything pre-written that will do this. In less than an
> > hour you could write a script or a C-program that does exactly what you
> > want. As far as I can tell, the child processes send log entries back to
> > the master process, which either pipes them, in order, to an external
> > program (ie rotatelogs), or out to a file. All you need to do is open
> > "access_log.mm-dd-YYYY" in append mode and write to it everything you get
> > from stdin :)
> >
> > I'm pretty sure that mod_log_spread is configurable to log files from host a
> > to be logged to a different file than from host b. Combine this with a
> > midnight cron job, and you've come pretty close to your goal. You should
> > (hopefully) be able to stutter spreadlogd long enough to get your logs
> > rotated.
> >
> > mod_log_spread would make sysadmin 1000x easier than managing individual
> > logs.
>
> I agree one large file would most likely be more easily managed, but ..
>
> Anyway, I guess I do indeed need to write something up, or give a shot to
> cronolog which Ive seen some posts mention in the mailing list
> archives(btw, if anyone has experience with cronolog please let me know of
> any good/bad experiences and any gotchas with it).
>
> I was hoping to look for something that has been tried and tested, but I
> guess right now my options are something custom or cronolog.
>
> Thanks for the help.
>
> > > -Jacob
> >
> >
> > ---------------------------------------------------------------------
> > The official User-To-User support forum of the Apache HTTP Server Project.
> > See <URL:http://httpd.apache.org/userslist.html> for more info.
> > To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> > " from the digest: users-digest-unsubscribe@httpd.apache.org
> > For additional commands, e-mail: users-help@httpd.apache.org
> >
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Question on bin/rotatelogs
Posted by Corp <ec...@corp.electracide.net>.
On Thu, 9 Jan 2003, Jacob Coby wrote:
> > I should have specified this was needed for hundreds of servers.
>
> Umm, yeah :-)
>
> Are these webservers clustered or can you combine all of their logs into one
> big log and get meaningful data? If so, mod_log_spread might do what you
> want.
>
> http://www.backhand.org/mod_log_spread/
These arent clustered. Some are behind Foundry SI's, others are directly
on the 'net. And we can't combime the logs due to the the requirements on
some of the servers(the logging is different based on the type of
server; Plus we only log hits to our mod_perl server, and not the proxy
redirects(each server runs a proxy on port80, and a modperl server on the
back end)).
That module does look very interesting, but wont work for me.
>
> > So, what I was hopefully looking for was something I can pipe the log into
> > so it can log a days hits, then automagically log the next days hits into
> > another log file. This will do away with my custom rotate script, and the
> > random sleep script, and a script which grabs a days hits.
>
> If the above doesn't help..
>
> I don't know of anything pre-written that will do this. In less than an
> hour you could write a script or a C-program that does exactly what you
> want. As far as I can tell, the child processes send log entries back to
> the master process, which either pipes them, in order, to an external
> program (ie rotatelogs), or out to a file. All you need to do is open
> "access_log.mm-dd-YYYY" in append mode and write to it everything you get
> from stdin :)
>
> I'm pretty sure that mod_log_spread is configurable to log files from host a
> to be logged to a different file than from host b. Combine this with a
> midnight cron job, and you've come pretty close to your goal. You should
> (hopefully) be able to stutter spreadlogd long enough to get your logs
> rotated.
>
> mod_log_spread would make sysadmin 1000x easier than managing individual
> logs.
I agree one large file would most likely be more easily managed, but ..
Anyway, I guess I do indeed need to write something up, or give a shot to
cronolog which Ive seen some posts mention in the mailing list
archives(btw, if anyone has experience with cronolog please let me know of
any good/bad experiences and any gotchas with it).
I was hoping to look for something that has been tried and tested, but I
guess right now my options are something custom or cronolog.
Thanks for the help.
> > -Jacob
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Question on bin/rotatelogs
Posted by Jacob Coby <jc...@listingbook.com>.
> I should have specified this was needed for hundreds of servers.
Umm, yeah :-)
Are these webservers clustered or can you combine all of their logs into one
big log and get meaningful data? If so, mod_log_spread might do what you
want.
http://www.backhand.org/mod_log_spread/
> So, what I was hopefully looking for was something I can pipe the log into
> so it can log a days hits, then automagically log the next days hits into
> another log file. This will do away with my custom rotate script, and the
> random sleep script, and a script which grabs a days hits.
If the above doesn't help..
I don't know of anything pre-written that will do this. In less than an
hour you could write a script or a C-program that does exactly what you
want. As far as I can tell, the child processes send log entries back to
the master process, which either pipes them, in order, to an external
program (ie rotatelogs), or out to a file. All you need to do is open
"access_log.mm-dd-YYYY" in append mode and write to it everything you get
from stdin :)
I'm pretty sure that mod_log_spread is configurable to log files from host a
to be logged to a different file than from host b. Combine this with a
midnight cron job, and you've come pretty close to your goal. You should
(hopefully) be able to stutter spreadlogd long enough to get your logs
rotated.
mod_log_spread would make sysadmin 1000x easier than managing individual
logs.
-Jacob
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Question on bin/rotatelogs
Posted by Corp <ec...@corp.electracide.net>.
I should have specified this was needed for hundreds of servers. So, I
cant use logrotate/cron to do this. I do use in cron currently a script
which runs before anything else and sleeps for a random period of time
then runs a custom rotation script I have. Problem is I cant have each and
every web server to be bounced at the same time :) And the reason why I
want to move away from cron and my custom rotate script is that then I
also need to grep out the prior days hits(since recall I cant bounce each
server at the same time, so there are bounces that happen at 6am in the
morning).
So, what I was hopefully looking for was something I can pipe the log into
so it can log a days hits, then automagically log the next days hits into
another log file. This will do away with my custom rotate script, and the
random sleep script, and a script which grabs a days hits.
Any ideas?
THanks.
On Thu, 9 Jan 2003, Jacob Coby wrote:
>
> > So, this will log a 24hour period into that file. What I want to be able
> > to do, if possible, is only log one days hits into one log file(example,
> > only 09/Jan/2003. The on midnight start to log to a new file for
> > 10/Jan/2003).
>
> Couple of options:
>
> 1) use logrotate(8) along with a cron that runs at midnight. Only thing you
> need to remember here is that post_rotate you need to sig -HUP to reset
> apache, and that whatever_log.1 will be the previous 24hr's log.
>
> 2) use a script to cp your log file to log.`date -I`, and then cat /dev/null
> > log file to reset it.
>
> 1) is more reliable
> 2) doesn't reset apache and cause your uptime to show < 24hrs or cause
> anything to actually reload. You will lose whatever events happen between
> the cp and the cat /dev/null (maybe one or two lines)
>
> -Jacob
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Question on bin/rotatelogs
Posted by Jacob Coby <jc...@listingbook.com>.
> So, this will log a 24hour period into that file. What I want to be able
> to do, if possible, is only log one days hits into one log file(example,
> only 09/Jan/2003. The on midnight start to log to a new file for
> 10/Jan/2003).
Couple of options:
1) use logrotate(8) along with a cron that runs at midnight. Only thing you
need to remember here is that post_rotate you need to sig -HUP to reset
apache, and that whatever_log.1 will be the previous 24hr's log.
2) use a script to cp your log file to log.`date -I`, and then cat /dev/null
> log file to reset it.
1) is more reliable
2) doesn't reset apache and cause your uptime to show < 24hrs or cause
anything to actually reload. You will lose whatever events happen between
the cp and the cat /dev/null (maybe one or two lines)
-Jacob
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org