Logging via rotatelog in a handler -- wise?

[Tofu Optimist <to...@yahoo.com>]

Hi -- 

I am a new user of MP2.  Pardon my basic questions.

Here's my situation:

My Apache generates 4 sets of logs.

Three of them are generated by Apache directly:
access, error, and a custom log tracking access for
certain file types.

The fourth is made by a MP2 handler dealing with
redirects.  Here's the snippet of relevant code.

sub mylog {
    my (@fields) = @_;
    my $entry = join ( "|", @fields );
    open my $fh, ">>" . &MYLOG or die "can't open " .
&MYLOG . ": $!";
    flock $fh, LOCK_EX;
    print $fh $entry . "\n";
    close $fh;
}

Currently I'm rolling my logs "manually" -- 
a process renames the four sets of logs, waits, then
restarts the server.

I am considering using piped logs
  http://httpd.apache.org/docs-2.0/logs.html#piped
with rotatelogs
 
http://httpd.apache.org/docs-2.0/programs/rotatelogs.html
instead to avoid the restart.

I understand how to instruct Apache to log via a pipe
to rotatelogs.  

My questions relate to the mylog() function:

Can I / should I direct output to a pipe?  
Would this do anything unpleasant to MP2 or AP2?

If doing this is OK, 
do I just open the pipe like a file?

<untested code>
open my $fh, "|bin/rotatelogs
/var/logs/%y_%m_%d_%H_%M_%S 14400" or die "can't open
pipe to rotatelogs: $!";
</untested code>

I'd remove the FLOCK, yes?

If I do this, should I close the pipe after use, as I
do now in mylog() with close $fh?

Are there any speed or stability concerns logging
through a rotatelog, vs. to a file?

Thanks very much for your advice and insights....

-TO





__________________________________
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html

-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html

Re: Logging via rotatelog in a handler -- wise?

[Stas Bekman <st...@stason.org>]

Tofu Optimist wrote:
> Hi -- 
> 
> I am a new user of MP2.  Pardon my basic questions.
> 
> Here's my situation:
> 
> My Apache generates 4 sets of logs.
> 
> Three of them are generated by Apache directly:
> access, error, and a custom log tracking access for
> certain file types.
> 
> The fourth is made by a MP2 handler dealing with
> redirects.  Here's the snippet of relevant code.
> 
> sub mylog {
>     my (@fields) = @_;
>     my $entry = join ( "|", @fields );
>     open my $fh, ">>" . &MYLOG or die "can't open " .
> &MYLOG . ": $!";
>     flock $fh, LOCK_EX;
>     print $fh $entry . "\n";
>     close $fh;
> }
> 
> Currently I'm rolling my logs "manually" -- 
> a process renames the four sets of logs, waits, then
> restarts the server.
> 
> I am considering using piped logs
>   http://httpd.apache.org/docs-2.0/logs.html#piped
> with rotatelogs
>  
> http://httpd.apache.org/docs-2.0/programs/rotatelogs.html
> instead to avoid the restart.
> 
> I understand how to instruct Apache to log via a pipe
> to rotatelogs.  
> 
> My questions relate to the mylog() function:
> 
> Can I / should I direct output to a pipe?  
> Would this do anything unpleasant to MP2 or AP2?

Not, unless you load the mod_soul module ;)

> If doing this is OK, 
> do I just open the pipe like a file?
> 
> <untested code>
> open my $fh, "|bin/rotatelogs
> /var/logs/%y_%m_%d_%H_%M_%S 14400" or die "can't open
> pipe to rotatelogs: $!";
> </untested code>

that's a *very* bad idea, as you will spawn a new process on each request. You 
need to open your log handler during the PerlOpenLogsHandler as explained here:
http://perl.apache.org/docs/2.0/user/handlers/server.html#Startup_Phases_Demonstration_Module
http://perl.apache.org/docs/2.0/user/handlers/server.html#PerlOpenLogsHandler
and then just log things at request time. So you will change the open call to 
log into the rotator.

> I'd remove the FLOCK, yes?

Several processes writing to the same file at the same time will mess things 
up, unless /bin/rotatelogs handles the flocking (you need to check that). But 
since in the suggested above solution, you will end up with several processes 
writing to the same pipe, you must flock it.

I think I need to add flocking the examples I've referenced above. it's needed 
in the child_init and child_exit phases, where there are several processes.

Also I didn't see in your code where you import LOCK_EX. Make sure you don't 
write:

   use POSIX

http://perl.apache.org/docs/1.0/guide/performance.html#Global_vs__Fully_Qualified_Variables
but:

   use Fcntl qw(:flock);

> If I do this, should I close the pipe after use, as I
> do now in mylog() with close $fh?
> 
> Are there any speed or stability concerns logging
> through a rotatelog, vs. to a file?

I'd think that it the difference should be insignificant, unless you have a 
really high vollume traffic, in which case benchmarking should help to tell 
the exact difference.

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com

-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
List etiquette: http://perl.apache.org/maillist/email-etiquette.html