You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl-cvs@perl.apache.org by sb...@hyperreal.org on 1998/12/20 17:32:34 UTC
cvs commit: modperl-site/guide CHANGES all.html control.html index.html obvious.html performance.html porting.html start.html
sbekman 98/12/20 08:32:33
Modified: guide CHANGES all.html control.html index.html
obvious.html performance.html porting.html
start.html
Log:
* fixed: @INC vs %INC obvious.html#Using_Apache_StatINC (thank to Ken Williams)
* new: Apache::SpeedLimit added to
performance.html#Limiting_the_request_rate_speed_
* modified: register_cleanup in Registry scripts (END{} blocks)
based on the last week tread
* new: obvious.html#Handling_the_User_pressed_Stop_
* Found a bug in Pod::Html - it tries to convert HTTP::Foo alike
tokens into hypertext link which breaks the code in the resulting
html. Applied the patch to Pod::Html::VERSION 1.01
1119a1120
> (?! :) # don't convert HTTP::Foo and alike
* Discovered that the guide is being searchable thru the
http://www.apache.org/search.html added a link to index.html
* Extended the control|Log_Rotation section (+Script from Randal)
* control: HUP vs TERM vs USR1. I have asked for validation of this
section, but received none... Added a note about slowness of
termination (Robin Berjon) and possible way to speed it up (Frank
D. Cringle). Added a mneumonics => numbers for SIGs (Marshall Dudley)
* added the missing USE_APACI=1 in start.html#Mod_Perl (Thanks to Tzvetan Stoyanov)
Revision Changes Path
1.4 +43 -0 modperl-site/guide/CHANGES
Index: CHANGES
===================================================================
RCS file: /export/home/cvs/modperl-site/guide/CHANGES,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -r1.3 -r1.4
--- CHANGES 1998/12/14 16:03:38 1.3
+++ CHANGES 1998/12/20 16:32:29 1.4
@@ -1,6 +1,48 @@
This is a CHANGES file for mod_perl mini_guide
+12.20.98
+
+
+* fixed: @INC vs %INC obvious.html#Using_Apache_StatINC (thank to Ken Williams)
+
+
+* new: Apache::SpeedLimit added to
+ performance.html#Limiting_the_request_rate_speed_
+
+
+* modified: register_cleanup in Registry scripts (END{} blocks)
+ based on the last week tread
+
+
+* new: obvious.html#Handling_the_User_pressed_Stop_
+
+
+* Found a bug in Pod::Html - it tries to convert HTTP::Foo alike
+ tokens into hypertext link which breaks the code in the resulting
+ html. Applied the patch to Pod::Html::VERSION 1.01
+
+
+1119a1120
+> (?! :) # don't convert HTTP::Foo and alike
+
+
+* Discovered that the guide is being searchable thru the
+ http://www.apache.org/search.html added a link to index.html
+
+
+* Extended the control|Log_Rotation section (+Script from Randal)
+
+
+* control: HUP vs TERM vs USR1. I have asked for validation of this
+ section, but received none... Added a note about slowness of
+ termination (Robin Berjon) and possible way to speed it up (Frank
+ D. Cringle). Added a mneumonics => numbers for SIGs (Marshall Dudley)
+
+
+* added the missing USE_APACI=1 in start.html#Mod_Perl (Thanks to Tzvetan Stoyanov)
+
+
12.13.98
@@ -29,6 +71,7 @@
Summary of Benchmarking to tune all 5 parameters
+##########################################################################
12.08.98
1.4 +322 -29 modperl-site/guide/all.html
Index: all.html
===================================================================
RCS file: /export/home/cvs/modperl-site/guide/all.html,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -r1.3 -r1.4
--- all.html 1998/12/14 16:03:39 1.3
+++ all.html 1998/12/20 16:32:29 1.4
@@ -15,7 +15,7 @@
<CENTER><P><B>Deploying apache/mod_perl accelerator to give a rocket speed
to your perl cgi-bin scripts.</B></P></CENTER>
-<CENTER><P><B>Version 1.03 Dec, 14 1998</B></P></CENTER>
+<CENTER><P><B>Version 1.04 Dec, 20 1998</B></P></CENTER>
<P>
<HR WIDTH="100%"></P>
@@ -53,6 +53,8 @@
<LI><A HREF="CHANGES">CHANGES</A></LI>
+<LI><A HREF="http://www.apache.org/search.html">Search apache.org along with this guide</A></LI>
+
</UL>
<CENTER><TABLE CELLSPACING=2 CELLPADDING=2 WIDTH="100%" >
@@ -439,7 +441,7 @@
<P>
<PRE> % perl Makefile.PL APACHE_SRC=../apache_1.3.2/src \
- DO_HTTPD=1 PERL_MARK_WHERE=1 EVERYTHING=1
+ DO_HTTPD=1 USE_APACI=1 PERL_MARK_WHERE=1 EVERYTHING=1
% make && make test && make install
</PRE>
<P>
@@ -646,7 +648,7 @@
<B>
<FONT SIZE=-1>
Written by <A HREF="help.html#author">Stas Bekman</A>.
- <BR>Last Modified at 12/09/98
+ <BR>Last Modified at 12/17/98
</FONT>
</B>
</TD>
@@ -1578,6 +1580,27 @@
</PRE>
</DL>
<P>
+It's worth mentioning that restart or termination can sometimes take quite
+a lot of time. Check out an PERL_DESTRUCT_LEVEL=-1 option during the
+mod_perl ./Configure stage, which speeds this up and leads to more robust
+operation in the face of problems, like running out of memory. It is only
+usable if no significant cleanup has to be done by perl END{} blocks and
+DESTROY methods when the child terminates, of course. What's significant
+cleanup? Any change of state outside of the current process that would not
+be handled by the operating system itself. So committing database
+transactions is significant but closing an ordinary file isn't.
+
+<P>
+Some folks used to numbers instead of symbolics, if you are looking for
+these check out your <CODE>kill(3)</CODE> man page, my page points to
+/usr/include/sys/signal.h . and the relevant entries are:
+
+<P>
+<PRE> #define SIGHUP 1 /* hangup, generated when terminal disconnects */
+ #define SIGTERM 15 /* software termination signal */
+ #define SIGUSR1 30 /* user defined signal 1 */
+</PRE>
+<P>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<CENTER><H1><A NAME="Using_apachectl_to_control_the_s">Using apachectl to control the server</A></H1></CENTER>
<P>
@@ -1721,7 +1744,7 @@
use vars qw($ua $proxy);
require LWP::UserAgent;
- use <A HREF="HTTP::Status">HTTP::Status</A>;
+ use HTTP::Status;
###### Config ########
my $test_script_url = "<A HREF="http://www.stas.com:81/perl/test.pl"">http://www.stas.com:81/perl/test.pl"</A>;;
@@ -1767,7 +1790,7 @@
my ($url) = @_;
# Fetch document
- my $res = $ua->request(<A HREF="HTTP::Request->">HTTP::Request-></A>;new(GET => $url));
+ my $res = $ua->request(HTTP::Request->new(GET => $url));
# Check the result status
return 1 if is_success($res->code);
@@ -2039,7 +2062,7 @@
}
use strict;
- use <A HREF="File::Basename">File::Basename</A>;
+ use File::Basename;
# process the passed params
my $cgi = shift || '';
@@ -2066,21 +2089,169 @@
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<CENTER><H1><A NAME="Log_Rotation">Log Rotation</A></H1></CENTER>
<P>
-A little bit off topic but good to know.
+A little bit off topic but good to know and use with mod_perl where your
+error_log can grow at 10-100Mb a day rate if your scripts spit lots of
+warnings...
<P>
To rotate the logs do:
<P>
<PRE> mv access_log access_log.renamed
- kill -USR1 `cat httpd.pid`
+ kill -HUP `cat httpd.pid`
sleep 10; # allow some children to complete requests and logging
# now it's safe to use access_log.renamed
.....
</PRE>
<P>
-The effect of SIGUSR1 is detailed in: <A
+The effect of SIGUSR1 and SIGHUP is detailed in: <A
HREF="http://www.apache.org/docs/stopping.html">http://www.apache.org/docs/stopping.html</A>
+<META>I tried to kill -USR1 and it didn't reset the logs!!! kill -HUP
+did work</META>
+
+<P>
+I use this script:
+
+<P>
+<PRE> #!/usr/apps/bin/perl -Tw
+
+ # this script does a log rotation. Called from crontab.
+
+ use strict;
+ $ENV{PATH}='/bin:/usr/bin';
+
+ ### configuration
+ my @logfiles = qw(access_log error_log);
+ umask 0;
+ my $server = "httpd_perl";
+ my $logs_dir = "/usr/apps/var/$server/logs";
+ my $restart_command = "/usr/apps/sbin/$server/apachectl restart";
+ my $gzip_exec = "/usr/intel/bin/gzip";
+
+ my ($sec,$min,$hour,$mday,$mon,$year) = localtime(time);
+ my $time = sprintf "%0.2d.%0.2d.%0.2d-%0.2d.%0.2d.%0.2d", $year,++$mon,$mday,$hour,$min,$sec;
+ $^I = ".".$time;
+
+ # rename log files
+ chdir $logs_dir;
+ @ARGV = @logfiles;
+ while (<>) {
+ close ARGV;
+ }
+
+ # now restart the server so the logs will be restarted
+ system $restart_command;
+
+ # compress log files
+ foreach (@logfiles) {
+ system "$gzip_exec $_.$time";
+ }
+</PRE>
+<P>
+Randal L. Schwartz contributed this:
+
+<BLOCKQUOTE>
+
+<P>
+Cron fires off setuid script called log-roller that looks like this:
+
+<P>
+<PRE> #!/usr/bin/perl -Tw
+ use strict;
+ use File::Basename;
+</PRE>
+<P>
+<PRE> $ENV{PATH} = "/usr/ucb:/bin:/usr/bin";
+</PRE>
+<P>
+<PRE> my $ROOT = "/WWW/apache"; # names are relative to this
+ my $CONF = "$ROOT/conf/httpd.conf"; # master conf
+ my $MIDNIGHT = "MIDNIGHT"; # name of program in each logdir
+</PRE>
+<P>
+<PRE> my ($user_id, $group_id, $pidfile); # will be set during parse of conf
+ die "not running as root" if $>;
+</PRE>
+<P>
+<PRE> chdir $ROOT or die "Cannot chdir $ROOT: $!";
+</PRE>
+<P>
+<PRE> my %midnights;
+ open CONF, "<$CONF" or die "Cannot open $CONF: $!";
+ while (<CONF>) {
+ if (/^User (\w+)/i) {
+ $user_id = getpwnam($1);
+ next;
+ }
+ if (/^Group (\w+)/i) {
+ $group_id = getgrnam($1);
+ next;
+ }
+ if (/^PidFile (.*)/i) {
+ $pidfile = $1;
+ next;
+ }
+ next unless /^ErrorLog (.*)/i;
+ my $midnight = (dirname $1)."/$MIDNIGHT";
+ next unless -x $midnight;
+ $midnights{$midnight}++;
+ }
+ close CONF;
+</PRE>
+<P>
+<PRE> die "missing User definition" unless defined $user_id;
+ die "missing Group definition" unless defined $group_id;
+ die "missing PidFile definition" unless defined $pidfile;
+</PRE>
+<P>
+<PRE> open PID, $pidfile or die "Cannot open $pidfile: $!";
+ <PID> =~ /(\d+)/;
+ my $httpd_pid = $1;
+ close PID;
+ die "missing pid definition" unless defined $httpd_pid and $httpd_pid;
+ kill 0, $httpd_pid or die "cannot find pid $httpd_pid: $!";
+</PRE>
+<P>
+<PRE> for (sort keys %midnights) {
+ defined(my $pid = fork) or die "cannot fork: $!";
+ if ($pid) {
+ ## parent:
+ waitpid $pid, 0;
+ } else {
+ my $dir = dirname $_;
+ ($(,$)) = ($group_id,$group_id);
+ ($<,$>) = ($user_id,$user_id);
+ chdir $dir or die "cannot chdir $dir: $!";
+ exec "./$MIDNIGHT";
+ die "cannot exec $MIDNIGHT: $!";
+ }
+ }
+</PRE>
+<P>
+<PRE> kill 1, $httpd_pid or die "Cannot sighup $httpd_pid: $!";
+</PRE>
+<P>
+And then individual MIDNIGHT scripts can look like this:
+
+<P>
+<PRE> #!/usr/bin/perl -Tw
+ use strict;
+</PRE>
+<P>
+<PRE> die "bad guy" unless getpwuid($<) =~ /^(root|nobody)$/;
+ my @LOGFILES = qw(access_log error_log);
+ umask 0;
+ $^I = ".".time;
+ @ARGV = @LOGFILES;
+ while (<>) {
+ close ARGV;
+ }
+</PRE>
+<P>
+Can you spot the security holes? Our trusted user base can't or won't. :)
+But these shouldn't be used in hostile situations.
+
+</BLOCKQUOTE>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<P><A HREF="index.html">[Back to the main page]</A></P>
@@ -2096,7 +2267,7 @@
<B>
<FONT SIZE=-1>
Written by <A HREF="help.html#author">Stas Bekman</A>.
- <BR>Last Modified at 12/14/98
+ <BR>Last Modified at 12/16/98
</FONT>
</B>
</TD>
@@ -2152,8 +2323,8 @@
<LI><A HREF="#Names_collisions_with_Modules_an">Names collisions with Modules and libs</A>
<LI><A HREF="#_END_or_DATA_tokens">__END__ or __DATA__ tokens</A>
<LI><A HREF="#Output_from_system_calls">Output from system calls</A>
- <LI><A HREF="#using_format_">using format()</A>
- <LI><A HREF="#using_exit_">using exit()</A>
+ <LI><A HREF="#Using_format_">Using format()</A>
+ <LI><A HREF="#Using_exit_">Using exit()</A>
<LI><A HREF="#Running_from_shell">Running from shell</A>
<LI><A HREF="#I_O_is_different">I/O is different</A>
<LI><A HREF="#HTTP_MIME_Headers">HTTP (MIME) Headers</A>
@@ -2449,13 +2620,13 @@
<P>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
-<CENTER><H3><A NAME="using_format_">using format()</A></H3></CENTER>
+<CENTER><H3><A NAME="Using_format_">Using format()</A></H3></CENTER>
<P>
Currently Possible Only if you have perl compiled with sfio
<P>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
-<CENTER><H3><A NAME="using_exit_">using exit()</A></H3></CENTER>
+<CENTER><H3><A NAME="Using_exit_">Using exit()</A></H3></CENTER>
<P>
Perl's <CODE>exit()</CODE> built-in function cannot be used in mod_perl
scripts. Unless you want the server child to exit (which makes the whole
@@ -2600,16 +2771,22 @@
only calls <CODE>perl_run()</CODE> once, during server startup. Any END
blocks encountered during main server startup, i.e. those pulled in by the
PerlRequire or by any PerlModule are suspended and run at server shutdown,
-aka child_exit (requires apache 1.3b3+). Any END blocks that are
-encountered during compilation of Apache::Registry scripts are called after
-the script done is running, including subsequent invocations when the
-script is cached in memory. All other END blocks encountered during other
-Perl*Handler callbacks, e.g. PerlChildInitHandler, will be suspended while
-the process is running and called during child_exit when the process is
-shutting down. Module authors may be wish to use $r->register_cleanup as
-an alternative to END blocks if this behavior is not desirable.
+aka child_exit (requires apache 1.3b3+).
+
+<P>
+Any END blocks that are encountered during compilation of Apache::Registry
+scripts <STRONG>are called after the script done its
+running</STRONG> (not during the cleanup phase though) including subsequent invocations when
+the script is cached in memory. All other END blocks encountered during
+other Perl*Handler callbacks, e.g. PerlChildInitHandler, will be suspended
+while the process is running and called during child_exit when the process
+is shutting down. Module authors may be wish to use $r->register_cleanup
+as an alternative to END blocks if this behavior is not desirable.
<P>
+The last paragraph is very important for the <A HREF="././obvious.html#Handling_the_User_pressed_Stop_">Handling the 'User pressed Stop button' case</A>
+
+<P>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<CENTER><H3><A NAME="Switches_w_T">Switches -w, -T</A></H3></CENTER>
<P>
@@ -3227,7 +3404,7 @@
<B>
<FONT SIZE=-1>
Written by <A HREF="help.html#author">Stas Bekman</A>.
- <BR>Last Modified at 12/14/98
+ <BR>Last Modified at 12/20/98
</FONT>
</B>
</TD>
@@ -3284,6 +3461,7 @@
<LI><A HREF="#Compiled_Regular_Expressions">Compiled Regular Expressions </A>
<LI><A HREF="#Debugging_your_code_in_Single_Se">Debugging your code in Single Server Mode</A>
<LI><A HREF="#_M_and_other_time_file_tests_u">-M and other time() file tests under mod_perl</A>
+ <LI><A HREF="#Handling_the_User_pressed_Stop_">Handling the 'User pressed Stop button' case</A>
</UL>
<!-- INDEX END -->
@@ -3456,7 +3634,7 @@
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<CENTER><H2><A NAME="Reloading_only_specific_files">Reloading only specific files</A></H2></CENTER>
<P>
-However checking all the Modules in <CODE>@INC</CODE> can add a big
+However checking all the Modules in <CODE>%INC</CODE> can add a some (big?)
overhead to server response times, and you certainly wouldn't want
Apache::StatINC module to be enabled in your production site's
configuration. But sometimes you want to have some Configuration file
@@ -3726,6 +3904,103 @@
request, you should reset the $^T variable as with any other perl script.
Just add <CODE>$^T = time;</CODE> at the beginning of the scripts.
+<P>
+<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
+<CENTER><H1><A NAME="Handling_the_User_pressed_Stop_">Handling the 'User pressed Stop button' case</A></H1></CENTER>
+<P>
+Ok, When user decided to press the stop button, apache will detect that
+thru the SIG{'PIPE'} and will cease the script execution. When we are
+talking about mod_cgi, there are generally no problem, since all opened
+files will be closed and all the resources will be freed (almost if you
+happened to use external lock files, most chances that the resources that
+are being locked by these will be left blocked and non-usable by any other,
+who will use this advisory locking scheme).
+
+<P>
+What's happening if your mod_perl script has some global variables, that
+are being used for resource locking?
+
+<P>
+It's possible not to notice the pitfall if the critical code section
+between lock and unlock is very short and finishes fast, so you never see
+this happens (you aren't fast enough to stop the code in the middle). But
+look at the following scenario:
+
+<P>
+<PRE> 1. lock resource
+ <b><critical section starts>
+ 2. sleep 20 (== do some time consuming processing)
+ <critical section ends></b>
+ 3. unlock resource
+</PRE>
+<P>
+If user presses 'Stop' and Apache sends SIGPIPE before stage 3, since we
+are in the mod_perl mode and we want the lock variable to be cached, it
+will be not unlocked. A kind of ``deadlock''.
+
+<P>
+Here is the working example. Run the server with -X, Press 'Stop' before
+the countup to 10 has been finished. Then rerun the script, it'll hang in
+<CODE>while(1)!</CODE> Resource is not available anymore.
+
+<P>
+<PRE> -------------------------------------------------------------
+ use vars qw(%CACHE);
+ use CGI;
+ $|=1;
+ my $q = new CGI;
+ print $q->header,$q->start_html;
+</PRE>
+<P>
+<PRE> print $q->p("$$ Going to lock!\n");
+</PRE>
+<P>
+<PRE> # actually the while loop below is not needed
+ # (since it's an internal lock and accessible only
+ # by the same process and it if it's locked... it's locked for the
+ # whole child's life
+ while (1) {
+ unless (defined $CACHE{LOCK} and $CACHE{LOCK} == 1) {
+ $CACHE{LOCK} = 1;
+ print $q->p("Got the lock!\n");
+ last;
+ }
+ }
+ print $q->p("Going to sleep (I mean working)!");
+ my $c=0;
+ foreach (1..10) {
+ sleep 1;
+ print $c++,"\n<BR>";
+ }
+</PRE>
+<P>
+<PRE> print $q->p("Going to unlock!");
+ $CACHE{LOCK} = 0;
+ print $q->p("Unlock!\n");
+ -------------------------------------------------------------
+</PRE>
+<P>
+You will ask, what is the solution for this problem? As noted in the
+<A HREF="././porting.html#END_blocks">END blocks</A> any END blocks that are encountered during compilation of Apache::Registry
+scripts are called after the script done is running, including subsequent
+invocations when the script is cached in memory. So if you are running in
+Apache::Registry mode, the following is your remedy:
+
+<P>
+<PRE> END {
+ $CACHE{LOCK} = 0;
+ }
+</PRE>
+<P>
+Notice that the END block will be run after the Apache::Registry::handler
+is finished (not during the cleanup phase though).
+
+<P>
+If you are into a perl API, use the <STRONG>register_cleanup()</STRONG> method of Apache.
+
+<P>
+<PRE> $r->register_cleanup(sub {$CACHE{LOCK} = 0;});
+</PRE>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<P><A HREF="index.html">[Back to the main page]</A></P>
@@ -3740,7 +4015,7 @@
<B>
<FONT SIZE=-1>
Written by <A HREF="help.html#author">Stas Bekman</A>.
- <BR>Last Modified at 12/09/98
+ <BR>Last Modified at 12/20/98
</FONT>
</B>
</TD>
@@ -4170,6 +4445,7 @@
<LI><A HREF="#Reducing_the_Memory_Usage">Reducing the Memory Usage</A>
<LI><A HREF="#Limiting_the_size_of_the_process">Limiting the size of the processes</A>
<LI><A HREF="#Limiting_the_resources_used_by_h">Limiting the resources used by httpd children</A>
+ <LI><A HREF="#Limiting_the_request_rate_speed_">Limiting the request rate speed (robots blocking)</A>
<LI><A HREF="#Benchmarks_Impressing_your_Boss">Benchmarks. Impressing your Boss and Colleagues.</A>
<UL>
@@ -4497,14 +4773,14 @@
example of use at modperl.com in a PerlRequire'd file:
<P>
-<PRE> use <A HREF="File::Find">File::Find</A> 'finddepth';
+<PRE> use File::Find 'finddepth';
use Apache::RegistryLoader ();
{
my $perl_dir = "perl/";
my $rl = Apache::RegistryLoader->new;
finddepth(sub {
return unless /\.pl$/;
- my $url = "/$<A HREF="File::Find::dir/">File::Find::dir/</A>$_";
+ my $url = "/$File::Find::dir/$_";
print "pre-loading $url\n";
my $status = $rl->handler($url);
@@ -4590,6 +4866,23 @@
<P>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
+<CENTER><H1><A NAME="Limiting_the_request_rate_speed_">Limiting the request rate speed (robots blocking)</A></H1></CENTER>
+<P>
+A limitation of using pattern matching to identify robots is that it only
+catches the robots that you know about, and only those that identify
+themselves by name. A few devious robots masquerade as users by using user
+agent strings that identify themselves as conventional browsers. To catch
+such robots, you'll have to be more sophisticated.
+
+<P>
+Apache::SpeedLimit comes for you to help, see:
+
+<P>
+<A
+HREF="http://www.modperl.com/chapters/ch6.html#Blocking_Greedy_Clients">http://www.modperl.com/chapters/ch6.html#Blocking_Greedy_Clients</A>
+
+<P>
+<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<CENTER><H1><A NAME="Benchmarks_Impressing_your_Boss">Benchmarks. Impressing your Boss and Colleagues.</A></H1></CENTER>
<P>
How much faster is mod_perl that CGI? There are many ways to benchmark the
@@ -5224,7 +5517,7 @@
###
foreach (1..$nof_requests_total) {
foreach my $url (@urls) {
- my $request = <A HREF="HTTP::Request->">HTTP::Request-></A>;new('GET', $url);
+ my $request = HTTP::Request->new('GET', $url);
$ua->register($request);
}
}
@@ -5610,7 +5903,7 @@
<B>
<FONT SIZE=-1>
Written by <A HREF="help.html#author">Stas Bekman</A>.
- <BR>Last Modified at 12/14/98
+ <BR>Last Modified at 12/20/98
</FONT>
</B>
</TD>
1.6 +176 -7 modperl-site/guide/control.html
Index: control.html
===================================================================
RCS file: /export/home/cvs/modperl-site/guide/control.html,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -r1.5 -r1.6
--- control.html 1998/12/14 16:03:39 1.5
+++ control.html 1998/12/20 16:32:30 1.6
@@ -126,6 +126,27 @@
</PRE>
</DL>
<P>
+It's worth mentioning that restart or termination can sometimes take quite
+a lot of time. Check out an PERL_DESTRUCT_LEVEL=-1 option during the
+mod_perl ./Configure stage, which speeds this up and leads to more robust
+operation in the face of problems, like running out of memory. It is only
+usable if no significant cleanup has to be done by perl END{} blocks and
+DESTROY methods when the child terminates, of course. What's significant
+cleanup? Any change of state outside of the current process that would not
+be handled by the operating system itself. So committing database
+transactions is significant but closing an ordinary file isn't.
+
+<P>
+Some folks used to numbers instead of symbolics, if you are looking for
+these check out your <CODE>kill(3)</CODE> man page, my page points to
+/usr/include/sys/signal.h . and the relevant entries are:
+
+<P>
+<PRE> #define SIGHUP 1 /* hangup, generated when terminal disconnects */
+ #define SIGTERM 15 /* software termination signal */
+ #define SIGUSR1 30 /* user defined signal 1 */
+</PRE>
+<P>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<CENTER><H1><A NAME="Using_apachectl_to_control_the_s">Using apachectl to control the server</A></H1></CENTER>
<P>
@@ -269,7 +290,7 @@
use vars qw($ua $proxy);
require LWP::UserAgent;
- use <A HREF="HTTP::Status">HTTP::Status</A>;
+ use HTTP::Status;
###### Config ########
my $test_script_url = "<A HREF="http://www.stas.com:81/perl/test.pl"">http://www.stas.com:81/perl/test.pl"</A>;;
@@ -315,7 +336,7 @@
my ($url) = @_;
# Fetch document
- my $res = $ua->request(<A HREF="HTTP::Request->">HTTP::Request-></A>;new(GET => $url));
+ my $res = $ua->request(HTTP::Request->new(GET => $url));
# Check the result status
return 1 if is_success($res->code);
@@ -587,7 +608,7 @@
}
use strict;
- use <A HREF="File::Basename">File::Basename</A>;
+ use File::Basename;
# process the passed params
my $cgi = shift || '';
@@ -614,21 +635,169 @@
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<CENTER><H1><A NAME="Log_Rotation">Log Rotation</A></H1></CENTER>
<P>
-A little bit off topic but good to know.
+A little bit off topic but good to know and use with mod_perl where your
+error_log can grow at 10-100Mb a day rate if your scripts spit lots of
+warnings...
<P>
To rotate the logs do:
<P>
<PRE> mv access_log access_log.renamed
- kill -USR1 `cat httpd.pid`
+ kill -HUP `cat httpd.pid`
sleep 10; # allow some children to complete requests and logging
# now it's safe to use access_log.renamed
.....
</PRE>
<P>
-The effect of SIGUSR1 is detailed in: <A
+The effect of SIGUSR1 and SIGHUP is detailed in: <A
HREF="http://www.apache.org/docs/stopping.html">http://www.apache.org/docs/stopping.html</A>
+<META>I tried to kill -USR1 and it didn't reset the logs!!! kill -HUP
+did work</META>
+
+<P>
+I use this script:
+
+<P>
+<PRE> #!/usr/apps/bin/perl -Tw
+
+ # this script does a log rotation. Called from crontab.
+
+ use strict;
+ $ENV{PATH}='/bin:/usr/bin';
+
+ ### configuration
+ my @logfiles = qw(access_log error_log);
+ umask 0;
+ my $server = "httpd_perl";
+ my $logs_dir = "/usr/apps/var/$server/logs";
+ my $restart_command = "/usr/apps/sbin/$server/apachectl restart";
+ my $gzip_exec = "/usr/intel/bin/gzip";
+
+ my ($sec,$min,$hour,$mday,$mon,$year) = localtime(time);
+ my $time = sprintf "%0.2d.%0.2d.%0.2d-%0.2d.%0.2d.%0.2d", $year,++$mon,$mday,$hour,$min,$sec;
+ $^I = ".".$time;
+
+ # rename log files
+ chdir $logs_dir;
+ @ARGV = @logfiles;
+ while (<>) {
+ close ARGV;
+ }
+
+ # now restart the server so the logs will be restarted
+ system $restart_command;
+
+ # compress log files
+ foreach (@logfiles) {
+ system "$gzip_exec $_.$time";
+ }
+</PRE>
+<P>
+Randal L. Schwartz contributed this:
+
+<BLOCKQUOTE>
+
+<P>
+Cron fires off setuid script called log-roller that looks like this:
+
+<P>
+<PRE> #!/usr/bin/perl -Tw
+ use strict;
+ use File::Basename;
+</PRE>
+<P>
+<PRE> $ENV{PATH} = "/usr/ucb:/bin:/usr/bin";
+</PRE>
+<P>
+<PRE> my $ROOT = "/WWW/apache"; # names are relative to this
+ my $CONF = "$ROOT/conf/httpd.conf"; # master conf
+ my $MIDNIGHT = "MIDNIGHT"; # name of program in each logdir
+</PRE>
+<P>
+<PRE> my ($user_id, $group_id, $pidfile); # will be set during parse of conf
+ die "not running as root" if $>;
+</PRE>
+<P>
+<PRE> chdir $ROOT or die "Cannot chdir $ROOT: $!";
+</PRE>
+<P>
+<PRE> my %midnights;
+ open CONF, "<$CONF" or die "Cannot open $CONF: $!";
+ while (<CONF>) {
+ if (/^User (\w+)/i) {
+ $user_id = getpwnam($1);
+ next;
+ }
+ if (/^Group (\w+)/i) {
+ $group_id = getgrnam($1);
+ next;
+ }
+ if (/^PidFile (.*)/i) {
+ $pidfile = $1;
+ next;
+ }
+ next unless /^ErrorLog (.*)/i;
+ my $midnight = (dirname $1)."/$MIDNIGHT";
+ next unless -x $midnight;
+ $midnights{$midnight}++;
+ }
+ close CONF;
+</PRE>
+<P>
+<PRE> die "missing User definition" unless defined $user_id;
+ die "missing Group definition" unless defined $group_id;
+ die "missing PidFile definition" unless defined $pidfile;
+</PRE>
+<P>
+<PRE> open PID, $pidfile or die "Cannot open $pidfile: $!";
+ <PID> =~ /(\d+)/;
+ my $httpd_pid = $1;
+ close PID;
+ die "missing pid definition" unless defined $httpd_pid and $httpd_pid;
+ kill 0, $httpd_pid or die "cannot find pid $httpd_pid: $!";
+</PRE>
+<P>
+<PRE> for (sort keys %midnights) {
+ defined(my $pid = fork) or die "cannot fork: $!";
+ if ($pid) {
+ ## parent:
+ waitpid $pid, 0;
+ } else {
+ my $dir = dirname $_;
+ ($(,$)) = ($group_id,$group_id);
+ ($<,$>) = ($user_id,$user_id);
+ chdir $dir or die "cannot chdir $dir: $!";
+ exec "./$MIDNIGHT";
+ die "cannot exec $MIDNIGHT: $!";
+ }
+ }
+</PRE>
+<P>
+<PRE> kill 1, $httpd_pid or die "Cannot sighup $httpd_pid: $!";
+</PRE>
+<P>
+And then individual MIDNIGHT scripts can look like this:
+
+<P>
+<PRE> #!/usr/bin/perl -Tw
+ use strict;
+</PRE>
+<P>
+<PRE> die "bad guy" unless getpwuid($<) =~ /^(root|nobody)$/;
+ my @LOGFILES = qw(access_log error_log);
+ umask 0;
+ $^I = ".".time;
+ @ARGV = @LOGFILES;
+ while (<>) {
+ close ARGV;
+ }
+</PRE>
+<P>
+Can you spot the security holes? Our trusted user base can't or won't. :)
+But these shouldn't be used in hostile situations.
+
+</BLOCKQUOTE>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<P><A HREF="index.html">[Back to the main page]</A></P>
@@ -644,7 +813,7 @@
<B>
<FONT SIZE=-1>
Written by <A HREF="help.html#author">Stas Bekman</A>.
- <BR>Last Modified at 12/14/98
+ <BR>Last Modified at 12/16/98
</FONT>
</B>
</TD>
1.6 +4 -1 modperl-site/guide/index.html
Index: index.html
===================================================================
RCS file: /export/home/cvs/modperl-site/guide/index.html,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -r1.5 -r1.6
--- index.html 1998/12/14 16:03:39 1.5
+++ index.html 1998/12/20 16:32:30 1.6
@@ -15,7 +15,7 @@
to your perl cgi-bin scripts.</B></P></CENTER>
-<CENTER><P><B>Version 1.03 Dec, 14 1998</B></P></CENTER>
+<CENTER><P><B>Version 1.04 Dec, 20 1998</B></P></CENTER>
<P>
@@ -69,6 +69,9 @@
<LI><A HREF="CHANGES">CHANGES</A></LI>
+
+
+<LI><A HREF="http://www.apache.org/search.html">Search apache.org along with this guide</A></LI>
</UL>
1.5 +100 -2 modperl-site/guide/obvious.html
Index: obvious.html
===================================================================
RCS file: /export/home/cvs/modperl-site/guide/obvious.html,v
retrieving revision 1.4
retrieving revision 1.5
diff -u -r1.4 -r1.5
--- obvious.html 1998/12/10 09:22:22 1.4
+++ obvious.html 1998/12/20 16:32:30 1.5
@@ -31,6 +31,7 @@
<LI><A HREF="#Compiled_Regular_Expressions">Compiled Regular Expressions </A>
<LI><A HREF="#Debugging_your_code_in_Single_Se">Debugging your code in Single Server Mode</A>
<LI><A HREF="#_M_and_other_time_file_tests_u">-M and other time() file tests under mod_perl</A>
+ <LI><A HREF="#Handling_the_User_pressed_Stop_">Handling the 'User pressed Stop button' case</A>
</UL>
<!-- INDEX END -->
@@ -203,7 +204,7 @@
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<CENTER><H2><A NAME="Reloading_only_specific_files">Reloading only specific files</A></H2></CENTER>
<P>
-However checking all the Modules in <CODE>@INC</CODE> can add a big
+However checking all the Modules in <CODE>%INC</CODE> can add a some (big?)
overhead to server response times, and you certainly wouldn't want
Apache::StatINC module to be enabled in your production site's
configuration. But sometimes you want to have some Configuration file
@@ -473,6 +474,103 @@
request, you should reset the $^T variable as with any other perl script.
Just add <CODE>$^T = time;</CODE> at the beginning of the scripts.
+<P>
+<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
+<CENTER><H1><A NAME="Handling_the_User_pressed_Stop_">Handling the 'User pressed Stop button' case</A></H1></CENTER>
+<P>
+Ok, When user decided to press the stop button, apache will detect that
+thru the SIG{'PIPE'} and will cease the script execution. When we are
+talking about mod_cgi, there are generally no problem, since all opened
+files will be closed and all the resources will be freed (almost if you
+happened to use external lock files, most chances that the resources that
+are being locked by these will be left blocked and non-usable by any other,
+who will use this advisory locking scheme).
+
+<P>
+What's happening if your mod_perl script has some global variables, that
+are being used for resource locking?
+
+<P>
+It's possible not to notice the pitfall if the critical code section
+between lock and unlock is very short and finishes fast, so you never see
+this happens (you aren't fast enough to stop the code in the middle). But
+look at the following scenario:
+
+<P>
+<PRE> 1. lock resource
+ <b><critical section starts>
+ 2. sleep 20 (== do some time consuming processing)
+ <critical section ends></b>
+ 3. unlock resource
+</PRE>
+<P>
+If user presses 'Stop' and Apache sends SIGPIPE before stage 3, since we
+are in the mod_perl mode and we want the lock variable to be cached, it
+will be not unlocked. A kind of ``deadlock''.
+
+<P>
+Here is the working example. Run the server with -X, Press 'Stop' before
+the countup to 10 has been finished. Then rerun the script, it'll hang in
+<CODE>while(1)!</CODE> Resource is not available anymore.
+
+<P>
+<PRE> -------------------------------------------------------------
+ use vars qw(%CACHE);
+ use CGI;
+ $|=1;
+ my $q = new CGI;
+ print $q->header,$q->start_html;
+</PRE>
+<P>
+<PRE> print $q->p("$$ Going to lock!\n");
+</PRE>
+<P>
+<PRE> # actually the while loop below is not needed
+ # (since it's an internal lock and accessible only
+ # by the same process and it if it's locked... it's locked for the
+ # whole child's life
+ while (1) {
+ unless (defined $CACHE{LOCK} and $CACHE{LOCK} == 1) {
+ $CACHE{LOCK} = 1;
+ print $q->p("Got the lock!\n");
+ last;
+ }
+ }
+ print $q->p("Going to sleep (I mean working)!");
+ my $c=0;
+ foreach (1..10) {
+ sleep 1;
+ print $c++,"\n<BR>";
+ }
+</PRE>
+<P>
+<PRE> print $q->p("Going to unlock!");
+ $CACHE{LOCK} = 0;
+ print $q->p("Unlock!\n");
+ -------------------------------------------------------------
+</PRE>
+<P>
+You will ask, what is the solution for this problem? As noted in the
+<A HREF="././porting.html#END_blocks">END blocks</A> any END blocks that are encountered during compilation of Apache::Registry
+scripts are called after the script done is running, including subsequent
+invocations when the script is cached in memory. So if you are running in
+Apache::Registry mode, the following is your remedy:
+
+<P>
+<PRE> END {
+ $CACHE{LOCK} = 0;
+ }
+</PRE>
+<P>
+Notice that the END block will be run after the Apache::Registry::handler
+is finished (not during the cleanup phase though).
+
+<P>
+If you are into a perl API, use the <STRONG>register_cleanup()</STRONG> method of Apache.
+
+<P>
+<PRE> $r->register_cleanup(sub {$CACHE{LOCK} = 0;});
+</PRE>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<P><A HREF="index.html">[Back to the main page]</A></P>
@@ -487,7 +585,7 @@
<B>
<FONT SIZE=-1>
Written by <A HREF="help.html#author">Stas Bekman</A>.
- <BR>Last Modified at 12/09/98
+ <BR>Last Modified at 12/20/98
</FONT>
</B>
</TD>
1.6 +23 -4 modperl-site/guide/performance.html
Index: performance.html
===================================================================
RCS file: /export/home/cvs/modperl-site/guide/performance.html,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -r1.5 -r1.6
--- performance.html 1998/12/14 16:03:39 1.5
+++ performance.html 1998/12/20 16:32:30 1.6
@@ -30,6 +30,7 @@
<LI><A HREF="#Reducing_the_Memory_Usage">Reducing the Memory Usage</A>
<LI><A HREF="#Limiting_the_size_of_the_process">Limiting the size of the processes</A>
<LI><A HREF="#Limiting_the_resources_used_by_h">Limiting the resources used by httpd children</A>
+ <LI><A HREF="#Limiting_the_request_rate_speed_">Limiting the request rate speed (robots blocking)</A>
<LI><A HREF="#Benchmarks_Impressing_your_Boss">Benchmarks. Impressing your Boss and Colleagues.</A>
<UL>
@@ -362,14 +363,14 @@
example of use at modperl.com in a PerlRequire'd file:
<P>
-<PRE> use <A HREF="File::Find">File::Find</A> 'finddepth';
+<PRE> use File::Find 'finddepth';
use Apache::RegistryLoader ();
{
my $perl_dir = "perl/";
my $rl = Apache::RegistryLoader->new;
finddepth(sub {
return unless /\.pl$/;
- my $url = "/$<A HREF="File::Find::dir/">File::Find::dir/</A>$_";
+ my $url = "/$File::Find::dir/$_";
print "pre-loading $url\n";
my $status = $rl->handler($url);
@@ -461,6 +462,24 @@
<P>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
+<CENTER><H1><A NAME="Limiting_the_request_rate_speed_">Limiting the request rate speed (robots blocking)</A></H1></CENTER>
+<P>
+A limitation of using pattern matching to identify robots is that it only
+catches the robots that you know about, and only those that identify
+themselves by name. A few devious robots masquerade as users by using user
+agent strings that identify themselves as conventional browsers. To catch
+such robots, you'll have to be more sophisticated.
+
+<P>
+Apache::SpeedLimit comes for you to help, see:
+
+<P>
+<A
+HREF="http://www.modperl.com/chapters/ch6.html#Blocking_Greedy_Clients">http://www.modperl.com/chapters/ch6.html#Blocking_Greedy_Clients</A>
+
+
+<P>
+<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<CENTER><H1><A NAME="Benchmarks_Impressing_your_Boss">Benchmarks. Impressing your Boss and Colleagues.</A></H1></CENTER>
<P>
How much faster is mod_perl that CGI? There are many ways to benchmark the
@@ -1100,7 +1119,7 @@
###
foreach (1..$nof_requests_total) {
foreach my $url (@urls) {
- my $request = <A HREF="HTTP::Request->">HTTP::Request-></A>;new('GET', $url);
+ my $request = HTTP::Request->new('GET', $url);
$ua->register($request);
}
}
@@ -1486,7 +1505,7 @@
<B>
<FONT SIZE=-1>
Written by <A HREF="help.html#author">Stas Bekman</A>.
- <BR>Last Modified at 12/14/98
+ <BR>Last Modified at 12/20/98
</FONT>
</B>
</TD>
1.7 +21 -13 modperl-site/guide/porting.html
Index: porting.html
===================================================================
RCS file: /export/home/cvs/modperl-site/guide/porting.html,v
retrieving revision 1.6
retrieving revision 1.7
diff -u -r1.6 -r1.7
--- porting.html 1998/12/14 16:03:40 1.6
+++ porting.html 1998/12/20 16:32:30 1.7
@@ -30,8 +30,8 @@
<LI><A HREF="#Names_collisions_with_Modules_an">Names collisions with Modules and libs</A>
<LI><A HREF="#_END_or_DATA_tokens">__END__ or __DATA__ tokens</A>
<LI><A HREF="#Output_from_system_calls">Output from system calls</A>
- <LI><A HREF="#using_format_">using format()</A>
- <LI><A HREF="#using_exit_">using exit()</A>
+ <LI><A HREF="#Using_format_">Using format()</A>
+ <LI><A HREF="#Using_exit_">Using exit()</A>
<LI><A HREF="#Running_from_shell">Running from shell</A>
<LI><A HREF="#I_O_is_different">I/O is different</A>
<LI><A HREF="#HTTP_MIME_Headers">HTTP (MIME) Headers</A>
@@ -328,13 +328,13 @@
<P>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
-<CENTER><H3><A NAME="using_format_">using format()</A></H3></CENTER>
+<CENTER><H3><A NAME="Using_format_">Using format()</A></H3></CENTER>
<P>
Currently Possible Only if you have perl compiled with sfio
<P>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
-<CENTER><H3><A NAME="using_exit_">using exit()</A></H3></CENTER>
+<CENTER><H3><A NAME="Using_exit_">Using exit()</A></H3></CENTER>
<P>
Perl's <CODE>exit()</CODE> built-in function cannot be used in mod_perl
scripts. Unless you want the server child to exit (which makes the whole
@@ -479,16 +479,24 @@
only calls <CODE>perl_run()</CODE> once, during server startup. Any END
blocks encountered during main server startup, i.e. those pulled in by the
PerlRequire or by any PerlModule are suspended and run at server shutdown,
-aka child_exit (requires apache 1.3b3+). Any END blocks that are
-encountered during compilation of Apache::Registry scripts are called after
-the script done is running, including subsequent invocations when the
-script is cached in memory. All other END blocks encountered during other
-Perl*Handler callbacks, e.g. PerlChildInitHandler, will be suspended while
-the process is running and called during child_exit when the process is
-shutting down. Module authors may be wish to use $r->register_cleanup as
-an alternative to END blocks if this behavior is not desirable.
+aka child_exit (requires apache 1.3b3+).
<P>
+Any END blocks that are encountered during compilation of Apache::Registry
+scripts <STRONG>are called after the script done its
+running</STRONG> (not during the cleanup phase though) including subsequent invocations when
+the script is cached in memory. All other END blocks encountered during
+other Perl*Handler callbacks, e.g. PerlChildInitHandler, will be suspended
+while the process is running and called during child_exit when the process
+is shutting down. Module authors may be wish to use $r->register_cleanup
+as an alternative to END blocks if this behavior is not desirable.
+
+<P>
+The last paragraph is very important for the <A HREF="././obvious.html#Handling_the_User_pressed_Stop_">Handling the 'User pressed Stop button' case</A>
+
+
+
+<P>
<P><B><FONT SIZE=-1><A HREF="#toc">[TOC]</A></FONT></B><HR WIDTH="100%"></P>
<CENTER><H3><A NAME="Switches_w_T">Switches -w, -T</A></H3></CENTER>
<P>
@@ -1110,7 +1118,7 @@
<B>
<FONT SIZE=-1>
Written by <A HREF="help.html#author">Stas Bekman</A>.
- <BR>Last Modified at 12/14/98
+ <BR>Last Modified at 12/20/98
</FONT>
</B>
</TD>
1.5 +2 -2 modperl-site/guide/start.html
Index: start.html
===================================================================
RCS file: /export/home/cvs/modperl-site/guide/start.html,v
retrieving revision 1.4
retrieving revision 1.5
diff -u -r1.4 -r1.5
--- start.html 1998/12/10 09:22:22 1.4
+++ start.html 1998/12/20 16:32:31 1.5
@@ -151,7 +151,7 @@
<P>
<PRE> % perl Makefile.PL APACHE_SRC=../apache_1.3.2/src \
- DO_HTTPD=1 PERL_MARK_WHERE=1 EVERYTHING=1
+ DO_HTTPD=1 USE_APACI=1 PERL_MARK_WHERE=1 EVERYTHING=1
% make && make test && make install
</PRE>
<P>
@@ -363,7 +363,7 @@
<B>
<FONT SIZE=-1>
Written by <A HREF="help.html#author">Stas Bekman</A>.
- <BR>Last Modified at 12/09/98
+ <BR>Last Modified at 12/17/98
</FONT>
</B>
</TD>