You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by justin <ju...@dslr.net> on 2003/12/18 01:08:33 UTC

introducing new code with no perceived user delays?

One registry module checks for a new file before a request starts.
Another Registry class does no checking at all. Apache::Reload checks a
file on disk before the request, or ALL module timestamps. None of these
solutions are satisfactory. If code is rolled out, 10+ modperl processes
all reloading it at the same time will halt forward motion for 30
seconds or more (depends on code size, init time, and so on).

Is it possible to design something that checks in the EXIT phase, and
not every request, just randomly, say, every N times, so all servers
don't freeze up .. if they find a named 're-cycle' file is newer than
their own life-span, then they child_exit ? Ideally, they eval or syntax
check the new module(s), before deciding to child_exit.

If they find they do not eval without error, they erase the re-cycle
command file and try not to exit until an admin realizes their mistake
;)

would such a module be a logical plug-in replacement for the Registry
handler, a sub-class, or something else? I'd like to write it, but am
interested to know if I'm barking up the wrong tree.

thanks -Justin


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: [patch] Re: introducing new code with no perceived user delays?

Posted by Stas Bekman <st...@stason.org>.
Fred Moyer wrote:
> A patch for http://perl.apache.org/docs/1.0/guide/scenario.pod.orig
> from discussion on the modperl list is below which includes some code
> posted to the list.

Thanks Fred, applied. (was waiting for apache.org to come back)

In the future, please wrap long lines. I had to manually add it.

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


---------------------------------------------------------------------
To unsubscribe, e-mail: docs-dev-unsubscribe@perl.apache.org
For additional commands, e-mail: docs-dev-help@perl.apache.org


[patch] Re: introducing new code with no perceived user delays?

Posted by Fred Moyer <fr...@taperfriendlymusic.org>.
A patch for http://perl.apache.org/docs/1.0/guide/scenario.pod.orig
from discussion on the modperl list is below which includes some code
posted to the list.

On Sat, 2003-12-20 at 21:17, Stas Bekman wrote:
> Fred Moyer wrote:
> 
> > looks familiar.  Do I get credit? :)
> 
> You obviously do get the credit, but in the contributor logs, not in the text 
> itself:
> http://perl.apache.org/search/swish.cgi?query=randal&sbm=&submit=search
> http://perl.apache.org/about/contributors/other.html#Contributors
> 
> > I'm not familiar with the protocol regarding credit for the mod_perl
> > documentation but this updated patch gives credit to the contributors :)
> 
> Fred, can you please send in a previous version (no contributor names in the 
> text) but as a unified patch, i.e generated with 'diff -u':
> 
>    diff -u orig new
> 
> See:
> http://perl.apache.org/download/docs.html#Submitting_documentation_patches
> 
> Thanks.

--- scenario.pod.orig	2003-12-20 02:18:35.000000000 +0000
+++ scenario.pod	2003-12-20 11:56:09.000000000 +0000
@@ -1653,6 +1653,10 @@
 
 =head1 mod_rewrite Examples
 
+Example code for using mod_rewrite with mod_perl application servers.
Several examples were taken from the mailing list.
+
+=head2 Rewriting Requests Based on File Extension
+
 In the mod_proxy + mod_perl servers scenario, C<ProxyPass> was used to
 redirect all requests to the mod_perl server, by matching the
 beginning of the relative URI (e.g. I</perl>). What should you do if
@@ -1686,16 +1690,20 @@
 It says: log all the rewrites thru the pipe to the C<rotatelogs>
 utility which will rotate the logs every 2 hours (86400 secs).
 
-More examples:
+=head2 Internet Exporer 5 favicon.ico 404
 
 Redirect all those IE5 requests for I<favicon.ico> to a central image:
 
   RewriteRule .*favicon.ico /wherever/favicon.ico [PT,NS]
 
+=head2 Hiding Extensions for Dynamic Pages
+
 A quick way to make dynamic pages look static:
 
   RewriteRule ^/wherever/([a-zA-Z]+).html /perl-bin/$1.cgi [PT]
 
+=head2 Serving Static Content Locally and Rewriting Everything Else
+
 Instead of keeping all your Perl scripts in I</perl> and your static
 content everywhere else, you could keep your static content in special
 directories and keep your Perl scripts everywhere else.  You can still
@@ -1738,6 +1746,35 @@
   RewriteRule ^/(.*) http://www.example.com:8080/$1 [P]
   ProxyPassReverse / http://www.example.com/
 
+=head2 Upgrading mod_perl Heavy Application Instances
+
+When using a light/heavy separation method one of the challenges of
running a production environment is being able to upgrade to newer
versions of mod_perl or your own application. The following method can
be used without having to do a server restart.
+
+Add the following rewrite rule to your httpd.conf file:
+
+  RewriteEngine On
+  RewriteMap maps txt:/etc/httpd.maps
+  RewriteRule ^(.*) http://${maps:appserver}$1 [proxy]
+
+Create the file /etc/httpd.maps and add the following entry:
+
+  appserver foo.com:9999
+
+Mod_rewrite rereads (or checks the mtime of) the file on every request
so the change takes effect immediately. To seamlessly upgrade your
application server to a new version, install a new version on a
different port. After checking for a quality installation, edit
/etc/httpd.maps to point to the new server. After the file is written
the next request the server processes will be redirected to the new
installation.
+
+=head2 Blocking IP Addresses
+
+The following rewrite code blocks IP addresses:
+
+  RewriteCond /web/site/var/blocked/REMOTE_ADDR-%{REMOTE_ADDR} -f
+  RewriteRule .* http://YOUR-HOST-BLOCKED-FOR-EXCESSIVE-CONSUMPTION
[redirect,last]
+
+To block IP address 10.1.2.3, simply touch
+
+  /web/site/var/blocked/REMOTE_ADDR-10.1.2.3
+
+This has an advantage over Apache parsing a long file of addresses in
that the OS is better at a file lookup.
+
 =head1 Caching in mod_proxy
 
 This is not really mod_perl related, so I'll just stress one



---------------------------------------------------------------------
To unsubscribe, e-mail: docs-dev-unsubscribe@perl.apache.org
For additional commands, e-mail: docs-dev-help@perl.apache.org


Re: introducing new code with no perceived user delays?

Posted by "Randal L. Schwartz" <me...@stonehenge.com>.
>>>>> "Stas" == Stas Bekman <st...@stason.org> writes:

Stas> You obviously do get the credit, but in the contributor logs, not in
Stas> the text itself:
Stas> http://perl.apache.org/search/swish.cgi?query=randal&sbm=&submit=search
Stas> http://perl.apache.org/about/contributors/other.html#Contributors

Yeah, I was half joking, and I think I even knew I was already in there.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<me...@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: introducing new code with no perceived user delays?

Posted by Stas Bekman <st...@stason.org>.
Fred Moyer wrote:

> looks familiar.  Do I get credit? :)

You obviously do get the credit, but in the contributor logs, not in the text 
itself:
http://perl.apache.org/search/swish.cgi?query=randal&sbm=&submit=search
http://perl.apache.org/about/contributors/other.html#Contributors

> I'm not familiar with the protocol regarding credit for the mod_perl
> documentation but this updated patch gives credit to the contributors :)

Fred, can you please send in a previous version (no contributor names in the 
text) but as a unified patch, i.e generated with 'diff -u':

   diff -u orig new

See:
http://perl.apache.org/download/docs.html#Submitting_documentation_patches

Thanks.

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: introducing new code with no perceived user delays?

Posted by Fred Moyer <fr...@taperfriendlymusic.org>.
On Sat, 2003-12-20 at 20:29, Randal L. Schwartz wrote:
> >>>>> "Fred" == Fred Moyer <fr...@taperfriendlymusic.org> writes:
> 
> Fred> < =head2 Blocking IP Addresses
> Fred> <
> Fred> < The following rewrite code blocks IP addresses:
> Fred> <
> Fred> <   RewriteCond /web/site/var/blocked/REMOTE_ADDR-%{REMOTE_ADDR} -f
> Fred> <   RewriteRule .* http://YOUR-HOST-BLOCKED-FOR-EXCESSIVE-CONSUMPTION
> Fred> [redirect,last]
> Fred> <
> Fred> < To block IP address 10.1.2.3, simply touch
> Fred> <
> Fred> <   /web/site/var/blocked/REMOTE_ADDR-10.1.2.3
> Fred> <
> Fred> < This has an advantage over Apache parsing a long file of addresses in
> Fred> that the OS is better at a file lookup.
> 
> Hey, that looks familiar.  Do I get credit? :)

I'm not familiar with the protocol regarding credit for the mod_perl
documentation but this updated patch gives credit to the contributors :)
  
1656,1659d1655
< Example code for using mod_rewrite with mod_perl application servers. Several examples were taken from the mailing list.
< 
< =head2 Rewriting Requests Based on File Extension
< 
1693c1689
< =head2 Internet Exporer 5 favicon.ico 404
---
> More examples:
1699,1700d1694
< =head2 Hiding Extensions for Dynamic Pages
< 
1705,1706d1698
< =head2 Serving Static Content Locally and Rewriting Everything Else
< 
1749,1777d1740
< =head2 Upgrading mod_perl Heavy Application Instances
< 
< Contributed to the mailing list on 12/18/03 by Rob Nagler
< When using a light/heavy separation method one of the challenges of running a production environment is being able to upgrade to newer versions of mod_perl or your own application. The following method can be used without having to do a server restart.
< 
< Add the following rewrite rule to your httpd.conf file:
< 
<   RewriteEngine On
<   RewriteMap maps txt:/etc/httpd.maps
<   RewriteRule ^(.*) http://${maps:appserver}$1 [proxy]
< 
< Create the file /etc/httpd.maps and add the following entry:
< 
<   appserver foo.com:9999
< 
< Mod_rewrite rereads (or checks the mtime of) the file on every request so the change takes effect immediately. To seamlessly upgrade your application server to a new version, install a new version on a different port. After checking for a quality installation, edit /etc/httpd.maps to point to the new server. After the file is written the next request the server processes will be redirected to the new installation.
< 
< =head2 Blocking IP Addresses
< 
< Contributed to the mailing list on 12/18/03 by Randal L. Schwartz
< The following rewrite code blocks IP addresses:
< 
<   RewriteCond /web/site/var/blocked/REMOTE_ADDR-%{REMOTE_ADDR} -f
<   RewriteRule .* http://YOUR-HOST-BLOCKED-FOR-EXCESSIVE-CONSUMPTION [redirect,last]
< 
< To block IP address 10.1.2.3, simply touch
< 
<   /web/site/var/blocked/REMOTE_ADDR-10.1.2.3
< 
< This has an advantage over Apache parsing a long file of addresses in that the OS is better at a file lookup.
< 




-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: introducing new code with no perceived user delays?

Posted by "Randal L. Schwartz" <me...@stonehenge.com>.
>>>>> "Fred" == Fred Moyer <fr...@taperfriendlymusic.org> writes:

Fred> < =head2 Blocking IP Addresses
Fred> <
Fred> < The following rewrite code blocks IP addresses:
Fred> <
Fred> <   RewriteCond /web/site/var/blocked/REMOTE_ADDR-%{REMOTE_ADDR} -f
Fred> <   RewriteRule .* http://YOUR-HOST-BLOCKED-FOR-EXCESSIVE-CONSUMPTION
Fred> [redirect,last]
Fred> <
Fred> < To block IP address 10.1.2.3, simply touch
Fred> <
Fred> <   /web/site/var/blocked/REMOTE_ADDR-10.1.2.3
Fred> <
Fred> < This has an advantage over Apache parsing a long file of addresses in
Fred> that the OS is better at a file lookup.

Hey, that looks familiar.  Do I get credit? :)

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<me...@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: introducing new code with no perceived user delays?

Posted by Fred Moyer <fr...@taperfriendlymusic.org>.
On Thu, 2003-12-18 at 04:15, Stas Bekman wrote:
> Rob Nagler wrote:
> > Brian Hirt writes:
> > 
> >>Once everything is running on the 
> >>new mod perl, i shut down the original mod perl application.  There is 
> >>about 0.5 seconds of lag while the proxy reloads the rules.
> > 
> > 
> > We use proxies, but instead of modifying the proxy config, we use a
> > Rewrite rule of the form:
> > 
> >     RewriteMap maps txt:/etc/httpd.maps
> >     RewriteRule ^(.*) http://${maps:appserver}$1 [proxy]
> > 
> > When we release, we edit the /etc/httpd.maps file to point to a
> > different port or machine.
> > 
> >     appserver foo.com:9999
> > 
> > mod_rewrite rereads (or checks mtime of) the file on every request so
> > the change takes effect immediately.
> 
> That sounds like something that we want to have in the mod_perl documentation. 
> Would someone come up with a doc patch? I think the perfect place for this 
> would be: http://perl.apache.org/docs/1.0/guide/scenario.html

I had some free time last night so I wrote up a patch to
http://perl.apache.org/docs/1.0/guide/scenario.pod.orig. It includes
some of these list threads and splits up the examples so they are a bit
clearer to someone reading it for the first time.  Hope this can be
useful.

1657,1659d1655
< Example code for using mod_rewrite with mod_perl application servers.
Several examples were taken from the mailing list.
<
< =head2 Rewriting Requests Based on File Extension
<
1693c1689
< =head2 Internet Exporer 5 favicon.ico 404
---
> More examples:
1699,1700d1694
< =head2 Hiding Extensions for Dynamic Pages
<
1705,1706d1698
< =head2 Serving Static Content Locally and Rewriting Everything Else
<
1749,1777d1740
< =head2 Upgrading mod_perl Heavy Application Instances
<
< When using a light/heavy separation method one of the challenges of
running a production environment is being able to upgrade to newer
versions of mod_perl or your own application. The following method can
be used without having to do a server restart.
<
< Add the following rewrite rule to your httpd.conf file:
<
<   RewriteEngine On
<   RewriteMap maps txt:/etc/httpd.maps
<   RewriteRule ^(.*) http://${maps:appserver}$1 [proxy]
<
< Create the file /etc/httpd.maps and add the following entry:
<
<   appserver foo.com:9999
<
< Mod_rewrite rereads (or checks the mtime of) the file on every request
so the change takes effect immediately. To seamlessly upgrade your
application server to a new version, install a new version on a
different port. After checking for a quality installation, edit
/etc/httpd.maps to point to the new server. After the file is written
the next request the server processes will be redirected to the new
installation.
<
< =head2 Blocking IP Addresses
<
< The following rewrite code blocks IP addresses:
<
<   RewriteCond /web/site/var/blocked/REMOTE_ADDR-%{REMOTE_ADDR} -f
<   RewriteRule .* http://YOUR-HOST-BLOCKED-FOR-EXCESSIVE-CONSUMPTION
[redirect,last]
<
< To block IP address 10.1.2.3, simply touch
<
<   /web/site/var/blocked/REMOTE_ADDR-10.1.2.3
<
< This has an advantage over Apache parsing a long file of addresses in
that the OS is better at a file lookup.
<


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: introducing new code with no perceived user delays?

Posted by Stas Bekman <st...@stason.org>.
Rob Nagler wrote:
> Brian Hirt writes:
> 
>>Once everything is running on the 
>>new mod perl, i shut down the original mod perl application.  There is 
>>about 0.5 seconds of lag while the proxy reloads the rules.
> 
> 
> We use proxies, but instead of modifying the proxy config, we use a
> Rewrite rule of the form:
> 
>     RewriteMap maps txt:/etc/httpd.maps
>     RewriteRule ^(.*) http://${maps:appserver}$1 [proxy]
> 
> When we release, we edit the /etc/httpd.maps file to point to a
> different port or machine.
> 
>     appserver foo.com:9999
> 
> mod_rewrite rereads (or checks mtime of) the file on every request so
> the change takes effect immediately.

That sounds like something that we want to have in the mod_perl documentation. 
Would someone come up with a doc patch? I think the perfect place for this 
would be: http://perl.apache.org/docs/1.0/guide/scenario.html

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: Re[2]: introducing new code with no perceived user delays?

Posted by "Randal L. Schwartz" <me...@stonehenge.com>.
>>>>> "justin" == justin  <ju...@dslr.net> writes:

justin> The last time I traced mod_rewrite on the front-end, and map files,
justin> discovered it opens, reads, and closes, the file *every single request*.
justin> I use a map like that to block IPs fast, but thought that was rather
justin> inefficient? should I not be concerned about such trifles?

I have this code in my mod_rewrite:

  RewriteCond /web/stonehenge/var/blocked/REMOTE_ADDR-%{REMOTE_ADDR} -f
  RewriteRule .* http://YOUR-HOST-BLOCKED-FOR-EXCESSIVE-CONSUMPTION [redirect,last]

Then, when I want to block 10.1.2.3, I simply touch

        /web/stonehenge/var/blocked/REMOTE_ADDR-10.1.2.3

I figure the OS is better at a file lookup than apache is at parsing
a long file. :)

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<me...@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: Re[2]: introducing new code with no perceived user delays?

Posted by Rob Nagler <na...@bivio.biz>.
justin writes:
> The last time I traced mod_rewrite on the front-end, and map files,
> discovered it opens, reads, and closes, the file *every single request*.
> I use a map like that to block IPs fast, but thought that was rather
> inefficient? should I not be concerned about such trifles?

We haven't found it to be a problem.  Inefficiency can be measured.
Is the server load average too high?  Is the disk the bottle neck?
And, inefficiency can be fixed.  If the disk is the bottle neck, write
the map to a ram disk.  If the load average is too high, it may be
there are other contention issues, find them and fix them.

Rob

-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re[2]: introducing new code with no perceived user delays?

Posted by justin <ju...@dslr.net>.
The last time I traced mod_rewrite on the front-end, and map files,
discovered it opens, reads, and closes, the file *every single request*.
I use a map like that to block IPs fast, but thought that was rather
inefficient? should I not be concerned about such trifles?

-Justin

RN> Brian Hirt writes:
>> Once everything is running on the 
>> new mod perl, i shut down the original mod perl application.  There is 
>> about 0.5 seconds of lag while the proxy reloads the rules.

RN> We use proxies, but instead of modifying the proxy config, we use a
RN> Rewrite rule of the form:

RN>     RewriteMap maps txt:/etc/httpd.maps
RN>     RewriteRule ^(.*) http://${maps:appserver}$1 [proxy]

RN> When we release, we edit the /etc/httpd.maps file to point to a
RN> different port or machine.

RN>     appserver foo.com:9999

RN> mod_rewrite rereads (or checks mtime of) the file on every request so
RN> the change takes effect immediately.

RN> Rob


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: introducing new code with no perceived user delays?

Posted by Rob Nagler <na...@bivio.biz>.
Brian Hirt writes:
> Once everything is running on the 
> new mod perl, i shut down the original mod perl application.  There is 
> about 0.5 seconds of lag while the proxy reloads the rules.

We use proxies, but instead of modifying the proxy config, we use a
Rewrite rule of the form:

    RewriteMap maps txt:/etc/httpd.maps
    RewriteRule ^(.*) http://${maps:appserver}$1 [proxy]

When we release, we edit the /etc/httpd.maps file to point to a
different port or machine.

    appserver foo.com:9999

mod_rewrite rereads (or checks mtime of) the file on every request so
the change takes effect immediately.

Rob

-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: introducing new code with no perceived user delays?

Posted by Brian Hirt <bh...@mobygames.com>.
The way i do this is having a lightweight apache proxy on port 80.   My 
mod perl application listens on a different port and receives requests 
from the proxy.  When i need to roll out new code, i start up a 2nd mod 
perl application on a 3rd port, update the rules for the proxy to go to 
the new code and reload the rules.  Once everything is running on the 
new mod perl, i shut down the original mod perl application.  There is 
about 0.5 seconds of lag while the proxy reloads the rules.     Rolling 
out a new release in the fashion is a simple process that can be coded 
into a very simple shell script and executed with a single command.

There are also other reasons to have a proxy -> mod perl set up which i 
wont go into right now.

--brian

On Dec 17, 2003, at 5:08 PM, justin wrote:

>
> One registry module checks for a new file before a request starts.
> Another Registry class does no checking at all. Apache::Reload checks a
> file on disk before the request, or ALL module timestamps. None of 
> these
> solutions are satisfactory. If code is rolled out, 10+ modperl 
> processes
> all reloading it at the same time will halt forward motion for 30
> seconds or more (depends on code size, init time, and so on).
>
> Is it possible to design something that checks in the EXIT phase, and
> not every request, just randomly, say, every N times, so all servers
> don't freeze up .. if they find a named 're-cycle' file is newer than
> their own life-span, then they child_exit ? Ideally, they eval or 
> syntax
> check the new module(s), before deciding to child_exit.
>
> If they find they do not eval without error, they erase the re-cycle
> command file and try not to exit until an admin realizes their mistake
> ;)
>
> would such a module be a logical plug-in replacement for the Registry
> handler, a sub-class, or something else? I'd like to write it, but am
> interested to know if I'm barking up the wrong tree.
>
> thanks -Justin
>
>
> -- 
> Reporting bugs: http://perl.apache.org/bugs/
> Mail list info: http://perl.apache.org/maillist/modperl.html


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: introducing new code with no perceived user delays?

Posted by Stas Bekman <st...@stason.org>.
justin wrote:
[...]
> by the way, is modperl2 on apache2 quite ok for heavy production use by
> now?

As long as you test your applications before you upgrade your production 
server, chances are that you will be just fine. If not, let us know what are 
the problems and we will fix them.

You will want to use mod_perl 1.99_12 (which we plan to release next week), as 
I have just plugged one important memory leak (in the IO system), which 
existed in the last few releases.

 > is an upgraade recommended for performance reasons at least?

A few people reported a much better performance under mp2, but you will have 
to test your app and see for yourself.
http://perl.apache.org/docs/1.0/guide/performance.html#Benchmarking_Response_Times
A report of how did mp2 scored vs. mp1 would be a useful thing to receive.

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re[2]: introducing new code with no perceived user delays?

Posted by justin <ju...@dslr.net>.
Well what I do, for some reason I can't remember, is to preload
everything (all CPAN etc modules) *except* dev code, in startup.pl

Then the children end up loading dev code on the first request and
getting fatter ..

I might be wrong but most of my VM usage after 10 minutes heavy running
appears to be temporary data from certain infrequent urls that suck in
many rows, pushing the size up over time, so I'm not sure parent
preloading the main code base would save me that much memory.. and then
I'd really have to kick the server in one go to get it to go over to new
code.. and lose the ability to do any phase-in of code at all..

thanks as usual.

by the way, is modperl2 on apache2 quite ok for heavy production use by
now? is an upgraade recommended for performance reasons at least?

-Justin

SB> justin wrote:
SB> [...]
>> SB> Second, you could write a similar to Apache::SizeLimit handler module, which 
>> SB> will determine when to terminate the process, but the problem is that you need 
>> SB> to reload the parent server anyway if you don't rely on Apache::Reload.
>> 
>> Penny drops -- I can do this in my own exit handler, no? where I am
>> doing housekeeping, and possible child termination anyway. I could even
>> syntax check the new code to stop corrupt files or *cough* programmer
>> mistakes.

SB> Of course.

>> Since I'm still using Registry, I'd disable its checking of
>> file timestamps. 

SB> Oops, sorry, I've completely forgotten that Apache::Registry does this 
SB> reloading. You can use Apache::RegistryBB which doesn't do that or write 
SB> another subclass as you have suggested. So yes, it's relevant that you are 
SB> using Registry.

SB>  > But why do I need to worry about the parent server, if
SB>  > the parent server has a fixed code base (a fixed set of library use's in
SB>  > startup.pl and not the "code" as such)?

SB> I've assumed that you preload everything at the server startup (You did know 
SB> that you can preload registry scripts [1], to improve memory sharing, didn't 
SB> you?) in which case the parent server which contains the preloaded data will 
SB> still keep the old code and new child process will "inherit" this old code. Of 
SB> course if you don't preload scripts, than you are fine.

SB> 1)
SB> http://perl.apache.org/docs/1.0/guide/performance.html#Preloading_Registry_Scripts_at_Server_Startup
SB> http://perl.apache.org/search/swish.cgi?query=Apache%3A%3ARegistryLoader&sbm=&submit=search

SB> __________________________________________________________________
SB> Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
SB> http://stason.org/     mod_perl Guide ---> http://perl.apache.org
SB> mailto:stas@stason.org http://use.perl.org http://apacheweek.com
SB> http://modperlbook.org http://apache.org   http://ticketmaster.com


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: introducing new code with no perceived user delays?

Posted by Stas Bekman <st...@stason.org>.
justin wrote:
[...]
> SB> Second, you could write a similar to Apache::SizeLimit handler module, which 
> SB> will determine when to terminate the process, but the problem is that you need 
> SB> to reload the parent server anyway if you don't rely on Apache::Reload.
> 
> Penny drops -- I can do this in my own exit handler, no? where I am
> doing housekeeping, and possible child termination anyway. I could even
> syntax check the new code to stop corrupt files or *cough* programmer
> mistakes.

Of course.

> Since I'm still using Registry, I'd disable its checking of
> file timestamps. 

Oops, sorry, I've completely forgotten that Apache::Registry does this 
reloading. You can use Apache::RegistryBB which doesn't do that or write 
another subclass as you have suggested. So yes, it's relevant that you are 
using Registry.

 > But why do I need to worry about the parent server, if
 > the parent server has a fixed code base (a fixed set of library use's in
 > startup.pl and not the "code" as such)?

I've assumed that you preload everything at the server startup (You did know 
that you can preload registry scripts [1], to improve memory sharing, didn't 
you?) in which case the parent server which contains the preloaded data will 
still keep the old code and new child process will "inherit" this old code. Of 
course if you don't preload scripts, than you are fine.

1)
http://perl.apache.org/docs/1.0/guide/performance.html#Preloading_Registry_Scripts_at_Server_Startup
http://perl.apache.org/search/swish.cgi?query=Apache%3A%3ARegistryLoader&sbm=&submit=search

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re[2]: introducing new code with no perceived user delays?

Posted by justin <ju...@dslr.net>.
SB> What you want to do is to shutdown servers if some timestamp file
SB> has changed, but you want to do it at random. First of all don't you
SB> think it's a problem  if different servers will run different code
SB> sets?

Yes, sure, for changes where old and new running at the same time, I
would be careful to do a full apachectl restart. But I'm after ease of
doing little changes. Cosmetic ones, mainly. Which since I am
refactoring a lot at present, are most of them. Dozens a day. I like to
see them in production, and get feedback immediately, rather than work
in isolation on a "v2" system for days then roll out a huge list of
them.

SB> Second, you could write a similar to Apache::SizeLimit handler module, which 
SB> will determine when to terminate the process, but the problem is that you need 
SB> to reload the parent server anyway if you don't rely on Apache::Reload.

Penny drops -- I can do this in my own exit handler, no? where I am
doing housekeeping, and possible child termination anyway. I could even
syntax check the new code to stop corrupt files or *cough* programmer
mistakes. Since I'm still using Registry, I'd disable its checking of
file timestamps. But why do I need to worry about the parent server, if
the parent server has a fixed code base (a fixed set of library use's in
startup.pl and not the "code" as such)?

SB> The only realistic solution I can think of is to subclass Apache::Reload not 
SB> to reload the modules when the timestamp file has changed, unless your rule 
SB> applies (e.g. randomize it, or whatever). You lose the shared memory benefit 
SB> here and better off not use Apache::Reload in production.

thanks!


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html


Re: introducing new code with no perceived user delays?

Posted by Stas Bekman <st...@stason.org>.
justin wrote:
> One registry module checks for a new file before a request starts.
> Another Registry class does no checking at all. Apache::Reload checks a
> file on disk before the request, or ALL module timestamps. None of these
> solutions are satisfactory. If code is rolled out, 10+ modperl processes
> all reloading it at the same time will halt forward motion for 30
> seconds or more (depends on code size, init time, and so on).
> 
> Is it possible to design something that checks in the EXIT phase, and
> not every request, just randomly, say, every N times, so all servers
> don't freeze up .. if they find a named 're-cycle' file is newer than
> their own life-span, then they child_exit ? Ideally, they eval or syntax
> check the new module(s), before deciding to child_exit.
> 
> If they find they do not eval without error, they erase the re-cycle
> command file and try not to exit until an admin realizes their mistake
> ;)
> 
> would such a module be a logical plug-in replacement for the Registry
> handler, a sub-class, or something else? I'd like to write it, but am
> interested to know if I'm barking up the wrong tree.

This has nothing to do with Registry. You just happen to use Registry, but the 
issue is applicable for any module running under mod_perl.

What you want to do is to shutdown servers if some timestamp file has changed, 
but you want to do it at random. First of all don't you think it's a problem 
if different servers will run different code sets?

Second, you could write a similar to Apache::SizeLimit handler module, which 
will determine when to terminate the process, but the problem is that you need 
to reload the parent server anyway if you don't rely on Apache::Reload.

The only realistic solution I can think of is to subclass Apache::Reload not 
to reload the modules when the timestamp file has changed, unless your rule 
applies (e.g. randomize it, or whatever). You lose the shared memory benefit 
here and better off not use Apache::Reload in production.

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


-- 
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html