You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Gordon Henriksen <go...@iclub.com> on 2002/03/07 18:42:41 UTC
PerlFreshRestart, mod_perl DSO, and Apache::StatINC
I'm looking at how to best avoid downtime during a code upgrade, as we often
do spot releases of critical code fixes and we're getting to the usage level
that I don't want to interrupt service to deploy that code. At the same
time, I want to avoid running 200 stat()'s per request for all of the loaded
modules.
We're presently running in this configuration:
Apache 1.3.22
static mod_perl 1.26
Apache::StatINC <--Want to get rid of it.
PerlFreshRestart OFF
I see three options open to me:
1. static mod_perl w/ PerlFreshRestart
Reloads %INC.
downside: Heresay claims historical instablity.
2. dynamic mod_perl
Tears down & cleans up Perl interpreter on graceful restart.
downside: Heresay claims historical instablity.
3. static mod_perl w/ Apache::StatInc
Runs many stat()'s per request.
downside: Runs many stat()'s per request.
Aside from the historical instability, the second option strikes me as the
cleanest and most robust. How has the current stability of these mechanisms?
Is it enterprise-worthy? I'm variously running on RedHat Linux 7.0 and 7.1.
--
Gordon Henriksen
IT & Engineering
ICLUBcentral Inc.
gordon@iclub.com
Re: PerlFreshRestart, mod_perl DSO, and Apache::StatINC
Posted by Geoffrey Young <ge...@modperlcookbook.org>.
Gordon Henriksen wrote:
>
> I'm looking at how to best avoid downtime during a code upgrade, as we often
> do spot releases of critical code fixes and we're getting to the usage level
> that I don't want to interrupt service to deploy that code. At the same
> time, I want to avoid running 200 stat()'s per request for all of the loaded
> modules.
look into the touchfile option in Apache::Reload - that might help
some.
--Geoff
Re: PerlFreshRestart, mod_perl DSO, and Apache::StatINC
Posted by Perrin Harkins <pe...@elem.com>.
Geoffrey Young wrote:
> we do that frequently here - 7 servers behind a BigIP. I've always
> wondered, though, whether this approach is foolproof for major
> upgrades for applications that maintain state - since a user might
> have a session created using a new-code box, then hit an old-code box
> on the next page view. it takes us many minutes to work through
> restarting the entire array.
>
> were you ever concerned about something like that?
We also used BigIP, with the sticky load-balancing option on. (Well, we
used two, and only the application servers were sticky. It didn't
matter which proxy/web server you went to.) This prevents the problem
you're talking about.
Of course if the upgrade involves changing some shared resource like a
database as well, you have to take the site off-line while you do it. I
suppose it's possible to rig up something crazy with multiple databases
and synchronization, but it's just not worth it.
- Perrin
Re: PerlFreshRestart, mod_perl DSO, and Apache::StatINC
Posted by Geoffrey Young <ge...@modperlcookbook.org>.
>
> The best way I've found to deal with this problem is to have multiple
> servers behind a load-balancer and do a "rolling" restart. If you have
> servers A and B, you take A out of the load balancer temporarilly,
> upgrade it, add it back in, take B out, upgrade it, add it back in.
> Using this technique, we were able to smoothly upgrade production
> servers on a very busy cluster of machines during normal business hours
> while customers were on the site.
we do that frequently here - 7 servers behind a BigIP. I've always
wondered, though, whether this approach is foolproof for major
upgrades for applications that maintain state - since a user might
have a session created using a new-code box, then hit an old-code box
on the next page view. it takes us many minutes to work through
restarting the entire array.
were you ever concerned about something like that?
--Geoff
Re: PerlFreshRestart, mod_perl DSO, and Apache::StatINC
Posted by Perrin Harkins <pe...@elem.com>.
Gordon Henriksen wrote:
> I see three options open to me:
>
> 1. static mod_perl w/ PerlFreshRestart
> Reloads %INC.
> downside: Heresay claims historical instablity.
>
> 2. dynamic mod_perl
> Tears down & cleans up Perl interpreter on graceful restart.
> downside: Heresay claims historical instablity.
>
> 3. static mod_perl w/ Apache::StatInc
> Runs many stat()'s per request.
> downside: Runs many stat()'s per request.
Frankly, those options all suck for anything other than development
servers. A production server on a busy site needs to be fully stopped
and restarted when you upgrade your code. Doing anything else is just
asking for trouble (strange closure issues, for example) and will trash
your shared memory to boot.
The best way I've found to deal with this problem is to have multiple
servers behind a load-balancer and do a "rolling" restart. If you have
servers A and B, you take A out of the load balancer temporarilly,
upgrade it, add it back in, take B out, upgrade it, add it back in.
Using this technique, we were able to smoothly upgrade production
servers on a very busy cluster of machines during normal business hours
while customers were on the site.
- Perrin
Re: PerlFreshRestart, mod_perl DSO, and Apache::StatINC
Posted by Perrin Harkins <pe...@elem.com>.
> We had been using Option 1 for a long time & we had
> absolutely no problems
But doesn't it totally wreck your shared memory? For me that would make
it unusable. I usually get a pretty large percentage of memory to be
shared and count on that for getting maximum capacity from each box.
- Perrin
Re: PerlFreshRestart, mod_perl DSO, and Apache::StatINC
Posted by Sreeji K Das <sr...@yahoo.com>.
We had been using Option 1 for a long time & we had
absolutely no problems (with
mod_perl-1.19/Apache-1.3.14/Perl-5.005).
However on upgrading to mod_perl-1.26, we were getting
hell lot of errors. I have tracked this to a bug in
perl_util.c & on fixing this PerlFreshRestart works
w/o any problems. I have posted this details 2/3 weeks
back but haven't got any reply.
I had to go for option 1, since I was not sure about
DSO & StatINC does stats() for every request that
comes in - not too good. (even with a touchfile, it
has to do stat() !)
Sreeji
--- Gordon Henriksen <go...@iclub.com> wrote: > I'm
looking at how to best avoid downtime during a
> code upgrade, as we often
> do spot releases of critical code fixes and we're
> getting to the usage level
> that I don't want to interrupt service to deploy
> that code. At the same
> time, I want to avoid running 200 stat()'s per
> request for all of the loaded
> modules.
>
> We're presently running in this configuration:
>
> Apache 1.3.22
> static mod_perl 1.26
> Apache::StatINC <--Want to get rid of it.
> PerlFreshRestart OFF
>
> I see three options open to me:
>
> 1. static mod_perl w/ PerlFreshRestart
> Reloads %INC.
> downside: Heresay claims historical
> instablity.
>
> 2. dynamic mod_perl
> Tears down & cleans up Perl interpreter on
> graceful restart.
> downside: Heresay claims historical
> instablity.
>
> 3. static mod_perl w/ Apache::StatInc
> Runs many stat()'s per request.
> downside: Runs many stat()'s per request.
>
> Aside from the historical instability, the second
> option strikes me as the
> cleanest and most robust. How has the current
> stability of these mechanisms?
> Is it enterprise-worthy? I'm variously running on
> RedHat Linux 7.0 and 7.1.
>
> --
>
> Gordon Henriksen
> IT & Engineering
> ICLUBcentral Inc.
> gordon@iclub.com
>
__________________________________________________
Do You Yahoo!?
Everything you'll ever need on one web page
from News and Sport to Email and Music Charts
http://uk.my.yahoo.com