You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Michael Schout <ms...@gkg.net> on 2010/05/27 17:11:15 UTC
Re: a better way to recognize module changes
On 09/11/2009 04:26 PM, Jonathan Swartz wrote:
> I'm thinking about an improved solution to recognizing module changes in
> a running server, without restarting the server.
This thread is quite old, but it inspired me to implement a similar
strategy for dealing with module changes under mod_perl.
In my case, I was only really interested in restarting the server when
modules under a certain namespace changed (all of my app's modules are
under a single namespace, "My" for example).
My solution involved forking off a watcher process when the server
starts up. This watcher processes uses File::ChangeNotify to watch for
changes in the given modules. A nice benefit of using ::ChangeNotify,
is if you are on Linux, and have the Linux::Inotify2 modules installed,
then this happens with no cpu overhead, and changes are seen almost
instantaneous. Once a change happens, my watcher processes detaches
from apache, and reloads it (retrying some configured number of times in
case there is a syntax error on the first change).
To do this, I created a module My::Apache2::Reload. I register the
handler in httpd.conf as:
PerlPostConfigHandler My::Apache2::Reload
When apache starts up, it immediately restarts itself, so my handler has
to arrange to only start the watcher subprocess one time. I do this on
the second restart like this:
> sub handler : method {
> return Apache2::Const::OK unless Apache2::ServerUtil::restart_count() == 2;
>
> start_watcher();
>
> return Apache2::Const::OK;
> }
start_watcher is implemented as follows:
> sub start_watcher {
> unless (fork) { # child
> require File::ChangeNotify;
>
> my $module_dir = '/path/to/my/modules';
> my $config_dir = '/path/to/my/configs';
>
> my $watcher = File::ChangeNotify->instantiate_watcher(
> directories => [$module_dir, $config_dir],
> filter => qr/\.(?:pm|conf|yml)$/);
>
> while (my @events = $watcher->wait_for_events) {
> my @event_types = map { $_->type } @events;
> if (any(@event_types) eq 'modify') {
> restart_apache();
> }
> }
> }
> }
And restart_apache() is called when something that is watched changes:
> sub restart_apache {
> disconnect_from_apache();
>
> require IPC::System::Simple;
>
> for my $attempt (1 .. $MaxRestartTries) {
> if ($attempt > 1) {
> sleep $RestartDelay;
> }
>
> eval {
> IPC::System::Simple::system('apache restart command');
> CORE::exit(0);
> };
> }
>
> # give up.
> CORE::exit(0);
> }
The disconnect_from_apache() is necessary because the watcher gets
killed off if you do not close all open filehandles in the child when
the restart is attempted. Also we need to detach from the apache
process group. This is done as follows:
> sub disconnect_from_apache {
> POSIX::setsid();
>
> # close all open fds.
> my $max_fds = POSIX::sysconf(&POSIX::_SC_OPEN_MAX) // 64;
>
> for my $fd (0 .. $max_fds) {
> POSIX::close($fd);
> }
> }
And thats it. This has worked wonderfully for me. Apache notices
changes in real time, and restarts happen quickly when developing.
If there is interest in a module like this, let me know and I will
package it up and put it on CPAN (obviously I wouldn't use the name
Apache2::Reload as that is taken already :)).
Regards,
Michael Schout
Re: a better way to recognize module changes
Posted by Fred Moyer <fr...@redhotpenguin.com>.
Curiously, I have been using Apache::Reload a lot lately and it has
been working really well most of the time.
I'm wondering if maybe another way to deal with troublesome module
reloads is to subclass Apache::Reload and override the reload process
for those specific namespaces.
On Thu, May 27, 2010 at 8:11 AM, Michael Schout <ms...@gkg.net> wrote:
> On 09/11/2009 04:26 PM, Jonathan Swartz wrote:
>> I'm thinking about an improved solution to recognizing module changes in
>> a running server, without restarting the server.
>
> This thread is quite old, but it inspired me to implement a similar
> strategy for dealing with module changes under mod_perl.
>
> In my case, I was only really interested in restarting the server when
> modules under a certain namespace changed (all of my app's modules are
> under a single namespace, "My" for example).
>
> My solution involved forking off a watcher process when the server
> starts up. This watcher processes uses File::ChangeNotify to watch for
> changes in the given modules. A nice benefit of using ::ChangeNotify,
> is if you are on Linux, and have the Linux::Inotify2 modules installed,
> then this happens with no cpu overhead, and changes are seen almost
> instantaneous. Once a change happens, my watcher processes detaches
> from apache, and reloads it (retrying some configured number of times in
> case there is a syntax error on the first change).
>
> To do this, I created a module My::Apache2::Reload. I register the
> handler in httpd.conf as:
>
> PerlPostConfigHandler My::Apache2::Reload
>
> When apache starts up, it immediately restarts itself, so my handler has
> to arrange to only start the watcher subprocess one time. I do this on
> the second restart like this:
>
>> sub handler : method {
>> return Apache2::Const::OK unless Apache2::ServerUtil::restart_count() == 2;
>>
>> start_watcher();
>>
>> return Apache2::Const::OK;
>> }
>
> start_watcher is implemented as follows:
>
>> sub start_watcher {
>> unless (fork) { # child
>> require File::ChangeNotify;
>>
>> my $module_dir = '/path/to/my/modules';
>> my $config_dir = '/path/to/my/configs';
>>
>> my $watcher = File::ChangeNotify->instantiate_watcher(
>> directories => [$module_dir, $config_dir],
>> filter => qr/\.(?:pm|conf|yml)$/);
>>
>> while (my @events = $watcher->wait_for_events) {
>> my @event_types = map { $_->type } @events;
>> if (any(@event_types) eq 'modify') {
>> restart_apache();
>> }
>> }
>> }
>> }
>
> And restart_apache() is called when something that is watched changes:
>
>> sub restart_apache {
>> disconnect_from_apache();
>>
>> require IPC::System::Simple;
>>
>> for my $attempt (1 .. $MaxRestartTries) {
>> if ($attempt > 1) {
>> sleep $RestartDelay;
>> }
>>
>> eval {
>> IPC::System::Simple::system('apache restart command');
>> CORE::exit(0);
>> };
>> }
>>
>> # give up.
>> CORE::exit(0);
>> }
>
> The disconnect_from_apache() is necessary because the watcher gets
> killed off if you do not close all open filehandles in the child when
> the restart is attempted. Also we need to detach from the apache
> process group. This is done as follows:
>
>> sub disconnect_from_apache {
>> POSIX::setsid();
>>
>> # close all open fds.
>> my $max_fds = POSIX::sysconf(&POSIX::_SC_OPEN_MAX) // 64;
>>
>> for my $fd (0 .. $max_fds) {
>> POSIX::close($fd);
>> }
>> }
>
>
> And thats it. This has worked wonderfully for me. Apache notices
> changes in real time, and restarts happen quickly when developing.
>
> If there is interest in a module like this, let me know and I will
> package it up and put it on CPAN (obviously I wouldn't use the name
> Apache2::Reload as that is taken already :)).
>
> Regards,
> Michael Schout
>
Re: a better way to recognize module changes
Posted by Michael Schout <ms...@gkg.net>.
On 05/27/2010 03:04 PM, Perrin Harkins wrote:
> On Thu, May 27, 2010 at 11:11 AM, Michael Schout <ms...@gkg.net> wrote:
>> My solution involved forking off a watcher process when the server
>> starts up.
>
> Wouldn't it be simpler to start a separate daemon for this?
The project this is for has apache's sandboxed for each developer.
Every developer runs their own apache on a port number based on their
UID. By having apache fork the watcher daemon, I don't have to worry
about which apache the watcher is in charge of. Also, apache will kill
the watcher when apache is killed, so I do not need to manage the
watcher daemon at all. Apache takes care of starting and stopping it
for me.
Regards,
Michael Schout
Re: a better way to recognize module changes
Posted by Jonathan Swartz <sw...@pobox.com>.
On May 27, 2010, at 1:04 PM, Perrin Harkins wrote:
> On Thu, May 27, 2010 at 11:11 AM, Michael Schout <ms...@gkg.net>
> wrote:
>> My solution involved forking off a watcher process when the server
>> starts up.
>
> Wouldn't it be simpler to start a separate daemon for this? You could
> launch it from apachectl if you don't want to run another command.
>
Also take a look at Server::Control::Apache (apachectlp), it is easy
to extend its behavior to, e.g., launch a separate daemon.
Re: a better way to recognize module changes
Posted by Perrin Harkins <pe...@elem.com>.
On Thu, May 27, 2010 at 11:11 AM, Michael Schout <ms...@gkg.net> wrote:
> My solution involved forking off a watcher process when the server
> starts up.
Wouldn't it be simpler to start a separate daemon for this? You could
launch it from apachectl if you don't want to run another command.
- Perrin