You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Elizabeth Mattijsen <li...@dijkmat.nl> on 2002/08/19 22:53:54 UTC
AutoLoader bypass?
It wasn't until recently that I realized that the functionality of AutoLoader might actually be counter-productive for mod_perl, at least in a prefork MPM.
Because routines are loaded when they are requested, they may be loaded in child processes, causing duplication in memory because they are not shared. They would be shared if they would be loaded in the initialization phase before the child processes are forked. But in that phase you wouldn't call all of those routines that you might need in the child processes.
I was therefore thinking about the development of a module that would just go through %INC and check each of the modules for auto/*.al files and do() all of them (possibly limited by a hash keyed to module names with subroutine lists). And then possibly disable AutoLoader altogether, so no memory would inadvertently be lost by routines being loaded by child processes.
Does such a beast exist already? If not, would such a module make sense? What would be a good name for it?
Liz
Re: AutoLoader bypass?
Posted by Elizabeth Mattijsen <li...@dijkmat.nl>.
At 06:24 PM 8/19/02 -0700, Randy J. Ray wrote:
>>Well, my C<use AutoLoader> would be _outside_ any of the loaded modules in
>>the mod_perl startup.pl script after all the modules necessary for proper
>>execution of _your_ mod_perl environment, are loaded.
>I see... you mean to have a line like this:
> use AutoLoader preload => { module => [qw(suba subb)] };
>be responsible for both including "module" (into the caller's namespace) *and* pre-loading the specified routines? That's different that what I had interpreted from your first idea. I though that the preload specification would be when the target module issues its call to "use AutoLoader".
Actually only the preloading part. Since by default, the preload routine would look at %INC, this C<use AutoLoader> should be _after_ any other C<use> commands.
>,,, That's different that what I had interpreted from your first idea. I though that the preload specification would be when the target module issues its call to "use AutoLoader".
No, I wouldn't want module authors to change their modules...
> From this vantage point, it does make more sense, yes. Especially since module authors would no be responsible for retro-fitting to their packages. I would be interested to see if this can be done cleanly, without making AutoLoader.pm any harder to read than it currently is :-).
Well, that's just a matter of documentation, really... ;-)
>(OK, that might be asking a bit much...)
Not really, AutoLoader is not a really big module at all...
Liz
Re: AutoLoader bypass?
Posted by "Randy J. Ray" <rj...@tellme.com>.
On 2002.08.19 14:47 Elizabeth Mattijsen wrote:
> At 02:37 PM 8/19/02 -0700, Randy J. Ray wrote:
> >> use AutoLoader 'preload'; # preload anything you can find in %INC
> >> use AutoLoader preload => { module => '' }; # all from specific module
> >> use AutoLoader preload => { module => [qw(suba subb)] }; # only specific
> >>Would that make sense?
> >Problem with that approach is that it doesn't offer any control over whether
> >you are pre-loading or not. If you are going to pre-load a certain sub-set
> of
> >routines all the time, just put them above the __END__ token and don't
> burden
> >AutoLoader with new functionality at all.
>
> Well, my C<use AutoLoader> would be _outside_ any of the loaded modules in
> the mod_perl startup.pl script after all the modules necessary for proper
> execution of _your_ mod_perl environment, are loaded.
I see... you mean to have a line like this:
use AutoLoader preload => { module => [qw(suba subb)] };
be responsible for both including "module" (into the caller's namespace) *and*
pre-loading the specified routines? That's different that what I had
interpreted from your first idea. I though that the preload specification
would be when the target module issues its call to "use AutoLoader".
From this vantage point, it does make more sense, yes. Especially since
module authors would no be responsible for retro-fitting to their packages. I
would be interested to see if this can be done cleanly, without making
AutoLoader.pm any harder to read than it currently is :-).
(OK, that might be asking a bit much...)
Randy
--
-------------------------------------------------------------------------------
Randy J. Ray | Men occasionally stumble over the truth, but most of them
rjray@tellme.com | pick themselves up and hurry off as if nothing had happened.
+1 650 930-9097 | -- Sir Winston Churchill
Re: AutoLoader bypass?
Posted by Elizabeth Mattijsen <li...@dijkmat.nl>.
At 02:37 PM 8/19/02 -0700, Randy J. Ray wrote:
>> use AutoLoader 'preload'; # preload anything you can find in %INC
>> use AutoLoader preload => { module => '' }; # all from specific module
>> use AutoLoader preload => { module => [qw(suba subb)] }; # only specific
>>Would that make sense?
>Problem with that approach is that it doesn't offer any control over whether
>you are pre-loading or not. If you are going to pre-load a certain sub-set of
>routines all the time, just put them above the __END__ token and don't burden
>AutoLoader with new functionality at all.
Well, my C<use AutoLoader> would be _outside_ any of the loaded modules in the mod_perl startup.pl script after all the modules necessary for proper execution of _your_ mod_perl environment, are loaded.
>What I was suggesting was a way that I, as the writer of (picking one of my
>own modules as an example) RPC::XML::Server can incorporate something in the
>compile-time logic so that Apache::RPC::Server contains:
> use RPC::XML::Server 'compile';
>And RPC::XML::Server can have:
>sub import {
> AutoLoader::preload(__PACKAGE__, @list_o_routines)
> if (lc $_[1] eq 'compile');
>}
>(Admittedly a simplistic example, but I hope it gets the point across)
That also makes sense, but wasn't my original idea.
I'd rather make this a class method, so:
AutoLoader->preload( @list_o_routines );
and have the module derive the module name from caller(). That would at least simplify the call and it would reduce the risk of abuse.
>This way, I only pre-load routines in the cases where I need it done. Your
>suggestion is good for modules that are only ever used under mod_perl, but
>modules may be designed to work in other environments. Oh, you could manage
>to get the same affect as my idea using a BEGIN block and conditional calls
>of "use AutoLoader", but the above seems to me to be much cleaner.
I'm more interested in the modules that either work in a mod_perl environment and a "normal" Perl environment. If a module is supposed to be used in mod_perl, it just shouldn't use AutoLoader.
Liz
Re: AutoLoader bypass?
Posted by "Randy J. Ray" <rj...@tellme.com>.
On 2002.08.19 14:19 Elizabeth Mattijsen wrote:
> use AutoLoader 'preload'; # preload anything you can find in %INC
>
> use AutoLoader preload => { module => '' }; # all from specific module
>
> use AutoLoader preload => { module => [qw(suba subb)] }; # only specific
>
>
> Would that make sense?
Problem with that approach is that it doesn't offer any control over whether
you are pre-loading or not. If you are going to pre-load a certain sub-set of
routines all the time, just put them above the __END__ token and don't burden
AutoLoader with new functionality at all.
What I was suggesting was a way that I, as the writer of (picking one of my
own modules as an example) RPC::XML::Server can incorporate something in the
compile-time logic so that Apache::RPC::Server contains:
use RPC::XML::Server 'compile';
And RPC::XML::Server can have:
sub import {
AutoLoader::preload(__PACKAGE__, @list_o_routines)
if (lc $_[1] eq 'compile');
}
(Admittedly a simplistic example, but I hope it gets the point across)
This way, I only pre-load routines in the cases where I need it done. Your
suggestion is good for modules that are only ever used under mod_perl, but
modules may be designed to work in other environments. Oh, you could manage
to get the same affect as my idea using a BEGIN block and conditional calls
of "use AutoLoader", but the above seems to me to be much cleaner.
Randy
--
-------------------------------------------------------------------------------
Randy J. Ray | Men occasionally stumble over the truth, but most of them
rjray@tellme.com | pick themselves up and hurry off as if nothing had happened.
+1 650 930-9097 | -- Sir Winston Churchill
Re: AutoLoader bypass?
Posted by Elizabeth Mattijsen <li...@dijkmat.nl>.
At 02:05 PM 8/19/02 -0700, Randy J. Ray wrote:
>>Because routines are loaded when they are requested, they may be loaded in
>>child processes, causing duplication in memory because they are not shared.
>>They would be shared if they would be loaded in the initialization phase
>>before the child processes are forked. But in that phase you wouldn't call
>>all of those routines that you might need in the child processes.
>The problem I would anticipate would be in having a portable way of locating the code to load without having it executed. You could pull some functionality out of AutoLoader, but then you have code duplication.
Indeed. Most of the necessary code is in AutoLoader::import and AutoLoader::AUTOLOAD already.
>Or, an idea that just hit me, you could provide a call in the AutoLoader module that does the job for you. It would have access to all the logic already in the module, and module-writers could use it conditionally a la:
> AutoLoader::preload(__PACKAGE__, @routines)
> if $running_under_modperl;
>Where the @routines list is optional, and defaults to every *.al file found for __PACKAGE__.
I was more thinking along:
use AutoLoader; # current behaviour
use AutoLoader 'AUTOLOAD'; # import AUTOLOAD
use AutoLoader 'preload'; # preload anything you can find in %INC
use AutoLoader preload => { module => '' }; # all from specific module
use AutoLoader preload => { module => [qw(suba subb)] }; # only specific
Would that make sense?
Liz
Re: AutoLoader bypass?
Posted by "Randy J. Ray" <rj...@tellme.com>.
On 2002.08.19 13:53 Elizabeth Mattijsen wrote:
> Because routines are loaded when they are requested, they may be loaded in
> child processes, causing duplication in memory because they are not shared.
> They would be shared if they would be loaded in the initialization phase
> before the child processes are forked. But in that phase you wouldn't call
> all of those routines that you might need in the child processes.
I don't know of a module or "standardized" way of doing this, but the CGI
module offers this to the user in the form of a "compile" directive in the
imports list. It tells the module proper to pre-compile the specified list of
routines, and (as I understand it) the functionality stemmed from the kind of
situation you describe, where application developers wanted to ensure that
load-on-demand functions be pre-loaded before forking.
The problem I would anticipate would be in having a portable way of locating
the code to load without having it executed. You could pull some functionality
out of AutoLoader, but then you have code duplication.
Or, an idea that just hit me, you could provide a call in the AutoLoader
module that does the job for you. It would have access to all the logic
already in the module, and module-writers could use it conditionally a la:
AutoLoader::preload(__PACKAGE__, @routines)
if $running_under_modperl;
Where the @routines list is optional, and defaults to every *.al file found
for __PACKAGE__.
Randy
--
-------------------------------------------------------------------------------
Randy J. Ray | Men occasionally stumble over the truth, but most of them
rjray@tellme.com | pick themselves up and hurry off as if nothing had happened.
+1 650 930-9097 | -- Sir Winston Churchill