You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@perl.apache.org by Jeremy Redburn <jr...@freeshell.org> on 2005/01/04 20:09:42 UTC
Re-using tipools outside mod_perl / apache
Hello,
I've been looking for advice (perlmonks, p5p, colleagues) on maintaining
a persistent pool of Perl interpreters, and all the advice seems to come
back to mod_perl 2. While I was hoping to find a simpler example, it
seems I will need to make my way through your codebase.
I was hoping some of you might have some advice in terms of decoupling
the tipool and interpreters work from apache and mod_perl itself so that
it can be used on its own. Even better would be a simple reduced case
used to test the interpreters pool.
Any advice will be greatly appreciated.
- Jeremy
ps. If anyone is interested, this work is part of some work I'm doing
with the Pegasus CIMOM (openpegasus.org). Pegasus as it stands allows
you to write providers in C and C++. I have enabled Perl support, but it
requires a lock on the Perl interpreter while in use and is thus not all
that useful. I'm looking to implement a pool of Perl interpreters so as
to enable external callbacks to the system from the Perl providers and
multiple calls to Perl providers simultaneously.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: Re-using tipools outside mod_perl / apache
Posted by Jeremy Redburn <jr...@freeshell.org>.
Thanks very much for your quick reply. I've got a similar set of
examples to your's below, but have been having trouble building the pool
required to keep track of the multiple interpreters.
After a brief look at modperl_interp.[ch] and modperl_types.h, it looks
like there are enough references to Apache structures and mod_perl
server config that I am probably better off starting from a blank page
and working to build up to a fully functioning pool. My initial attempts
were to deconstruct the existing mod_perl code until I had reduced it to
something that worked for me, but I'm starting to think that was the
wrong approach.
Thanks again.
Jeremy
Stas Bekman wrote:
> Jeremy Redburn wrote:
>
>> Hello,
>>
>> I've been looking for advice (perlmonks, p5p, colleagues) on
>> maintaining a persistent pool of Perl interpreters, and all the advice
>> seems to come back to mod_perl 2. While I was hoping to find a simpler
>> example, it seems I will need to make my way through your codebase.
>>
>> I was hoping some of you might have some advice in terms of decoupling
>> the tipool and interpreters work from apache and mod_perl itself so
>> that it can be used on its own. Even better would be a simple reduced
>> case used to test the interpreters pool.
>
>
> Jeremy you can find a simple case of having two interpreters (w/o using
> our code) below [1]
>
>> Any advice will be greatly appreciated.
>
>
> Doing that should be as simple as taking the code in
> modperl-2.0/src/modules/perl/modperl_interp.[ch]
> and using it where you need it. (I think you may need to get some
> structs from modperl_types.h). You can see examples on how it's used by
> grepping for functions in the other files under src/modules/perl
>
> Feel free to ask any questions if you have after you've looked at the
> code. Make sure to get the latest svn:
> http://perl.apache.org/download/source.html#Development_mod_perl_2_0_Source_Distribution
>
>
>> ps. If anyone is interested, this work is part of some work I'm doing
>> with the Pegasus CIMOM (openpegasus.org). Pegasus as it stands allows
>> you to write providers in C and C++. I have enabled Perl support, but
>> it requires a lock on the Perl interpreter while in use and is thus
>> not all that useful. I'm looking to implement a pool of Perl
>> interpreters so as to enable external callbacks to the system from the
>> Perl providers and multiple calls to Perl providers simultaneously.
>
>
> I'm planning to restart soon my work on DBI::Pool so some of the
> concepts might be re-used there, but since DBI::Pool will manage
> connection pools it'll not involve perl internally, the users will
> provide the interpreters (e.g. ithreads).
>
> [1] this is just a test program I've used to reproduce some problem in
> perl note that it doesn't do cloning and no context switching. see [2]
> below for another program that does do that
>
> /* pool.c */
>
> #include <EXTERN.h>
> #include <perl.h>
>
> /*
> * gcc -o pool pool.c `perl -MExtUtils::Embed -e ccopts -e ldopts`
> -Wall -g
> */
>
> int main()
> {
> char *argv[] = {"perl", "-e", "warn", NULL};
> int argc = 3;
>
> PerlInterpreter *my_perl1 = perl_alloc();
> PerlInterpreter *my_perl2;
> SV *sv;
>
> perl_construct(my_perl1);
> perl_parse(my_perl1, NULL, argc, argv, 0);
> perl_run(my_perl1);
> sv = Perl_newSVpv(my_perl1, "foo", 3);
> warn(SvPVX(sv));
>
> my_perl2 = perl_alloc();
> perl_construct(my_perl2);
> perl_parse(my_perl2, NULL, argc, argv, 0);
> perl_run(my_perl2);
>
> Perl_sv_free(my_perl2, sv);
>
> perl_destruct(my_perl2);
> perl_free(my_perl2);
>
> Perl_set_context(my_perl1);
> perl_destruct(my_perl1);
> perl_free(my_perl1);
>
> return 0;
> }
>
> [2] this one does a proper context switching:
>
> /* clone.c */
> #include <EXTERN.h>
> #include <perl.h>
>
> /*
> * gcc -o clone clone.c `perl -MExtUtils::Embed -e ccopts -e ldopts`
> -Wall -g
> */
>
> #define TEST "sub foo {}"
>
> int main(int argc, char **argv, char **env)
> {
> char *embedding[] = { "", "-le", "0" };
> PerlInterpreter *my_perl = perl_alloc();
> PerlInterpreter *perl2;
>
> PERL_SET_CONTEXT(my_perl);
> perl_construct(my_perl);
>
> perl_parse(my_perl, NULL, 3, embedding, (char **)NULL);
>
> ENTER;SAVETMPS;
> Perl_eval_pv(my_perl, TEST, TRUE); /* loaded only by the first perl */
> FREETMPS;LEAVE;
>
> perl2 = perl_clone(my_perl, CLONEf_KEEP_PTR_TABLE);
>
> PERL_SET_CONTEXT(perl2);
> perl_destruct(perl2);
> perl_free(perl2);
>
> PERL_SET_CONTEXT(my_perl);
> perl_destruct(my_perl);
> perl_free(my_perl);
>
> exit(0);
> }
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: Re-using tipools outside mod_perl / apache
Posted by Jeremy Redburn <jr...@freeshell.org>.
Nevermind. My apologies for sending something to the list before I'd
really thought it through -- this flu has gotten to me worse than I'd
thought.
Jeremy Redburn wrote:
> Stas and others on p5p,
>
> Sorry for my earlier reply -- I meant to reply to this thread and
> mistakenly replied via p5p.
>
>
> In any event, I've done some work towards implementing the pool of
> interpreters, based on the mp2 work. The issue I've run into is that I'm
> not quite sure how to handle 'my_perl'. I used to have:
>
> #define my_perl __interpreter
>
> in my code when I had a single interpreter, but now with the pool of
> interpreters, I'm at a loss as to how to handle defining my_perl.
>
> If anyone has any pointers, they would be greatly appreciated.
>
> Jeremy
>
> Stas Bekman wrote:
>
>> Jeremy Redburn wrote:
>>
>>> Hello,
>>>
>>> I've been looking for advice (perlmonks, p5p, colleagues) on
>>> maintaining a persistent pool of Perl interpreters, and all the
>>> advice seems to come back to mod_perl 2. While I was hoping to find a
>>> simpler example, it seems I will need to make my way through your
>>> codebase.
>>>
>>> I was hoping some of you might have some advice in terms of
>>> decoupling the tipool and interpreters work from apache and mod_perl
>>> itself so that it can be used on its own. Even better would be a
>>> simple reduced case used to test the interpreters pool.
>>
>>
>>
>> Jeremy you can find a simple case of having two interpreters (w/o
>> using our code) below [1]
>>
>>> Any advice will be greatly appreciated.
>>
>>
>>
>> Doing that should be as simple as taking the code in
>> modperl-2.0/src/modules/perl/modperl_interp.[ch]
>> and using it where you need it. (I think you may need to get some
>> structs from modperl_types.h). You can see examples on how it's used
>> by grepping for functions in the other files under src/modules/perl
>>
>> Feel free to ask any questions if you have after you've looked at the
>> code. Make sure to get the latest svn:
>> http://perl.apache.org/download/source.html#Development_mod_perl_2_0_Source_Distribution
>>
>>
>>> ps. If anyone is interested, this work is part of some work I'm doing
>>> with the Pegasus CIMOM (openpegasus.org). Pegasus as it stands allows
>>> you to write providers in C and C++. I have enabled Perl support, but
>>> it requires a lock on the Perl interpreter while in use and is thus
>>> not all that useful. I'm looking to implement a pool of Perl
>>> interpreters so as to enable external callbacks to the system from
>>> the Perl providers and multiple calls to Perl providers simultaneously.
>>
>>
>>
>> I'm planning to restart soon my work on DBI::Pool so some of the
>> concepts might be re-used there, but since DBI::Pool will manage
>> connection pools it'll not involve perl internally, the users will
>> provide the interpreters (e.g. ithreads).
>>
>> [1] this is just a test program I've used to reproduce some problem in
>> perl note that it doesn't do cloning and no context switching. see [2]
>> below for another program that does do that
>>
>> /* pool.c */
>>
>> #include <EXTERN.h>
>> #include <perl.h>
>>
>> /*
>> * gcc -o pool pool.c `perl -MExtUtils::Embed -e ccopts -e ldopts`
>> -Wall -g
>> */
>>
>> int main()
>> {
>> char *argv[] = {"perl", "-e", "warn", NULL};
>> int argc = 3;
>>
>> PerlInterpreter *my_perl1 = perl_alloc();
>> PerlInterpreter *my_perl2;
>> SV *sv;
>>
>> perl_construct(my_perl1);
>> perl_parse(my_perl1, NULL, argc, argv, 0);
>> perl_run(my_perl1);
>> sv = Perl_newSVpv(my_perl1, "foo", 3);
>> warn(SvPVX(sv));
>>
>> my_perl2 = perl_alloc();
>> perl_construct(my_perl2);
>> perl_parse(my_perl2, NULL, argc, argv, 0);
>> perl_run(my_perl2);
>>
>> Perl_sv_free(my_perl2, sv);
>>
>> perl_destruct(my_perl2);
>> perl_free(my_perl2);
>>
>> Perl_set_context(my_perl1);
>> perl_destruct(my_perl1);
>> perl_free(my_perl1);
>>
>> return 0;
>> }
>>
>> [2] this one does a proper context switching:
>>
>> /* clone.c */
>> #include <EXTERN.h>
>> #include <perl.h>
>>
>> /*
>> * gcc -o clone clone.c `perl -MExtUtils::Embed -e ccopts -e ldopts`
>> -Wall -g
>> */
>>
>> #define TEST "sub foo {}"
>>
>> int main(int argc, char **argv, char **env)
>> {
>> char *embedding[] = { "", "-le", "0" };
>> PerlInterpreter *my_perl = perl_alloc();
>> PerlInterpreter *perl2;
>>
>> PERL_SET_CONTEXT(my_perl);
>> perl_construct(my_perl);
>>
>> perl_parse(my_perl, NULL, 3, embedding, (char **)NULL);
>>
>> ENTER;SAVETMPS;
>> Perl_eval_pv(my_perl, TEST, TRUE); /* loaded only by the first
>> perl */
>> FREETMPS;LEAVE;
>>
>> perl2 = perl_clone(my_perl, CLONEf_KEEP_PTR_TABLE);
>>
>> PERL_SET_CONTEXT(perl2);
>> perl_destruct(perl2);
>> perl_free(perl2);
>>
>> PERL_SET_CONTEXT(my_perl);
>> perl_destruct(my_perl);
>> perl_free(my_perl);
>>
>> exit(0);
>> }
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
> For additional commands, e-mail: dev-help@perl.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: Re-using tipools outside mod_perl / apache
Posted by Jeremy Redburn <jr...@freeshell.org>.
Stas and others on p5p,
Sorry for my earlier reply -- I meant to reply to this thread and
mistakenly replied via p5p.
In any event, I've done some work towards implementing the pool of
interpreters, based on the mp2 work. The issue I've run into is that I'm
not quite sure how to handle 'my_perl'. I used to have:
#define my_perl __interpreter
in my code when I had a single interpreter, but now with the pool of
interpreters, I'm at a loss as to how to handle defining my_perl.
If anyone has any pointers, they would be greatly appreciated.
Jeremy
Stas Bekman wrote:
> Jeremy Redburn wrote:
>
>> Hello,
>>
>> I've been looking for advice (perlmonks, p5p, colleagues) on
>> maintaining a persistent pool of Perl interpreters, and all the advice
>> seems to come back to mod_perl 2. While I was hoping to find a simpler
>> example, it seems I will need to make my way through your codebase.
>>
>> I was hoping some of you might have some advice in terms of decoupling
>> the tipool and interpreters work from apache and mod_perl itself so
>> that it can be used on its own. Even better would be a simple reduced
>> case used to test the interpreters pool.
>
>
> Jeremy you can find a simple case of having two interpreters (w/o using
> our code) below [1]
>
>> Any advice will be greatly appreciated.
>
>
> Doing that should be as simple as taking the code in
> modperl-2.0/src/modules/perl/modperl_interp.[ch]
> and using it where you need it. (I think you may need to get some
> structs from modperl_types.h). You can see examples on how it's used by
> grepping for functions in the other files under src/modules/perl
>
> Feel free to ask any questions if you have after you've looked at the
> code. Make sure to get the latest svn:
> http://perl.apache.org/download/source.html#Development_mod_perl_2_0_Source_Distribution
>
>
>> ps. If anyone is interested, this work is part of some work I'm doing
>> with the Pegasus CIMOM (openpegasus.org). Pegasus as it stands allows
>> you to write providers in C and C++. I have enabled Perl support, but
>> it requires a lock on the Perl interpreter while in use and is thus
>> not all that useful. I'm looking to implement a pool of Perl
>> interpreters so as to enable external callbacks to the system from the
>> Perl providers and multiple calls to Perl providers simultaneously.
>
>
> I'm planning to restart soon my work on DBI::Pool so some of the
> concepts might be re-used there, but since DBI::Pool will manage
> connection pools it'll not involve perl internally, the users will
> provide the interpreters (e.g. ithreads).
>
> [1] this is just a test program I've used to reproduce some problem in
> perl note that it doesn't do cloning and no context switching. see [2]
> below for another program that does do that
>
> /* pool.c */
>
> #include <EXTERN.h>
> #include <perl.h>
>
> /*
> * gcc -o pool pool.c `perl -MExtUtils::Embed -e ccopts -e ldopts`
> -Wall -g
> */
>
> int main()
> {
> char *argv[] = {"perl", "-e", "warn", NULL};
> int argc = 3;
>
> PerlInterpreter *my_perl1 = perl_alloc();
> PerlInterpreter *my_perl2;
> SV *sv;
>
> perl_construct(my_perl1);
> perl_parse(my_perl1, NULL, argc, argv, 0);
> perl_run(my_perl1);
> sv = Perl_newSVpv(my_perl1, "foo", 3);
> warn(SvPVX(sv));
>
> my_perl2 = perl_alloc();
> perl_construct(my_perl2);
> perl_parse(my_perl2, NULL, argc, argv, 0);
> perl_run(my_perl2);
>
> Perl_sv_free(my_perl2, sv);
>
> perl_destruct(my_perl2);
> perl_free(my_perl2);
>
> Perl_set_context(my_perl1);
> perl_destruct(my_perl1);
> perl_free(my_perl1);
>
> return 0;
> }
>
> [2] this one does a proper context switching:
>
> /* clone.c */
> #include <EXTERN.h>
> #include <perl.h>
>
> /*
> * gcc -o clone clone.c `perl -MExtUtils::Embed -e ccopts -e ldopts`
> -Wall -g
> */
>
> #define TEST "sub foo {}"
>
> int main(int argc, char **argv, char **env)
> {
> char *embedding[] = { "", "-le", "0" };
> PerlInterpreter *my_perl = perl_alloc();
> PerlInterpreter *perl2;
>
> PERL_SET_CONTEXT(my_perl);
> perl_construct(my_perl);
>
> perl_parse(my_perl, NULL, 3, embedding, (char **)NULL);
>
> ENTER;SAVETMPS;
> Perl_eval_pv(my_perl, TEST, TRUE); /* loaded only by the first perl */
> FREETMPS;LEAVE;
>
> perl2 = perl_clone(my_perl, CLONEf_KEEP_PTR_TABLE);
>
> PERL_SET_CONTEXT(perl2);
> perl_destruct(perl2);
> perl_free(perl2);
>
> PERL_SET_CONTEXT(my_perl);
> perl_destruct(my_perl);
> perl_free(my_perl);
>
> exit(0);
> }
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: Re-using tipools outside mod_perl / apache
Posted by Stas Bekman <st...@stason.org>.
Jeremy Redburn wrote:
> Hello,
>
> I've been looking for advice (perlmonks, p5p, colleagues) on maintaining
> a persistent pool of Perl interpreters, and all the advice seems to come
> back to mod_perl 2. While I was hoping to find a simpler example, it
> seems I will need to make my way through your codebase.
>
> I was hoping some of you might have some advice in terms of decoupling
> the tipool and interpreters work from apache and mod_perl itself so that
> it can be used on its own. Even better would be a simple reduced case
> used to test the interpreters pool.
Jeremy you can find a simple case of having two interpreters (w/o using
our code) below [1]
> Any advice will be greatly appreciated.
Doing that should be as simple as taking the code in
modperl-2.0/src/modules/perl/modperl_interp.[ch]
and using it where you need it. (I think you may need to get some structs
from modperl_types.h). You can see examples on how it's used by grepping
for functions in the other files under src/modules/perl
Feel free to ask any questions if you have after you've looked at the
code. Make sure to get the latest svn:
http://perl.apache.org/download/source.html#Development_mod_perl_2_0_Source_Distribution
> ps. If anyone is interested, this work is part of some work I'm doing
> with the Pegasus CIMOM (openpegasus.org). Pegasus as it stands allows
> you to write providers in C and C++. I have enabled Perl support, but it
> requires a lock on the Perl interpreter while in use and is thus not all
> that useful. I'm looking to implement a pool of Perl interpreters so as
> to enable external callbacks to the system from the Perl providers and
> multiple calls to Perl providers simultaneously.
I'm planning to restart soon my work on DBI::Pool so some of the concepts
might be re-used there, but since DBI::Pool will manage connection pools
it'll not involve perl internally, the users will provide the interpreters
(e.g. ithreads).
[1] this is just a test program I've used to reproduce some problem in
perl note that it doesn't do cloning and no context switching. see [2]
below for another program that does do that
/* pool.c */
#include <EXTERN.h>
#include <perl.h>
/*
* gcc -o pool pool.c `perl -MExtUtils::Embed -e ccopts -e ldopts` -Wall -g
*/
int main()
{
char *argv[] = {"perl", "-e", "warn", NULL};
int argc = 3;
PerlInterpreter *my_perl1 = perl_alloc();
PerlInterpreter *my_perl2;
SV *sv;
perl_construct(my_perl1);
perl_parse(my_perl1, NULL, argc, argv, 0);
perl_run(my_perl1);
sv = Perl_newSVpv(my_perl1, "foo", 3);
warn(SvPVX(sv));
my_perl2 = perl_alloc();
perl_construct(my_perl2);
perl_parse(my_perl2, NULL, argc, argv, 0);
perl_run(my_perl2);
Perl_sv_free(my_perl2, sv);
perl_destruct(my_perl2);
perl_free(my_perl2);
Perl_set_context(my_perl1);
perl_destruct(my_perl1);
perl_free(my_perl1);
return 0;
}
[2] this one does a proper context switching:
/* clone.c */
#include <EXTERN.h>
#include <perl.h>
/*
* gcc -o clone clone.c `perl -MExtUtils::Embed -e ccopts -e ldopts` -Wall -g
*/
#define TEST "sub foo {}"
int main(int argc, char **argv, char **env)
{
char *embedding[] = { "", "-le", "0" };
PerlInterpreter *my_perl = perl_alloc();
PerlInterpreter *perl2;
PERL_SET_CONTEXT(my_perl);
perl_construct(my_perl);
perl_parse(my_perl, NULL, 3, embedding, (char **)NULL);
ENTER;SAVETMPS;
Perl_eval_pv(my_perl, TEST, TRUE); /* loaded only by the first perl */
FREETMPS;LEAVE;
perl2 = perl_clone(my_perl, CLONEf_KEEP_PTR_TABLE);
PERL_SET_CONTEXT(perl2);
perl_destruct(perl2);
perl_free(perl2);
PERL_SET_CONTEXT(my_perl);
perl_destruct(my_perl);
perl_free(my_perl);
exit(0);
}
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: Re-using tipools outside mod_perl / apache
Posted by Perrin Harkins <pe...@elem.com>.
On Tue, 2005-01-04 at 14:09 -0500, Jeremy Redburn wrote:
> I've been looking for advice (perlmonks, p5p, colleagues) on maintaining
> a persistent pool of Perl interpreters, and all the advice seems to come
> back to mod_perl 2.
We're very tightly tied to Apache 2 for the processing model. If you
want something more independent, you might try looking at PPerl or
SpeedyCGI.
For a much easier route, you might want to just make HTTP requests to a
mod_perl server to run your code persistently, or implement the FastCGI
protocol and use FastCGI to manage a pool of interpreters.
- Perrin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org