You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Matthew Westcott <ma...@torchbox.com> on 2005/02/22 16:02:35 UTC
Sharing variables across Apache threads with threads::shared
Hi,
I'm having trouble with the basics of sharing a variable across
multiple Apache threads using threads::shared . The only code snippet
I've been able to find on this subject is from the list archives from
last month:
http://gossamer-threads.com/lists/modperl/modperl/77651#77651
I'm stuck at a much earlier stage than that, so I'm a bit lost in the
details of that code - I think I'm working along the same sort of lines
though.
My code is as follows:
---
package MyApache::MagicNumber;
use strict;
use warnings;
use threads;
use threads::shared;
use Apache::Const -compile => qw(OK);
use Apache::RequestRec ();
use Apache::RequestIO ();
my $magic_number : shared;
sub post_config {
$magic_number = int(rand(100));
my $now = gmtime(time);
`echo "$now: MagicNumber initialised to $magic_number by PID $$" >>
/tmp/magicnumber.log`;
return Apache::OK;
}
sub handler {
my $r = shift;
$r->content_type('text/html');
if ($r->args =~ /increment/) {
$magic_number++;
my $now = gmtime(time);
`echo "$now: magic_number incremented to $magic_number by PID
$$" >> /tmp/magicnumber.log`;
}
print "<html><body><p>the magic number
is</p><h1>$magic_number</h1></body></html>";
return Apache::OK;
}
1;
---
This is set up in httpd.conf like this:
PerlModule MyApache::MagicNumber
PerlPostConfigHandler MyApache::MagicNumber::post_config
<Location /magicnumber>
SetHandler perl-script
PerlResponseHandler MyApache::MagicNumber
</Location>
Repeatedly calling http://localhost/magicnumber?increment indicates
that each thread is keeping its own copy of $magic_number - they are
all starting from the same initial value set in post_config, at least.
magicnumber.log contains the following:
Tue Feb 22 14:06:14 2005: MagicNumber initialised to 63 by PID 17452
Tue Feb 22 14:06:15 2005: MagicNumber initialised to 2 by PID 17454
Tue Feb 22 14:06:20 2005: magic_number incremented to 3 by PID 17459
Tue Feb 22 14:06:21 2005: magic_number incremented to 4 by PID 17459
Tue Feb 22 14:06:22 2005: magic_number incremented to 5 by PID 17459
Tue Feb 22 14:06:23 2005: magic_number incremented to 6 by PID 17459
Tue Feb 22 14:06:24 2005: magic_number incremented to 7 by PID 17459
Tue Feb 22 14:06:38 2005: magic_number incremented to 3 by PID 17462
Tue Feb 22 14:06:38 2005: magic_number incremented to 4 by PID 17462
Tue Feb 22 14:06:39 2005: magic_number incremented to 5 by PID 17462
Tue Feb 22 14:06:41 2005: magic_number incremented to 6 by PID 17462
Tue Feb 22 14:06:45 2005: magic_number incremented to 3 by PID 17458
Tue Feb 22 14:06:46 2005: magic_number incremented to 4 by PID 17458
Tue Feb 22 14:06:50 2005: magic_number incremented to 7 by PID 17462
I've checked that threads::shared is working correctly outside of
mod_perl (via the examples on
http://www.perl.com/pub/a/2002/06/11/threads.html ) - any suggestions
where I'm going wrong?
I'm running mod_perl 1.99_12 on Apache 2.0.47 with Perl 5.8.0.
Thanks,
- Matthew
Re: Sharing variables across Apache threads with threads::shared
Posted by "Richard F. Rebel" <rr...@whenu.com>.
Hello Matt,
Some obvious things...
Are you running a threaded apache mpm such as worker? If not, no
threads, only processes, thus no sharing. You'd have to use another
sharing mechanism in this case such as sysv ipc shared memory, or mmaps.
When you use the worker mpm, the maximum number of apache threads per
process is 64. If you have 75 client connections, then you end up with
at least two processes depending on how you set up apache. Each process
will have it's own copy of the variable that will be incremented
independently of the others. Shared variables will not be shared across
PID's ($$).
You are not locking your variables before you increment them. As in
most things, perl doesn't believe in having to use a shotgun to keep
people out of your living room... Thusly, the locking of shared
variables is advisary, not enforced. So, you can corrupt your variable
by having many threads blindly updating it. Call lock for the variable
in the block where you execute it.
BTW, I have never tried using the post config cycle for apache, I don't
know if that's an appropriate place to established shared vars. I do
all of mine in classes loaded from my perl startup file.
If you want to print out a given thread within an apache processes view
of your shared variable, you should print out:
my $tid = ${APR::OS::thread_current()};
For some reason that returned a scalar ref in my code. This is what I
use but from what I understand more recent versions of mod_perl have
moved this elsewhere. When I want to print to the error log what a
given thread sees in a shared var I use:
{
lock($shared);
print STDERR "$$->$tid \$shared = '$shared'\n";
}
which prints out something like:
14522->234222 $shared = 'shared value'
Hope this helps you.
Richard F. Rebel
On Tue, 2005-02-22 at 15:02 +0000, Matthew Westcott wrote:
> Hi,
> I'm having trouble with the basics of sharing a variable across
> multiple Apache threads using threads::shared . The only code snippet
> I've been able to find on this subject is from the list archives from
> last month:
> http://gossamer-threads.com/lists/modperl/modperl/77651#77651
> I'm stuck at a much earlier stage than that, so I'm a bit lost in the
> details of that code - I think I'm working along the same sort of lines
> though.
>
> My code is as follows:
> ---
>
> package MyApache::MagicNumber;
>
> use strict;
> use warnings;
>
> use threads;
> use threads::shared;
>
> use Apache::Const -compile => qw(OK);
>
> use Apache::RequestRec ();
> use Apache::RequestIO ();
>
> my $magic_number : shared;
>
> sub post_config {
> $magic_number = int(rand(100));
> my $now = gmtime(time);
> `echo "$now: MagicNumber initialised to $magic_number by PID $$" >>
> /tmp/magicnumber.log`;
> return Apache::OK;
> }
>
> sub handler {
> my $r = shift;
> $r->content_type('text/html');
> if ($r->args =~ /increment/) {
> $magic_number++;
> my $now = gmtime(time);
> `echo "$now: magic_number incremented to $magic_number by PID
> $$" >> /tmp/magicnumber.log`;
> }
> print "<html><body><p>the magic number
> is</p><h1>$magic_number</h1></body></html>";
>
> return Apache::OK;
> }
>
> 1;
>
> ---
> This is set up in httpd.conf like this:
>
> PerlModule MyApache::MagicNumber
> PerlPostConfigHandler MyApache::MagicNumber::post_config
>
> <Location /magicnumber>
> SetHandler perl-script
> PerlResponseHandler MyApache::MagicNumber
> </Location>
>
> Repeatedly calling http://localhost/magicnumber?increment indicates
> that each thread is keeping its own copy of $magic_number - they are
> all starting from the same initial value set in post_config, at least.
> magicnumber.log contains the following:
>
> Tue Feb 22 14:06:14 2005: MagicNumber initialised to 63 by PID 17452
> Tue Feb 22 14:06:15 2005: MagicNumber initialised to 2 by PID 17454
> Tue Feb 22 14:06:20 2005: magic_number incremented to 3 by PID 17459
> Tue Feb 22 14:06:21 2005: magic_number incremented to 4 by PID 17459
> Tue Feb 22 14:06:22 2005: magic_number incremented to 5 by PID 17459
> Tue Feb 22 14:06:23 2005: magic_number incremented to 6 by PID 17459
> Tue Feb 22 14:06:24 2005: magic_number incremented to 7 by PID 17459
> Tue Feb 22 14:06:38 2005: magic_number incremented to 3 by PID 17462
> Tue Feb 22 14:06:38 2005: magic_number incremented to 4 by PID 17462
> Tue Feb 22 14:06:39 2005: magic_number incremented to 5 by PID 17462
> Tue Feb 22 14:06:41 2005: magic_number incremented to 6 by PID 17462
> Tue Feb 22 14:06:45 2005: magic_number incremented to 3 by PID 17458
> Tue Feb 22 14:06:46 2005: magic_number incremented to 4 by PID 17458
> Tue Feb 22 14:06:50 2005: magic_number incremented to 7 by PID 17462
>
> I've checked that threads::shared is working correctly outside of
> mod_perl (via the examples on
> http://www.perl.com/pub/a/2002/06/11/threads.html ) - any suggestions
> where I'm going wrong?
>
> I'm running mod_perl 1.99_12 on Apache 2.0.47 with Perl 5.8.0.
>
> Thanks,
> - Matthew
>
--
Richard F. Rebel
cat /dev/null > `tty`
Re: Sharing variables across Apache threads with threads::shared
Posted by Matthew Westcott <ma...@torchbox.com>.
On 22 Feb 2005, at 15:22, Richard F. Rebel wrote:
> Hello Matt,
>
> Some obvious things...
>
> Are you running a threaded apache mpm such as worker? If not, no
> threads, only processes, thus no sharing. You'd have to use another
> sharing mechanism in this case such as sysv ipc shared memory, or
> mmaps.
>
> When you use the worker mpm, the maximum number of apache threads per
> process is 64. If you have 75 client connections, then you end up with
> at least two processes depending on how you set up apache. Each
> process
> will have it's own copy of the variable that will be incremented
> independently of the others. Shared variables will not be shared
> across
> PID's ($$).
Hi Richard,
That would indeed seem to be the problem - I'm running the prefork MPM.
It sounds like I'm better off going back to the drawing board
rather than switching to worker and getting bitten by the 64 thread
limit later!
Many thanks for the advice,
- Matthew