PerlModule lifetime / threads

[Some Guy <te...@gmail.com>]

Hey all,

I am trying to port a module I wrote in C to one that uses mod_perl.  The C
module would spawn a thread in the child_init process and poll a file.  The
file polled contained data that the header parser phase would use.  I
protected the data via a RW Lock.

Now when doing this in mod_perl I see the following things:

if I have a ChildInitHandler, any changes I make to globals are not
reflected in a HeaderParserHandler.  My globals do not have the :shared
attribute - must they?

I've had better luck getting rid of the ChildInit directive and initializing
globals like a normal script.  Then my handler routine can see what I've
initialized.  I can even spawn a thread.  Unfortunately, changes to a shared
array that my thread makes are not picked up by the handler.

I'm definitely missing something here as I'm approaching this from a C
module perspective where I had access to the right hooks and knew what
process I'd be running in.  Is there a way to do what I want in mod_perl -
namely have a HeaderParserHandler that uses data for some logic where that
data is updated from some other thread that I spawn?

Thanks,

SB

Re: PerlModule lifetime / threads

[Perrin Harkins <pe...@elem.com>]

On Thu, May 20, 2010 at 11:17 AM, Some Guy <te...@gmail.com> wrote:
> If the MPM is threaded, a rwlock is used where the cleanup obtains the
> writelock and request handlers are obtaining a read lock.

You don't need to do this.  Variables are not shared between perl
threads unless you make them shared.  If you avoid this, your code
will work with the more common prefork MPM as well.

> I threw a sleep in the cleanup to see what happens, and it appears to delay
> sending the results of the next request.

It doesn't do that for me.  Maybe you're running with only one
process/thread/interpreter?

- Perrin

Re: PerlModule lifetime / threads

[Some Guy <te...@gmail.com>]

I tried the cleanup handler solution, and threw a sleep in there for ha
has.  Basically I do the following in my handler:

Register a cleanup on the request's pool
Check if I need to do something to the user agent if it is found in an array
that is backed by a data file
If it's not in the array, decline the request, else, do my thing.
In the cleanup:
Check if the file has changed since i last checked
Reload the file into the array if necessary

If the MPM is threaded, a rwlock is used where the cleanup obtains the
writelock and request handlers are obtaining a read lock.

I threw a sleep in the cleanup to see what happens, and it appears to delay
sending the results of the next request.  I issued a request, then issued
another.  I didn't get the results of the second request until the sleep
from the first returned.  No locks used by the request handler are held
during sleep time, and the MPM is threaded.  I had expected the request to
be flushed to the client before the cleanup code is run.  Is this wrong?

Thanks,

SB

On Thu, May 20, 2010 at 9:58 AM, Some Guy <te...@gmail.com> wrote:

> This module is meant for other parties to use, so I can't define what MPM
> they use.  I would like it to work in either case.  I don't mind throwing in
> locking logic based on the results of querying whether the MPM is threaded
> or not - I just want to better understand that scenario and how it applies
> to modules written for mod_perl.
>
> Thanks,
>
> SB
>   On Thu, May 20, 2010 at 9:20 AM, Michael Peters <mp...@plusthree.com>wrote:
>
>> On 05/20/2010 08:54 AM, Some Guy wrote:
>>
>>> The cleanup handler is a brilliant idea.  That removes the need for a
>>> polling thread.  If I attach the cleanup to the request's pool and run
>>> in a threaded MPM, this led to contention for the globals that I'm
>>> updating in C.  Is this the case with PerlModules, or can I get away
>>> without locking them?
>>>
>>
>> Why use a threaded MPM? Is there something else you're running that relies
>> on it? We generally recommend doing with the prefork mpm because you're
>> running in share-nothing mode which performs better (and uses less memory on
>> decent operating systems).
>>
>> --
>> Michael Peters
>> Plus Three, LP
>>
>
>

Re: PerlModule lifetime / threads

[Some Guy <te...@gmail.com>]

This module is meant for other parties to use, so I can't define what MPM
they use.  I would like it to work in either case.  I don't mind throwing in
locking logic based on the results of querying whether the MPM is threaded
or not - I just want to better understand that scenario and how it applies
to modules written for mod_perl.

Thanks,

SB
On Thu, May 20, 2010 at 9:20 AM, Michael Peters <mp...@plusthree.com>wrote:

> On 05/20/2010 08:54 AM, Some Guy wrote:
>
>> The cleanup handler is a brilliant idea.  That removes the need for a
>> polling thread.  If I attach the cleanup to the request's pool and run
>> in a threaded MPM, this led to contention for the globals that I'm
>> updating in C.  Is this the case with PerlModules, or can I get away
>> without locking them?
>>
>
> Why use a threaded MPM? Is there something else you're running that relies
> on it? We generally recommend doing with the prefork mpm because you're
> running in share-nothing mode which performs better (and uses less memory on
> decent operating systems).
>
> --
> Michael Peters
> Plus Three, LP
>

Re: PerlModule lifetime / threads

[Michael Peters <mp...@plusthree.com>]

On 05/20/2010 08:54 AM, Some Guy wrote:
> The cleanup handler is a brilliant idea.  That removes the need for a
> polling thread.  If I attach the cleanup to the request's pool and run
> in a threaded MPM, this led to contention for the globals that I'm
> updating in C.  Is this the case with PerlModules, or can I get away
> without locking them?

Why use a threaded MPM? Is there something else you're running that 
relies on it? We generally recommend doing with the prefork mpm because 
you're running in share-nothing mode which performs better (and uses 
less memory on decent operating systems).

-- 
Michael Peters
Plus Three, LP

Re: PerlModule lifetime / threads

[Some Guy <te...@gmail.com>]

The cleanup handler is a brilliant idea.  That removes the need for a
polling thread.  If I attach the cleanup to the request's pool and run in a
threaded MPM, this led to contention for the globals that I'm updating in
C.  Is this the case with PerlModules, or can I get away without locking
them?

Thanks,

SB

On Wed, May 19, 2010 at 6:13 PM, Perrin Harkins <pe...@elem.com> wrote:

> On Wed, May 19, 2010 at 4:37 PM, Some Guy <te...@gmail.com> wrote:
> > if I have a ChildInitHandler, any changes I make to globals are not
> > reflected in a HeaderParserHandler.  My globals do not have the :shared
> > attribute - must they?
>
> Only if you're trying to update them from a different thread.  If
> there aren't multiple threads involved, you can update globals from
> anywhere.  If that's not working for you there's probably a mistake in
> the way you're referring to the variables.  If you post some code we
> may be able to help.
>
> > Is there a way to do what I want in mod_perl -
> > namely have a HeaderParserHandler that uses data for some logic where
> that
> > data is updated from some other thread that I spawn?
>
> A more common approach with perl is to make a handler that runs in an
> early phase which reloads the data as needed, and not use a separate
> thread.  If you're concerned about reloading the data adding too much
> delay to a web request, you could do this in a cleanup handler.
>
> - Perrin
>

Re: PerlModule lifetime / threads

[Perrin Harkins <pe...@elem.com>]

On Wed, May 19, 2010 at 4:37 PM, Some Guy <te...@gmail.com> wrote:
> if I have a ChildInitHandler, any changes I make to globals are not
> reflected in a HeaderParserHandler.  My globals do not have the :shared
> attribute - must they?

Only if you're trying to update them from a different thread.  If
there aren't multiple threads involved, you can update globals from
anywhere.  If that's not working for you there's probably a mistake in
the way you're referring to the variables.  If you post some code we
may be able to help.

> Is there a way to do what I want in mod_perl -
> namely have a HeaderParserHandler that uses data for some logic where that
> data is updated from some other thread that I spawn?

A more common approach with perl is to make a handler that runs in an
early phase which reloads the data as needed, and not use a separate
thread.  If you're concerned about reloading the data adding too much
delay to a web request, you could do this in a cleanup handler.

- Perrin