custom background thread and module sharing a data structure

[Andrej van der Zee <an...@gmail.com>]

Hi,

I need to modify Apache and run one custom background thread. In addition,
my custom modules have to be able to share a data structure with this
background thread. Did anybody do this before? Is there an example I can use
as a starting point?

Thank you,
Andrej



-- 
Andrej van der Zee
2-40-19 Koenji-minami
Suginami-ku, Tokyo
166-0003 JAPAN
Mobile: +81-(0)80-65251092
Phone/Fax: +81-(0)3-3318-3155

Re: custom background thread and module sharing a data structure

[Andrej van der Zee <an...@gmail.com>]

Hi,

Thanks for the info.

For the worker mpm, both cross thread and cross process protection will be
> needed. apr_proc_mutex.h family supplies cross-process protection,
> apr_thread_mutex.h provides cross thread protection.
>
> You "locking" method would be a wrapper method that first obtains a process
> level lock, and then inside that lock it obtains a thread level lock.
>

Is there anything against using the interprocess- and thread-library from
C++ boost and use those instead the ones provided by the APR-library? What I
need to do is start a background thread and share some data with a custom
module. Although C++ boost is cross-platform, for now my primary target is
Linux only.

Your opinion is appreciated!

Cheers,
Andrej

Re: custom background thread and module sharing a data structure

[Andrej van der Zee <an...@gmail.com>]

Hi,



> My point is that within a single process, multiple threads can service
> requests that can end up firing your module code. If you only do process
> locking you can still have more than 1 thread executing your module code at
> the same time.
>
> Just a process level lock will *not* guarantee that only a single thread is
> executing your code within the process holding the lock. A inter thread lock
> is needed to synchronize multiple threads within that processes from
> stomping on a shared resource.
>
>
Yeah you are right, two thread in the same process in the mpm worker model
still execute the same code to access the shared inter-process resource. So
I guess I need to add the thread-based mutexes within a APR_HAS_THREADS
define, like in some other modules I have seen. Thanks for clearing this up!

Cheers,
Andrej

Re: custom background thread and module sharing a data structure

[Saju Pillai <sa...@gmail.com>]

Andrej van der Zee wrote:
> Hi,
> 
> Thanks for your comments. See below...
> 
> Worker mpm is a multithreaded, multiprocess mpm. Multiple child processes
>> host multiple worker threads that run your module code.
>>
>> If your module services 2 concurrent requests in 2 different threads in the
>> same process and both the threads need to access critical code at the same
>> time, you will have to use thread locking to prevent race conditions.
> 
> 
> Yes, I understand the concept of mpm worker. My point was slightly
> different: When I start a background thread in the post_config()-hook, it is
> created in the address space of the server, and not of the processes that
> run worker threads for handling HTTP requests. Therefore, threads that
> handle HTTP requests always execute in a different process than the
> background thread, and thus thread-based locking is not necessary for
> sharing inter-process data.


My point is that within a single process, multiple threads can service 
requests that can end up firing your module code. If you only do process 
locking you can still have more than 1 thread executing your module code 
at the same time.

Just a process level lock will *not* guarantee that only a single thread 
is executing your code within the process holding the lock. A inter 
thread lock is needed to synchronize multiple threads within that 
processes from stomping on a shared resource.


srp
-- 
http://saju.net.in

Re: custom background thread and module sharing a data structure

[Andrej van der Zee <an...@gmail.com>]

Hi,

Thanks for your comments. See below...

Worker mpm is a multithreaded, multiprocess mpm. Multiple child processes
> host multiple worker threads that run your module code.
>
> If your module services 2 concurrent requests in 2 different threads in the
> same process and both the threads need to access critical code at the same
> time, you will have to use thread locking to prevent race conditions.


Yes, I understand the concept of mpm worker. My point was slightly
different: When I start a background thread in the post_config()-hook, it is
created in the address space of the server, and not of the processes that
run worker threads for handling HTTP requests. Therefore, threads that
handle HTTP requests always execute in a different process than the
background thread, and thus thread-based locking is not necessary for
sharing inter-process data.

re: Using non APR locking mechanisms - sure they will work as long as you
> are using compatible mechanisms (like if apr_threads is based on pthreads,
> the alternate locking mechanism should work with pthreads).
>

That's good news!

Cheers,
Andrej

Re: custom background thread and module sharing a data structure

[Saju Pillai <sa...@gmail.com>]

Andrej van der Zee wrote:
> Hi Saju,
> 
> For the worker mpm, both cross thread and cross process protection will be
>> needed. apr_proc_mutex.h family supplies cross-process protection,
>> apr_thread_mutex.h provides cross thread protection.
>>
>> You "locking" method would be a wrapper method that first obtains a process
>> level lock, and then inside that lock it obtains a thread level lock.
>>
> 
> In my specific case, do I really need thread-level locking in addition to
> process-level locking? Since I am running a background thread that is
> started in post_config(), worker-threads in the mpm worker model are running
> in a different process then my background thread anyway. So I guess
> process-level locking should suffice, or am I overlooking something?

Worker mpm is a multithreaded, multiprocess mpm. Multiple child 
processes host multiple worker threads that run your module code.

If your module services 2 concurrent requests in 2 different threads in 
the same process and both the threads need to access critical code at 
the same time, you will have to use thread locking to prevent race 
conditions.

re: Using non APR locking mechanisms - sure they will work as long as 
you are using compatible mechanisms (like if apr_threads is based on 
pthreads, the alternate locking mechanism should work with pthreads).


srp
-- 
http://saju.net.in

Re: custom background thread and module sharing a data structure

[Andrej van der Zee <an...@gmail.com>]

Hi Saju,

For the worker mpm, both cross thread and cross process protection will be
> needed. apr_proc_mutex.h family supplies cross-process protection,
> apr_thread_mutex.h provides cross thread protection.
>
> You "locking" method would be a wrapper method that first obtains a process
> level lock, and then inside that lock it obtains a thread level lock.
>

In my specific case, do I really need thread-level locking in addition to
process-level locking? Since I am running a background thread that is
started in post_config(), worker-threads in the mpm worker model are running
in a different process then my background thread anyway. So I guess
process-level locking should suffice, or am I overlooking something?

Thank you,
Andrej

Re: custom background thread and module sharing a data structure

[Saju Pillai <sa...@gmail.com>]

Sorin Manolache wrote:
> On Wed, Mar 11, 2009 at 02:08, Andrej van der Zee
> <an...@gmail.com> wrote:
>> Hi,
>>
>> I need to modify Apache and run one custom background thread. In addition,
>> my custom modules have to be able to share a data structure with this
>> background thread. Did anybody do this before? Is there an example I can use
>> as a starting point?
> 

> 
> Regarding the shm protection against inconsistency: Now here are some
> details that I don't remember. There are some mutex mechanisms in
> apr_thread_mutex and in apr_proc_mutex. The first protects against
> concurrent access of threads in the same process, the second against
> concurrent access by processes. I remember I used the first, but I
> don't remember why and how did I avoid corruption by concurrent
> accesses of different processes.
> 

For the worker mpm, both cross thread and cross process protection will 
be needed. apr_proc_mutex.h family supplies cross-process protection, 
apr_thread_mutex.h provides cross thread protection.

You "locking" method would be a wrapper method that first obtains a 
process level lock, and then inside that lock it obtains a thread level 
lock.


srp
-- 
http://saju.net.in

Re: custom background thread and module sharing a data structure

[Andrej van der Zee <an...@gmail.com>]

Hi,


> Yes, I did this. I will send you a commented source file sometimes
> today or early tomorrow (Central European Time), I don't really have
> time now.
>
>
Great, I will be waiting for that!

Thanks,
Andrej

Re: custom background thread and module sharing a data structure

[Dave Ingram <da...@dmi.me.uk>]

Hi Sorin,

Sorin Manolache wrote:
> On Wed, Mar 11, 2009 at 02:08, Andrej van der Zee
> <an...@gmail.com> wrote:
>   
>> Hi,
>>
>> I need to modify Apache and run one custom background thread. In addition,
>> my custom modules have to be able to share a data structure with this
>> background thread. Did anybody do this before? Is there an example I can use
>> as a starting point?
>>     
>
> Yes, I did this. I will send you a commented source file sometimes
> today or early tomorrow (Central European Time), I don't really have
> time now.
>   
I'm interested in something similar -- would you please send me some
example sources as well?

Thanks,


Dave

Re: custom background thread and module sharing a data structure

[Sorin Manolache <so...@gmail.com>]

On Wed, Mar 11, 2009 at 02:08, Andrej van der Zee
<an...@gmail.com> wrote:
> Hi,
>
> I need to modify Apache and run one custom background thread. In addition,
> my custom modules have to be able to share a data structure with this
> background thread. Did anybody do this before? Is there an example I can use
> as a starting point?

Yes, I did this. I will send you a commented source file sometimes
today or early tomorrow (Central European Time), I don't really have
time now.

The basic idea is to spawn the thread via the apr_thread_* functions
in the post_config hook. The post_config hook is run as root _before_
apache spawns its children and threads. So you are sure that the
thread is unique and that its handler is inherited by the spawned
children.

The post_config hook is run every time apache is restarted or its conf
reloaded (/etc/init.d/apache restart or reload). Thus, you run the
risk to start a new thread each time you reload apache's
configuration. In order to avoid that, register your
thread-stopper/thread-killer in the cleanup function of the conf pool.
The conf pool is destroyed before apache reloads/restarts, so the
thread will be killed.

Regarding the data structure:

You use the same post_config hook (to take advantage of root
privileges and the fact that it is one of the few hooks that are run
in single-process single-thread mode). You create a shared memory
segment via the apr_shm_* header/functions. You register the deleter
of the shm segment in the conf pool cleanup function. This last thing
is important as your module would not only leak memory otherwise but
it would also eat up the operating system's shm descriptors. You can
list the OS's shm segments with "ipcs -m". Check that their number
does not grow after apache reloads/restarts.

Regarding the shm protection against inconsistency: Now here are some
details that I don't remember. There are some mutex mechanisms in
apr_thread_mutex and in apr_proc_mutex. The first protects against
concurrent access of threads in the same process, the second against
concurrent access by processes. I remember I used the first, but I
don't remember why and how did I avoid corruption by concurrent
accesses of different processes.

S


-- 
A: Because it reverses the logical flow of conversation.
Q: Why is top-posting frowned upon?
A: Top-posting.
Q: What is the most annoying thing in e-mail?