Sharing information across Apache child processes

[Rajalakshmi Iyer <ra...@blismedia.com>]

Hello,

I have a requirement whereby my application's configuration information
(comprising a few complex data structures) needs to be shared across the
various Apache child processes.

Currently, the configuration is being individually loaded by each child
process, which makes it hard for configuration changes to propagate.

What is the best way / place to have a common configuration for the
application?

Please advise.

Thanks in advance,
R.

-- 

@BlisMedia <http://twitter.com/BlisMedia>

www.blismedia.com <http://blismedia.com>

This email and any attachments to it may be confidential and are intended solely 
for the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent 
those of BlisMedia Ltd, a company registered in England and Wales with 
registered number 06455773. Its registered office is 3rd Floor, 101 New 
Cavendish St, London, W1W 6XH, United Kingdom.

If you are not the intended recipient of this email, you must neither take 
any action based upon its contents, nor copy or show it to anyone. Please 
contact the sender if you believe you have received this email in error. 

Re: Sharing information across Apache child processes

[Joshua Marantz <jm...@google.com>]

There's a variation on this theme you might consider.  You can use a few
counters in shared memory, and save config information in a file.  If one
of the child processes learns of a config update (say by handling an HTTP
request for one from somewhere) it can update the file (atomically, say,
via write-to-temp-and-rename) and bump a shared-memory counter (atomically
via shared-memory mutex).  Other child processes can quickly poll on
counter-changes and re-read config files when they change.

The advantage of this approach over storing the whole config in
shared-memory is that we only put a fixed number of integers in shared
memory, rather than our whole configuration structure, which might be large
or vary in size depending on the config.

mod_pagespeed uses this technique
<https://code.google.com/p/modpagespeed/source/browse/trunk/src/pagespeed/kernel/cache/purge_context.h>.
Note there is no apr code directly in that file as it's abstracted behind
some C++ interfaces, but 'Variable' is really an int64's worth of mutexed
shared memory in operation.

-Josh


-

On Tue, Sep 30, 2014 at 9:20 AM, Abdi Abdirahman <ab...@gmail.com>
wrote:

> On 09/29/2014 05:14 PM, Sorin Manolache wrote:
>
>> On 2014-09-29 13:39, Rajalakshmi Iyer wrote:
>>
>>> Hello,
>>>
>>> I have a requirement whereby my application's configuration information
>>> (comprising a few complex data structures) needs to be shared across the
>>> various Apache child processes.
>>>
>>> Currently, the configuration is being individually loaded by each child
>>> process, which makes it hard for configuration changes to propagate.
>>>
>>> What is the best way / place to have a common configuration for the
>>> application?
>>>
>>> Please advise.
>>>
>>
>> I suppose you want to update the configuration without running "apache2
>> -k graceful" (or "apache2ctl graceful").
>>
>> In this case you could use a segment of memory that is shared across the
>> apache children. You'll have to create the shared segment memory before the
>> parent forks its children (for example in post_config). The shared memory
>> is then inherited by the forked children.
>>
>> You'll need a method to update the contents of the shared memory segment
>> and a multiple-readers-single-writer inter-process exclusion mechanism in
>> order to safely read and write from the shared segment.
>>
>> Sorin
>>
>>  Hello All,
>
> You can check out mod_cluster approach (https://github.com/
> modcluster/mod_cluster/tree/master/native/mod_proxy_cluster) - it uses
> shared memory as data storage and additional thread per process to check
> updates in storage. This way you'll keep per-process module config in sync
> with shared data.
> Not perfect solution though, but I doubt that there is any flawless way to
> share config structures.
>
> ________________
> Best regards,
> Abdi A.
>

Re: Sharing information across Apache child processes

[Abdi Abdirahman <ab...@gmail.com>]

On 09/29/2014 05:14 PM, Sorin Manolache wrote:
> On 2014-09-29 13:39, Rajalakshmi Iyer wrote:
>> Hello,
>>
>> I have a requirement whereby my application's configuration information
>> (comprising a few complex data structures) needs to be shared across the
>> various Apache child processes.
>>
>> Currently, the configuration is being individually loaded by each child
>> process, which makes it hard for configuration changes to propagate.
>>
>> What is the best way / place to have a common configuration for the
>> application?
>>
>> Please advise.
>
> I suppose you want to update the configuration without running 
> "apache2 -k graceful" (or "apache2ctl graceful").
>
> In this case you could use a segment of memory that is shared across 
> the apache children. You'll have to create the shared segment memory 
> before the parent forks its children (for example in post_config). The 
> shared memory is then inherited by the forked children.
>
> You'll need a method to update the contents of the shared memory 
> segment and a multiple-readers-single-writer inter-process exclusion 
> mechanism in order to safely read and write from the shared segment.
>
> Sorin
>
Hello All,

You can check out mod_cluster approach 
(https://github.com/modcluster/mod_cluster/tree/master/native/mod_proxy_cluster) 
- it uses shared memory as data storage and additional thread per 
process to check updates in storage. This way you'll keep per-process 
module config in sync with shared data.
Not perfect solution though, but I doubt that there is any flawless way 
to share config structures.

________________
Best regards,
Abdi A.

Re: Sharing information across Apache child processes

[Sorin Manolache <so...@gmail.com>]

On 2014-09-29 13:39, Rajalakshmi Iyer wrote:
> Hello,
>
> I have a requirement whereby my application's configuration information
> (comprising a few complex data structures) needs to be shared across the
> various Apache child processes.
>
> Currently, the configuration is being individually loaded by each child
> process, which makes it hard for configuration changes to propagate.
>
> What is the best way / place to have a common configuration for the
> application?
>
> Please advise.

I suppose you want to update the configuration without running "apache2 
-k graceful" (or "apache2ctl graceful").

In this case you could use a segment of memory that is shared across the 
apache children. You'll have to create the shared segment memory before 
the parent forks its children (for example in post_config). The shared 
memory is then inherited by the forked children.

You'll need a method to update the contents of the shared memory segment 
and a multiple-readers-single-writer inter-process exclusion mechanism 
in order to safely read and write from the shared segment.

Sorin