You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Vadim Chekan <ko...@gmail.com> on 2005/11/16 09:09:20 UTC

Shared memory on Win (Was: ldap crash on exit)

Some time ago I reported a reason why Ldap module can not work on
Windows and crashes when caching is enabled. It is because shared
memory that is obtained by processes has different base address for
each process.

I thought to replace pointers with offsets everywhere but it became
really ugly. To get any pointer you have to make a function and call
it.
This made me thinking how projects like PostgreSQL which recently has
been ported to Win32 and are very shared memory oriented did it. As a
result I found that there is a function in Win API that allows to map
shared memory with desired base address!

Currently APR uses Win32 API MapViewOfFile(). But there is a
modification of this function, MapViewOfFileEx()
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/mapviewoffileex.asp
which has an additional argument
================================
lpBaseAddress
    [in] A pointer to the memory address in the calling process
address space where mapping begins.
...
    If lpBaseAddress is NULL, the operating system chooses the mapping
address. In this scenario, the function is equivalent to the
MapViewOfFile function.
================================

So, I'm quite sure APR needs to use this function to map shared memory
to the same address in all processes and make win application similar
to unix.

Vadim Chekan.

Re: Shared memory on Win (Was: ldap crash on exit)

Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Jess Holle wrote:
> Dropping the cache upon a graceful seems like a small price to pay to 
> me, but I can see others begging to differ...

I'm not thinking of dropping the cache, but allocating it process-local,
any time we see single process mode.  At least for Win32, perhaps for
other single-process platforms.  *Especially* until the cache code is
completely stable; losing one process is bad, losing the whole server
(and entering an ugly cycle of respawning children only to hit corrupt
shared cache data) is bad.

Bill

Re: Shared memory on Win (Was: ldap crash on exit)

Posted by Jess Holle <je...@ptc.com>.
Dropping the cache upon a graceful seems like a small price to pay to 
me, but I can see others begging to differ...

William A. Rowe, Jr. wrote:

> Jess Holle wrote:
>
>> I'd long since given up and been patching all the mod_*ldap stuff to 
>> pretend shared memory does not exist on Windows.
>>
>> This seems to work fine.  While it would be great to have everything 
>> similarly and well on all the platforms, what's the real downside 
>> here given that there is only one worker process on Windows?
>
>
> Actually, what's the downside if they configure one process, 400 threads
> on solaris?  Shouldn't we take the same optimization?  Doesn't Netware
> share this issue (sorry, I'm still foggy on the threads-as-processes deal
> over on the Netware server.)
>
> ap_mpm_query will tell us if we have 'sibling' processes.  Simple flag,
> simple exception.
>
> The downside I can think of off hand is that if the current cache has
> been designed to handle gracefuls - then poof, your cache doesn't 
> persist,
> and that's especially bad for anyone who's set MaxRequests 10000 to mop
> up any crufty third party module leakage.
>
> Bill

Re: Shared memory on Win (Was: ldap crash on exit)

Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Jess Holle wrote:
> I'd long since given up and been patching all the mod_*ldap stuff to 
> pretend shared memory does not exist on Windows.
> 
> This seems to work fine.  While it would be great to have everything 
> similarly and well on all the platforms, what's the real downside here 
> given that there is only one worker process on Windows?

Actually, what's the downside if they configure one process, 400 threads
on solaris?  Shouldn't we take the same optimization?  Doesn't Netware
share this issue (sorry, I'm still foggy on the threads-as-processes deal
over on the Netware server.)

ap_mpm_query will tell us if we have 'sibling' processes.  Simple flag,
simple exception.

The downside I can think of off hand is that if the current cache has
been designed to handle gracefuls - then poof, your cache doesn't persist,
and that's especially bad for anyone who's set MaxRequests 10000 to mop
up any crufty third party module leakage.

Bill

Re: Shared memory on Win (Was: ldap crash on exit)

Posted by Jess Holle <je...@ptc.com>.
I'd long since given up and been patching all the mod_*ldap stuff to 
pretend shared memory does not exist on Windows.

This seems to work fine.  While it would be great to have everything 
similarly and well on all the platforms, what's the real downside here 
given that there is only one worker process on Windows?

Also not using shared memory allowed us to keep using a local read/write 
lock rather than a global lock for a while, but maintaining this diff 
became unwieldy over time, so I gave up on this.

--
Jess Holle

Graham Leggett wrote:

> Michael Vergoz wrote:
>
>> also note that mod_auth_ldap is experimental in Apache.
>
> Only in v2.0 - mod_ldap and mod_authnz_ldap are no longer 
> experiemental in v2.2, which is imminent for release.
>
> If someone can find a fix for this it would be very cool.
>
> Regards,
> Graham
> -- 


Re: Shared memory on Win (Was: ldap crash on exit)

Posted by Graham Leggett <mi...@sharp.fm>.
Michael Vergoz wrote:

> also note that mod_auth_ldap is experimental in Apache.

Only in v2.0 - mod_ldap and mod_authnz_ldap are no longer experiemental 
in v2.2, which is imminent for release.

If someone can find a fix for this it would be very cool.

Regards,
Graham
--

Re: Shared memory on Win (Was: ldap crash on exit)

Posted by Michael Vergoz <mv...@binarysec.com>.
hi,

you can't specify a base address on Apache because portability and  
compatibilty with different other softs. And that way will not fix  
this problem.
you can load and manage a shared map in Apache but you can't expand  
it. That problem is due in part to the Apache MPM prefork/worker engine.

PostGreSQL doesn't have the same problem because it doesn't have to  
interrupt some CGI or exec programs . For example actualy PHP can't  
work on threaded environnement because its global variables and note  
that its parser isn't atomic. these problems are also present with  
Perl, Tomcat etc...

i had work for a long time about this subject and the only way i have  
found it's to build a new SHM inter-process protocol associated with  
a new MPM engine.

it's a design problem and Apache isn't the person in charge. They can  
done what i say but it's little nightmare.

also note that mod_auth_ldap is experimental in Apache.

- michael

Le 16 nov. 05 à 12:09, Vadim Chekan a écrit :

> Some time ago I reported a reason why Ldap module can not work on
> Windows and crashes when caching is enabled. It is because shared
> memory that is obtained by processes has different base address for
> each process.
>
> I thought to replace pointers with offsets everywhere but it became
> really ugly. To get any pointer you have to make a function and call
> it.
> This made me thinking how projects like PostgreSQL which recently has
> been ported to Win32 and are very shared memory oriented did it. As a
> result I found that there is a function in Win API that allows to map
> shared memory with desired base address!
>
> Currently APR uses Win32 API MapViewOfFile(). But there is a
> modification of this function, MapViewOfFileEx()
> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/ 
> fileio/fs/mapviewoffileex.asp
> which has an additional argument
> ================================
> lpBaseAddress
>     [in] A pointer to the memory address in the calling process
> address space where mapping begins.
> ...
>     If lpBaseAddress is NULL, the operating system chooses the mapping
> address. In this scenario, the function is equivalent to the
> MapViewOfFile function.
> ================================
>
> So, I'm quite sure APR needs to use this function to map shared memory
> to the same address in all processes and make win application similar
> to unix.
>
> Vadim Chekan.