DSO support in Rhapsody, mapped files, and other fun

[Wilfredo Sanchez <ws...@apple.com>]

  I'm adding DSO support to Rhapsody (ie. Mac OS X Server), and I'm looking for advice. The problem I has is that our dynamic linker does not (yet) support the unloading of bundles.  It seems that  
when you launch apache, the parent process reads the config, which loads all the modules.  Then the children each fork and clear the configuration, which unloads all the modules, and the re-reads the  
config, which reloads the modules. (This seems pretty inefficient, but that's for later.)

  Since I probably can't get the linker unload support done any time soon, I'd like not to reload a module that's already been loaded. I can do this by caching module paths at load time and not  
redoing ones that are already loaded. Does that seem problematic? Seems like the only time this would really hurt is when you change a module's binary, which can be worked around by doing a hard  
restart of the server.

  On another topic, we're trying to see what we can do with web server performance and so we're trying to optimze the hell out of Apache since that's our shipping server. DSO support may help  
because I've been building Apache with most of the modules turned on, since I don't want people to have to rebuild to get features, but mod_status, and so on aren't big performance winners, so I'm  
hoping that by only selecting modules you need, you can bump up performance some. (We also want to provide a WebObjects module as an add-on, and don't want to make people recompile Apache for that..)  


  So we have a MacOS 8 server that's beating out Apache on Rhapsody by a fair margin, and we think that's gotta be nuts, and the hardware's no quite fair and so on, but still, we should be able to  
stomp on MacOS 8 and NT, because we're cool and so on. And so, we got the network guys to look at the network stack, and indeed, we now are a lot better, but not we're really *stomping* on anyone, so  
we're looking harder and currently it looks like Apache is CPU bound. Now they are testing with 1.3.0 and all those static modules, so I think we can address some of that with DSO support and not  
configuring everything in.

  But Joe seems to think that Apache is opening a file (eg. index.html) mmap'ing it, and then closing and unmapping it.  We're thinking that caching these mapped files would be righteously cool, in  
that it might really bump up performance.  Joe's getting his impressions from kernel traces, and not so much from looking at Apache code, so I'm wondering if Apache has a strategy for caching mapped  
files and where I might look for it, or if it's something we'd have to contribute.

  I'm now working with 1.3.1, so once I get things ironed out I'll be resubmitting all of my patches that I sent before, but against 1.3.1, if it looks like they haven't been committed. I'll make an  
effort to break up the configure/makefile work I did before so that you don't have to fear and loath the sheer size of it.

	Thanks for the help.
	-Fred

Re: DSO support in Rhapsody, mapped files, and other fun

[Dan Jacobowitz <dr...@false.org>]

On Wed, Aug 05, 1998 at 11:03:54AM -0700, Dean Gaudet wrote:
> 
> On Wed, 5 Aug 1998, Wilfredo Sanchez wrote:
> 
> The double configuration is a feature.  Pretty much everyone stumbles into
> it at some point.  (That, and discovering that -X respects
> MaxRequestsPerChild are usually head scratchers ;) 

Awww, and I thought I was special :)


Which brings me back to the mod_perl oddities (again).

The problem, as far as I can tell, is with the initialization.  I can't
tell for certain whether or not this happening, but shouldn't a
module's init_module() function be called when it is loaded, instead of
only before the second config file re-reading?

Also, and this one has me totally puzzled, the first time mod_perl gets
loaded it looks at the symbol perl_running.  It finds a 1 there.  This
is bizarre because the only function that changes this variable has
definitely not yet been called.  Then, the second time it is loaded (is
the module actually unloaded and reloaded?) the symbol is correctly 0.

Any idea where this could be coming from?  I haven't checked for this
specific issue but I'm told that it works with no other modules loaded;
I'll retry that soon and see if perl_running is correct in that case.

Dan

Re: DSO support in Rhapsody, mapped files, and other fun

[Dean Gaudet <dg...@arctic.org>]

On Wed, 5 Aug 1998, Wilfredo Sanchez wrote:

>   I'm adding DSO support to Rhapsody (ie. Mac OS X Server), and I'm
> looking for advice. The problem I has is that our dynamic linker does
> not (yet) support the unloading of bundles.  It seems that when you
> launch apache, the parent process reads the config, which loads all the
> modules.  Then the children each fork and clear the configuration, which
> unloads all the modules, and the re-reads the config, which reloads the
> modules. (This seems pretty inefficient, but that's for later.) 

The double configuration is a feature.  Pretty much everyone stumbles into
it at some point.  (That, and discovering that -X respects
MaxRequestsPerChild are usually head scratchers ;) 

>   Since I probably can't get the linker unload support done any time
> soon, I'd like not to reload a module that's already been loaded. I can
> do this by caching module paths at load time and not redoing ones that
> are already loaded. Does that seem problematic? Seems like the only time
> this would really hurt is when you change a module's binary, which can
> be worked around by doing a hard restart of the server. 

Sounds like it would work.

>   But Joe seems to think that Apache is opening a file (eg. index.html)
> mmap'ing it, and then closing and unmapping it.  We're thinking that
> caching these mapped files would be righteously cool, in that it might
> really bump up performance.  Joe's getting his impressions from kernel
> traces, and not so much from looking at Apache code, so I'm wondering if
> Apache has a strategy for caching mapped files and where I might look
> for it, or if it's something we'd have to contribute. 

Apache is multiprocess.  Caching in every process would mean massive
numbers of different page tables... thrash thrash.  You can disable the
mmap() if your implementation isn't efficient enough.  Or you can tweak
the threshold at which it starts using mmap, search for MMAP_THRESHOLD. 
Use something like lmbench to figure out what the good threshold for your
system is.

BTW, just disabling mod_status doesn't remove the inefficiency of "Rule
STATUS=yes". 

Dean