You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Rainer Jung <ra...@kippdata.de> on 2010/06/15 23:03:19 UTC
MPM sizing defaults and config
The default configuration for various MPMs
differs from the example configuration file
conf/extra/httpd-mpm.conf.
Before bringing those two in sync, I want
to propose the values we want to use as new
defaults as well as for the extras configuration
file.
Prefork
======= conf default proposed
StartServers 5 5 5
MinSpareServers 5 5 3
MaxSpareServers 10 10 15
MaxClients 150 256 256
MaxRequestsPerChild 0 10000 0
Worker/Event
============ conf default proposed
StartServers 2 3 2
MinSpareThreads 25 75 25
MaxSpareThreads 75 250 100
MaxClients 150 400 400
ThreadsPerChild 25 25 25
MaxRequestsPerChild 0 10000 0
Simple
====== conf default proposed
SimpleProcCount 5 5 5?
SimpleThreadCount 5 5 5?
WinNT
===== conf default proposed
ThreadsPerChild 150 64 150
MaxRequestsPerChild 0 0 0
Netware
======= conf default proposed
StartThreads 250 50 50
MinSpareThreads 25 10 25
MaxSpareThreads 250 100 100
MaxThreads 1000 2048 1000
MaxRequestsPerChild 0 0 0
ThreadStackSize 65536 65536 65536
MaxMemFree 100 0 - (remove)
OS2
=== conf default proposed
StartServers 2 2 2
MinSpareThreads 5 5 5
MaxSpareThreads 10 10 20
MaxRequestsPerChild 0 10000 0
Any comments?
Regards,
Rainer
Re: MPM sizing defaults and config
Posted by Rainer Jung <ra...@kippdata.de>.
On 16.06.2010 04:27, William A. Rowe Jr. wrote:
> On 6/15/2010 7:27 PM, Guenter Knauf wrote:
>> Hi,
>> Am 16.06.2010 00:37, schrieb William A. Rowe Jr.:
>>>>>> Netware
>>>>>> ======= conf default proposed
>>>>>> StartThreads 250 50 50
>>>>>> MinSpareThreads 25 10 25
>>>>>> MaxSpareThreads 250 100 100
>>>>>> MaxThreads 1000 2048 1000
>>>>>> MaxRequestsPerChild 0 0 0
>>>>>> ThreadStackSize 65536 65536 65536
>>>>>> MaxMemFree 100 0 - (remove)
>>>>>
>>>>> ThreadStackSize seems a bit dicey, would rather see 128k default.
>>>>
>>>> Because it is the only MPM with non system default used, I wondered
>>>> whether there's a bit of history behind that value.
>>>
>>> I'm certain :) Guenter? Brad?
>> well, I dont think a higher ThreadStackSize can hurt much beside higher
>> memory usage (which shouldnt be an issue nowadays). I cant tell why we
>> have 64k as default, but I can tell that this is 8 times as much as what
>> our linker uses by default if you dont specify a stacksize, so brobably
>> it was just a value which Brad thought would be enough when he ported
>> 2.0 to NetWare ...; also IIRC 8k was way too less for modules like
>> mod_rewrite ... - but everything else seems to be coded cleaner so that
>> every greater memory block is aloocated rather than pushed on stack.
>> Also I can add that for mysql I use 128k too since had probs with the
>> testsuite where some tests crashed with 64k. So in general I consider a
>> higher stackzise more safe for bad coding where hughe memory blocks are
>> pushed on the stack.
>
> Ditto with win32, but ThreadStackSize should be expressing a MAX stack
> extent, not the base. Win32 will always be starting with 4kb minimum,
> but the ThreadStackSize constrains the number of threads which could be
> allocated based on max stacks.
The Windows MPM uses ThreadStackSize for the reservation of stack memory
(worker threads), it doesn't commit it. System default seems to be 1MB.
Side comment: APR on Win instead seems to set the committed size.
For Netware I can't find any documentation whether the argument to
NXContextAlloc refers to stack reservation or actually allocation,
whether to initial or to max. Could be some simple mode of a fixed stack
size, already alloated when NXContextAlloc() is called. Not sure.
For Unix we rely on APR, which uses pthreads and there it is the
*minimum* stack size that's set (unclear whether only reserved or
actually allocated). On Solaris (64Bit kernel) as an example, default
values are 1MB for 32Bit processes and 2MB for 64Bit.
Rainer
Re: MPM sizing defaults and config
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 6/15/2010 7:27 PM, Guenter Knauf wrote:
> Hi,
> Am 16.06.2010 00:37, schrieb William A. Rowe Jr.:
>>>>> Netware
>>>>> ======= conf default proposed
>>>>> StartThreads 250 50 50
>>>>> MinSpareThreads 25 10 25
>>>>> MaxSpareThreads 250 100 100
>>>>> MaxThreads 1000 2048 1000
>>>>> MaxRequestsPerChild 0 0 0
>>>>> ThreadStackSize 65536 65536 65536
>>>>> MaxMemFree 100 0 - (remove)
>>>>
>>>> ThreadStackSize seems a bit dicey, would rather see 128k default.
>>>
>>> Because it is the only MPM with non system default used, I wondered
>>> whether there's a bit of history behind that value.
>>
>> I'm certain :) Guenter? Brad?
> well, I dont think a higher ThreadStackSize can hurt much beside higher
> memory usage (which shouldnt be an issue nowadays). I cant tell why we
> have 64k as default, but I can tell that this is 8 times as much as what
> our linker uses by default if you dont specify a stacksize, so brobably
> it was just a value which Brad thought would be enough when he ported
> 2.0 to NetWare ...; also IIRC 8k was way too less for modules like
> mod_rewrite ... - but everything else seems to be coded cleaner so that
> every greater memory block is aloocated rather than pushed on stack.
> Also I can add that for mysql I use 128k too since had probs with the
> testsuite where some tests crashed with 64k. So in general I consider a
> higher stackzise more safe for bad coding where hughe memory blocks are
> pushed on the stack.
Ditto with win32, but ThreadStackSize should be expressing a MAX stack
extent, not the base. Win32 will always be starting with 4kb minimum,
but the ThreadStackSize constrains the number of threads which could be
allocated based on max stacks.
Re: MPM sizing defaults and config
Posted by Guenter Knauf <fu...@apache.org>.
Hi,
Am 16.06.2010 00:37, schrieb William A. Rowe Jr.:
>>>> Netware
>>>> ======= conf default proposed
>>>> StartThreads 250 50 50
>>>> MinSpareThreads 25 10 25
>>>> MaxSpareThreads 250 100 100
>>>> MaxThreads 1000 2048 1000
>>>> MaxRequestsPerChild 0 0 0
>>>> ThreadStackSize 65536 65536 65536
>>>> MaxMemFree 100 0 - (remove)
>>>
>>> ThreadStackSize seems a bit dicey, would rather see 128k default.
>>
>> Because it is the only MPM with non system default used, I wondered
>> whether there's a bit of history behind that value.
>
> I'm certain :) Guenter? Brad?
well, I dont think a higher ThreadStackSize can hurt much beside higher
memory usage (which shouldnt be an issue nowadays). I cant tell why we
have 64k as default, but I can tell that this is 8 times as much as what
our linker uses by default if you dont specify a stacksize, so brobably
it was just a value which Brad thought would be enough when he ported
2.0 to NetWare ...; also IIRC 8k was way too less for modules like
mod_rewrite ... - but everything else seems to be coded cleaner so that
every greater memory block is aloocated rather than pushed on stack.
Also I can add that for mysql I use 128k too since had probs with the
testsuite where some tests crashed with 64k. So in general I consider a
higher stackzise more safe for bad coding where hughe memory blocks are
pushed on the stack.
But lets also hear what Brad thinks - anyway its here only a config file
change.
Gün.
Re: MPM sizing defaults and config
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 6/15/2010 4:42 PM, Rainer Jung wrote:
> On 15.06.2010 23:09, William A. Rowe Jr. wrote:
>> As a broad general question - why not equivalent number of MaxClients
>> across all MPMs?
>
> I was uncertain about that. Users often tend to try to fix performance
> problems by adding concurrency to the web server. If your web server is
> configured very tight, then this will help. But if you actually have a
> throughput problem (slow backends, network etc.), then allowing evengo
> more concurrency will make things worse.
We don't disagree.
> So I wanted to stick away from sizing to what's possible and stay closer
> to what's reasonable. Concerning Unix my gut feeling was that 256
> processes (prefork) and 16x25=400 threads doesn't put much pressure on
> system resources of modern entry servers. I raised the MaxClients for
> the threaded MPMs, because I expect them to be a bit cheaper. But no
> strong opinion here about the best numbers. Keeping MaxClients in sync
> would make the MPMs better comparable out of the box, but on the other
> hand precisely this type of comparison might not make much sense.
Here again, keeping it tight is probably good. Threads do cost (stacks,
fd's, lots of other resources) although they don't generally eat everything
that a process would.
None the less, if the 'out of the box' server should support 250 clients,
that's what we aught to do. I'd hate people to jump from prefork to worker,
for example, only to find their resource profile and concurrency is changed
entirely under their nose.
In fact my 'quick test' servers I hang onto all the time only have 32 workers
each, for this very reason. I don't care to have them gobble resources and
I rarely need more concurrency, myself.
> For event it's quite possible, that the idleness of worker threads
> fluctuates. So bigger thread pools (per process) might reduce risk of
> starvation.
But that's true of prefork or worker as well, if they are blocked on
something like CGI. Here again, a consistent MaxClients, with the
user-predictable caviat that they might wish to raise this for those
who have slow/blocking response content.
> Only the Netware MPM uses a thread stack size different from the OS
> default at the moment (OK, Windows uses hard-coded 65536 for the service
> thread and a stderr_thread but not for the worker threads). So do we
> want to now switch over to an httpd specific value or should we stick
> with system default?
It does? I guess I never backported this from trunk, but on trunk it
most certainly supports ThreadStackSize, and that number happens to
default to 256k from the build schema (and yes, some trivial required
threads get bumped down to a sensible 64k).
>>> Netware
>>> ======= conf default proposed
>>> StartThreads 250 50 50
>>> MinSpareThreads 25 10 25
>>> MaxSpareThreads 250 100 100
>>> MaxThreads 1000 2048 1000
>>> MaxRequestsPerChild 0 0 0
>>> ThreadStackSize 65536 65536 65536
>>> MaxMemFree 100 0 - (remove)
>>
>> ThreadStackSize seems a bit dicey, would rather see 128k default.
>
> Because it is the only MPM with non system default used, I wondered
> whether there's a bit of history behind that value.
I'm certain :) Guenter? Brad?
Re: MPM sizing defaults and config
Posted by Rainer Jung <ra...@kippdata.de>.
On 15.06.2010 23:09, William A. Rowe Jr. wrote:
> As a broad general question - why not equivalent number of MaxClients
> across all MPMs?
I was uncertain about that. Users often tend to try to fix performance
problems by adding concurrency to the web server. If your web server is
configured very tight, then this will help. But if you actually have a
throughput problem (slow backends, network etc.), then allowing even
more concurrency will make things worse.
So I wanted to stick away from sizing to what's possible and stay closer
to what's reasonable. Concerning Unix my gut feeling was that 256
processes (prefork) and 16x25=400 threads doesn't put much pressure on
system resources of modern entry servers. I raised the MaxClients for
the threaded MPMs, because I expect them to be a bit cheaper. But no
strong opinion here about the best numbers. Keeping MaxClients in sync
would make the MPMs better comparable out of the box, but on the other
hand precisely this type of comparison might not make much sense.
> On 6/15/2010 4:03 PM, Rainer Jung wrote:
>> The default configuration for various MPMs
>> differs from the example configuration file
>> conf/extra/httpd-mpm.conf.
>>
>> Before bringing those two in sync, I want
>> to propose the values we want to use as new
>> defaults as well as for the extras configuration
>> file.
>>
>> Prefork
>> ======= conf default proposed
>> StartServers 5 5 5
>> MinSpareServers 5 5 3
>> MaxSpareServers 10 10 15
>> MaxClients 150 256 256
>> MaxRequestsPerChild 0 10000 0
>
> This seems bursty - can't we raise MaxSpare to at least 50?
Right, we should in order to prevent continuous forking and dying.
>> Worker/Event
>> ============ conf default proposed
>> StartServers 2 3 2
>> MinSpareThreads 25 75 25
>> MaxSpareThreads 75 250 100
>> MaxClients 150 400 400
>> ThreadsPerChild 25 25 25
>> MaxRequestsPerChild 0 10000 0
>
> If threaded, and stable, why not some 50 or 100 threads per child?
OK, especially since for Event I think that there is a problem of
correctly balancing load between child processes. For worker it is
clear, when a child is saturated, i.e. when each thread has a connection.
For event it's quite possible, that the idleness of worker threads
fluctuates. So bigger thread pools (per process) might reduce risk of
starvation.
>> Simple
>> ====== conf default proposed
>> SimpleProcCount 5 5 5?
>> SimpleThreadCount 5 5 5?
>>
>> WinNT
>> ===== conf default proposed
>> ThreadsPerChild 150 64 150
>> MaxRequestsPerChild 0 0 0
>
> +1
>
> ThreadStackSize could probably be cut to 128k default (256k processwide-default).
Only the Netware MPM uses a thread stack size different from the OS
default at the moment (OK, Windows uses hard-coded 65536 for the service
thread and a stderr_thread but not for the worker threads). So do we
want to now switch over to an httpd specific value or should we stick
with system default?
>> Netware
>> ======= conf default proposed
>> StartThreads 250 50 50
>> MinSpareThreads 25 10 25
>> MaxSpareThreads 250 100 100
>> MaxThreads 1000 2048 1000
>> MaxRequestsPerChild 0 0 0
>> ThreadStackSize 65536 65536 65536
>> MaxMemFree 100 0 - (remove)
>
> ThreadStackSize seems a bit dicey, would rather see 128k default.
Because it is the only MPM with non system default used, I wondered
whether there's a bit of history behind that value.
>> OS2
>> === conf default proposed
>> StartServers 2 2 2
>> MinSpareThreads 5 5 5
>> MaxSpareThreads 10 10 20
>> MaxRequestsPerChild 0 10000 0
>
> ThreadsPerChild?
i read the code like this: number of threads per child is dynamic.
Maximum is not configurable but instead defined by HARD_THREAD_LIMIT,
which is set to 256 in the MPM source.
Rainer
Re: MPM sizing defaults and config
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
As a broad general question - why not equivalent number of MaxClients
across all MPMs?
On 6/15/2010 4:03 PM, Rainer Jung wrote:
> The default configuration for various MPMs
> differs from the example configuration file
> conf/extra/httpd-mpm.conf.
>
> Before bringing those two in sync, I want
> to propose the values we want to use as new
> defaults as well as for the extras configuration
> file.
>
> Prefork
> ======= conf default proposed
> StartServers 5 5 5
> MinSpareServers 5 5 3
> MaxSpareServers 10 10 15
> MaxClients 150 256 256
> MaxRequestsPerChild 0 10000 0
This seems bursty - can't we raise MaxSpare to at least 50?
> Worker/Event
> ============ conf default proposed
> StartServers 2 3 2
> MinSpareThreads 25 75 25
> MaxSpareThreads 75 250 100
> MaxClients 150 400 400
> ThreadsPerChild 25 25 25
> MaxRequestsPerChild 0 10000 0
If threaded, and stable, why not some 50 or 100 threads per child?
> Simple
> ====== conf default proposed
> SimpleProcCount 5 5 5?
> SimpleThreadCount 5 5 5?
>
> WinNT
> ===== conf default proposed
> ThreadsPerChild 150 64 150
> MaxRequestsPerChild 0 0 0
+1
ThreadStackSize could probably be cut to 128k default (256k processwide-default).
> Netware
> ======= conf default proposed
> StartThreads 250 50 50
> MinSpareThreads 25 10 25
> MaxSpareThreads 250 100 100
> MaxThreads 1000 2048 1000
> MaxRequestsPerChild 0 0 0
> ThreadStackSize 65536 65536 65536
> MaxMemFree 100 0 - (remove)
ThreadStackSize seems a bit dicey, would rather see 128k default.
> OS2
> === conf default proposed
> StartServers 2 2 2
> MinSpareThreads 5 5 5
> MaxSpareThreads 10 10 20
> MaxRequestsPerChild 0 10000 0
ThreadsPerChild?
Re: MPM sizing defaults and config
Posted by Rainer Jung <ra...@kippdata.de>.
On 21.06.2010 11:34, Brian Havard wrote:
> On 21/06/10 09:20, Igor Galić wrote:
>> From docs/manual/upgrading.xml:
>>
>> +<li>Platform support has been removed for BeOS, OS/2, TPF, and
>> + even older platforms such as A/UX, Next, and Tandem. These were
>> + believed to be broken anyway.</li>
>>
>> What piece am I missing?
>>
>
> The docs need updating. OS/2 support has been revived.
Fixed in r956524.
Rainer
Re: MPM sizing defaults and config
Posted by Brian Havard <br...@gmail.com>.
On 21/06/10 09:20, Igor Galić wrote:
> ----- "Rainer Jung" <ra...@kippdata.de> wrote:
>
>> OS2
>> === conf default proposed
>> StartServers 2 2 2
>> MinSpareThreads 5 5 5
>> MaxSpareThreads 10 10 20
>> MaxRequestsPerChild 0 10000 0
>>
>> Any comments?
>>
> From docs/manual/upgrading.xml:
>
> + <li>Platform support has been removed for BeOS, OS/2, TPF, and
> + even older platforms such as A/UX, Next, and Tandem. These were
> + believed to be broken anyway.</li>
>
> What piece am I missing?
>
The docs need updating. OS/2 support has been revived.
Re: MPM sizing defaults and config
Posted by Igor Galić <i....@brainsware.org>.
----- "Rainer Jung" <ra...@kippdata.de> wrote:
> OS2
> === conf default proposed
> StartServers 2 2 2
> MinSpareThreads 5 5 5
> MaxSpareThreads 10 10 20
> MaxRequestsPerChild 0 10000 0
>
> Any comments?
>From docs/manual/upgrading.xml:
+ <li>Platform support has been removed for BeOS, OS/2, TPF, and
+ even older platforms such as A/UX, Next, and Tandem. These were
+ believed to be broken anyway.</li>
What piece am I missing?
> Regards,
>
> Rainer
--
Igor Galić
Tel: +43 (0) 699 122 96 338
Fax: +43(0) 1 91 333 41
Mail: i.galic@brainsware.org
URL: http://brainsware.org/