You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Eric Covener <co...@gmail.com> on 2013/11/09 13:13:21 UTC
RLIMIT_NPROC on Linux?
I was looking at a typical apr_thread_create failure for creating a
large # of threads on a system, and the only solution was to increase
roots RLIMIT_NPROC as opposed to the (httpd.conf configured) "User"
limits
But every manpage I read says that after the setuid(), we should have
the new users limits, which means only tuning those limits should be
necessary.
Can anyone explain/translate/debunk?
I'm doing a single child with 1100 threads. I toggle root/nobody
having 1000/1200 available maxuprocs on an idle system. It works when
root has 1200 and nobody has 1000, and not the other way around.
--
Eric Covener
covener@gmail.com
Re: RLIMIT_NPROC on Linux?
Posted by Eric Covener <co...@gmail.com>.
See also PR#55763 -- root has a soft limit of 1024 ulimit -u on RHEL6.
On Sat, Nov 9, 2013 at 7:46 AM, Eric Covener <co...@gmail.com> wrote:
> Behavior seems to be that after the setuid, we don't get the new users
> limits -- we just get a one-time check to make sure our currently
> single-thread process won't push us over the new users' limit.
> Confirmed in /proc/$pid/limits on a child in start_threads() that
> roots limit is in place.
>
> On Sat, Nov 9, 2013 at 7:13 AM, Eric Covener <co...@gmail.com> wrote:
>> I was looking at a typical apr_thread_create failure for creating a
>> large # of threads on a system, and the only solution was to increase
>> roots RLIMIT_NPROC as opposed to the (httpd.conf configured) "User"
>> limits
>>
>> But every manpage I read says that after the setuid(), we should have
>> the new users limits, which means only tuning those limits should be
>> necessary.
>>
>> Can anyone explain/translate/debunk?
>>
>> I'm doing a single child with 1100 threads. I toggle root/nobody
>> having 1000/1200 available maxuprocs on an idle system. It works when
>> root has 1200 and nobody has 1000, and not the other way around.
>>
>> --
>> Eric Covener
>> covener@gmail.com
>
>
>
> --
> Eric Covener
> covener@gmail.com
--
Eric Covener
covener@gmail.com
Re: RLIMIT_NPROC on Linux?
Posted by Eric Covener <co...@gmail.com>.
Behavior seems to be that after the setuid, we don't get the new users
limits -- we just get a one-time check to make sure our currently
single-thread process won't push us over the new users' limit.
Confirmed in /proc/$pid/limits on a child in start_threads() that
roots limit is in place.
On Sat, Nov 9, 2013 at 7:13 AM, Eric Covener <co...@gmail.com> wrote:
> I was looking at a typical apr_thread_create failure for creating a
> large # of threads on a system, and the only solution was to increase
> roots RLIMIT_NPROC as opposed to the (httpd.conf configured) "User"
> limits
>
> But every manpage I read says that after the setuid(), we should have
> the new users limits, which means only tuning those limits should be
> necessary.
>
> Can anyone explain/translate/debunk?
>
> I'm doing a single child with 1100 threads. I toggle root/nobody
> having 1000/1200 available maxuprocs on an idle system. It works when
> root has 1200 and nobody has 1000, and not the other way around.
>
> --
> Eric Covener
> covener@gmail.com
--
Eric Covener
covener@gmail.com
Re: RLIMIT_NPROC on Linux?
Posted by Eric Covener <co...@gmail.com>.
On Tue, Nov 12, 2013 at 1:16 PM, William A. Rowe Jr.
<wr...@rowe-clan.net> wrote:
> On Tue, 12 Nov 2013 09:04:13 -0500
> Eric Covener <co...@gmail.com> wrote:
>
>> On Mon, Nov 11, 2013 at 7:30 AM, Ruediger Pluem <rp...@apache.org>
>> wrote:
>> >
>> >
>> > Eric Covener wrote:
>> >> I was looking at a typical apr_thread_create failure for creating a
>> >> large # of threads on a system, and the only solution was to
>> >> increase roots RLIMIT_NPROC as opposed to the (httpd.conf
>> >> configured) "User" limits
>> >
>> > I assume that you configured that via /etc/security/limits.conf?
>>
>> Yep. I still haven't figured out if the target users ulimits ever
>> matter (around setuid() call, as implied by the manual) or if it's
>> 100% root.
>
> So to further clarify, your understanding right now is that...
>
> - root's soft ulimit overriden by any ulimit -u in the startup
> script is honored for the parent httpd process
yes
>
> - ulimit is not replaced with User's limits during startup/setuid
>
by the time we're creating worker threads, roots ulimit still matters,
which is consistent with fork() doc
> - the User's hard ulimit is checked and causes the startup to fail
> if the (root) soft ulimit then exceeds the user's max hard limit?
Not following on this one.
My guess is that the Users limit will be checked at setuid(), where
we'll be adding one more thread. At this point, you'd have to think
any existing worker threads count against the Users limit (even though
when they were being spawned, roots ulimit was the operative one). I
have not been able to return to testing it, was hoping someone already
had it collected as a FAQ.
Re: RLIMIT_NPROC on Linux?
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On Tue, 12 Nov 2013 09:04:13 -0500
Eric Covener <co...@gmail.com> wrote:
> On Mon, Nov 11, 2013 at 7:30 AM, Ruediger Pluem <rp...@apache.org>
> wrote:
> >
> >
> > Eric Covener wrote:
> >> I was looking at a typical apr_thread_create failure for creating a
> >> large # of threads on a system, and the only solution was to
> >> increase roots RLIMIT_NPROC as opposed to the (httpd.conf
> >> configured) "User" limits
> >
> > I assume that you configured that via /etc/security/limits.conf?
>
> Yep. I still haven't figured out if the target users ulimits ever
> matter (around setuid() call, as implied by the manual) or if it's
> 100% root.
So to further clarify, your understanding right now is that...
- root's soft ulimit overriden by any ulimit -u in the startup
script is honored for the parent httpd process
- ulimit is not replaced with User's limits during startup/setuid
- the User's hard ulimit is checked and causes the startup to fail
if the (root) soft ulimit then exceeds the user's max hard limit?
Is that correct?
I'm working on a simple module to report the limits at various phases
of the startup, since this affects any httpd version, and patching all
of the flavors to report this info seems convoluted for diagnostics.
Should have an update/source by tomorrow.
Re: RLIMIT_NPROC on Linux?
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On Mon, 18 Nov 2013 12:52:26 -0500
Eric Covener <co...@gmail.com> wrote:
>
> # Set the maximum number of file descriptors allowed per child
> process. if [ "x$ULIMIT_MAX_FILES" != "x" ] ; then
> $ULIMIT_MAX_FILES
> fi
>
>
> Should we do the same for threads/processes? I was going to propose
> this for some kind of startup code, and realized there was some old
> precedent. The downside is that we don't yet know MaxClients is
> large, as we could in code.
Keep in mind, we also don't know if we are launching one rotatelogs per
log file, our cgi requirements, etc etc etc. There are too many variables
to know what our actual process count requirement really is.
We could go with a number such as current ulimit -Su + MaxClients, or we
could go with the hardlimit (of root, or of the httpd User), or... ???
Re: RLIMIT_NPROC on Linux?
Posted by Eric Covener <co...@gmail.com>.
current apachectl:
#
# Set this variable to a command that increases the maximum
# number of file descriptors allowed per child process. This is
# critical for configurations that use many file descriptors,
# such as mass vhosting, or a multithreaded server.
ULIMIT_MAX_FILES="ulimit -S -n `ulimit -H -n`"
# -------------------- --------------------
# |||||||||||||||||||| END CONFIGURATION SECTION ||||||||||||||||||||
# Set the maximum number of file descriptors allowed per child process.
if [ "x$ULIMIT_MAX_FILES" != "x" ] ; then
$ULIMIT_MAX_FILES
fi
Should we do the same for threads/processes? I was going to propose
this for some kind of startup code, and realized there was some old
precedent. The downside is that we don't yet know MaxClients is
large, as we could in code.
On Tue, Nov 12, 2013 at 3:00 PM, Eric Covener <co...@gmail.com> wrote:
>>
>> IMHO this explains it as limits.conf is a configuration file for PAM. If you don't use
>> any PAM methods (haven't worked out which would be needed) in the code the limits will not
>> be applied after setuid. Of course pam_limits.so need to be configured for session for your app
>> as well.
>
> Ah, that helps - -thanks!
--
Eric Covener
covener@gmail.com
Re: RLIMIT_NPROC on Linux?
Posted by Eric Covener <co...@gmail.com>.
>
> IMHO this explains it as limits.conf is a configuration file for PAM. If you don't use
> any PAM methods (haven't worked out which would be needed) in the code the limits will not
> be applied after setuid. Of course pam_limits.so need to be configured for session for your app
> as well.
Ah, that helps - -thanks!
Re: RLIMIT_NPROC on Linux?
Posted by Ruediger Pluem <rp...@apache.org>.
Eric Covener wrote:
> On Mon, Nov 11, 2013 at 7:30 AM, Ruediger Pluem <rp...@apache.org> wrote:
>>
>>
>> Eric Covener wrote:
>>> I was looking at a typical apr_thread_create failure for creating a
>>> large # of threads on a system, and the only solution was to increase
>>> roots RLIMIT_NPROC as opposed to the (httpd.conf configured) "User"
>>> limits
>>
>> I assume that you configured that via /etc/security/limits.conf?
>
> Yep. I still haven't figured out if the target users ulimits ever
> matter (around setuid() call, as implied by the manual) or if it's
> 100% root.
>
IMHO this explains it as limits.conf is a configuration file for PAM. If you don't use
any PAM methods (haven't worked out which would be needed) in the code the limits will not
be applied after setuid. Of course pam_limits.so need to be configured for session for your app
as well.
Regards
Rüdiger
Re: RLIMIT_NPROC on Linux?
Posted by Eric Covener <co...@gmail.com>.
On Mon, Nov 11, 2013 at 7:30 AM, Ruediger Pluem <rp...@apache.org> wrote:
>
>
> Eric Covener wrote:
>> I was looking at a typical apr_thread_create failure for creating a
>> large # of threads on a system, and the only solution was to increase
>> roots RLIMIT_NPROC as opposed to the (httpd.conf configured) "User"
>> limits
>
> I assume that you configured that via /etc/security/limits.conf?
Yep. I still haven't figured out if the target users ulimits ever
matter (around setuid() call, as implied by the manual) or if it's
100% root.
Re: RLIMIT_NPROC on Linux?
Posted by Ruediger Pluem <rp...@apache.org>.
Eric Covener wrote:
> I was looking at a typical apr_thread_create failure for creating a
> large # of threads on a system, and the only solution was to increase
> roots RLIMIT_NPROC as opposed to the (httpd.conf configured) "User"
> limits
I assume that you configured that via /etc/security/limits.conf?
Regards
Rüdiger