You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by "Leif Hedstrom (JIRA)" <ji...@apache.org> on 2009/12/16 04:37:18 UTC
[jira] Created: (TS-88) traffic_server not able to setrlimit > 1024
on FDs
traffic_server not able to setrlimit > 1024 on FDs
--------------------------------------------------
Key: TS-88
URL: https://issues.apache.org/jira/browse/TS-88
Project: Traffic Server
Issue Type: Bug
Components: Core
Affects Versions: 2.0a
Reporter: Leif Hedstrom
When starting TS, i get a warning / error like this:
[example_alarm_bin.sh] sent alarm: loki.ogre.com [Tue Dec 15 20:33:55 2009] connection throttle too high, 10000 (throttle) + 192 (internal use) > 8192 (file descriptor limit), using throttle of 8000
stracing traffic_manager, I see:
root@loki 332/1 # strace -f -e trace=setrlimit ./bin/traffic_manager 2>&1 | grep setrlimit
[pid 24114] setrlimit(RLIMIT_CPU, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
[pid 24114] setrlimit(RLIMIT_FSIZE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
[pid 24114] setrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
[pid 24126] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
[pid 24126] setrlimit(RLIMIT_NOFILE, {rlim_cur=10000, rlim_max=10000}) = -1 EPERM (Operation not permitted)
The last call is from traffic_server, indicating that it's not able to setrlimit() to 10000. This would only happen (I think) if at this point it's running as a non-root user. If I run traffic_server as root directly, I don't have this problem (i.e. when it calls setrlimit() as root, it works fine).
This is using a standard installation in /usr/local.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (TS-88) traffic_server not able to setrlimit >
1024 on FDs
Posted by "George Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
George Paul resolved TS-88.
---------------------------
Resolution: Fixed
This patch 'TS88_patch2.diff' has been committed to trunk.
-George
> traffic_server not able to setrlimit > 1024 on FDs
> --------------------------------------------------
>
> Key: TS-88
> URL: https://issues.apache.org/jira/browse/TS-88
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0.0a
> Reporter: Leif Hedstrom
> Assignee: George Paul
> Fix For: 2.0.0a
>
> Attachments: TS88_patch1.diff, TS88_patch2.diff
>
>
> When starting TS, i get a warning / error like this:
> [example_alarm_bin.sh] sent alarm: loki.ogre.com [Tue Dec 15 20:33:55 2009] connection throttle too high, 10000 (throttle) + 192 (internal use) > 8192 (file descriptor limit), using throttle of 8000
> stracing traffic_manager, I see:
> root@loki 332/1 # strace -f -e trace=setrlimit ./bin/traffic_manager 2>&1 | grep setrlimit
> [pid 24114] setrlimit(RLIMIT_CPU, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_FSIZE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
> [pid 24126] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
> [pid 24126] setrlimit(RLIMIT_NOFILE, {rlim_cur=10000, rlim_max=10000}) = -1 EPERM (Operation not permitted)
> The last call is from traffic_server, indicating that it's not able to setrlimit() to 10000. This would only happen (I think) if at this point it's running as a non-root user. If I run traffic_server as root directly, I don't have this problem (i.e. when it calls setrlimit() as root, it works fine).
> This is using a standard installation in /usr/local.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (TS-88) traffic_server not able to setrlimit > 1024
on FDs
Posted by "George Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
George Paul updated TS-88:
--------------------------
Attachment: TS88_patch2.diff
This patch 'TS88_patch2.diff' replaces the previous patch 'TS88_patch1.diff' to fix a compilation issue on RH-4.3.x.
Please review and let me know if there are any issues.
Tested on Ubuntu904 and Fedora11.
-George
> traffic_server not able to setrlimit > 1024 on FDs
> --------------------------------------------------
>
> Key: TS-88
> URL: https://issues.apache.org/jira/browse/TS-88
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0.0a
> Reporter: Leif Hedstrom
> Assignee: George Paul
> Fix For: 2.0.0a
>
> Attachments: TS88_patch1.diff, TS88_patch2.diff
>
>
> When starting TS, i get a warning / error like this:
> [example_alarm_bin.sh] sent alarm: loki.ogre.com [Tue Dec 15 20:33:55 2009] connection throttle too high, 10000 (throttle) + 192 (internal use) > 8192 (file descriptor limit), using throttle of 8000
> stracing traffic_manager, I see:
> root@loki 332/1 # strace -f -e trace=setrlimit ./bin/traffic_manager 2>&1 | grep setrlimit
> [pid 24114] setrlimit(RLIMIT_CPU, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_FSIZE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
> [pid 24126] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
> [pid 24126] setrlimit(RLIMIT_NOFILE, {rlim_cur=10000, rlim_max=10000}) = -1 EPERM (Operation not permitted)
> The last call is from traffic_server, indicating that it's not able to setrlimit() to 10000. This would only happen (I think) if at this point it's running as a non-root user. If I run traffic_server as root directly, I don't have this problem (i.e. when it calls setrlimit() as root, it works fine).
> This is using a standard installation in /usr/local.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (TS-88) traffic_server not able to setrlimit >
1024 on FDs
Posted by "Leif Hedstrom (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-88?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829136#action_12829136 ]
Leif Hedstrom commented on TS-88:
---------------------------------
Reviewed and tested the updated patch, this now works and looks good for me. The traffic_server process is now getting the correct limits, even when running as nobody:
root@loki 261/0 # cat /proc/5956/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited ms
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size unlimited unlimited bytes
Max core file size 0 0 bytes
Max resident set unlimited unlimited bytes
Max processes 1024 unlimited processes
Max open files 10000 10000 files
> traffic_server not able to setrlimit > 1024 on FDs
> --------------------------------------------------
>
> Key: TS-88
> URL: https://issues.apache.org/jira/browse/TS-88
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0.0a
> Reporter: Leif Hedstrom
> Assignee: George Paul
> Fix For: 2.0.0a
>
> Attachments: TS88_patch1.diff, TS88_patch2.diff
>
>
> When starting TS, i get a warning / error like this:
> [example_alarm_bin.sh] sent alarm: loki.ogre.com [Tue Dec 15 20:33:55 2009] connection throttle too high, 10000 (throttle) + 192 (internal use) > 8192 (file descriptor limit), using throttle of 8000
> stracing traffic_manager, I see:
> root@loki 332/1 # strace -f -e trace=setrlimit ./bin/traffic_manager 2>&1 | grep setrlimit
> [pid 24114] setrlimit(RLIMIT_CPU, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_FSIZE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
> [pid 24126] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
> [pid 24126] setrlimit(RLIMIT_NOFILE, {rlim_cur=10000, rlim_max=10000}) = -1 EPERM (Operation not permitted)
> The last call is from traffic_server, indicating that it's not able to setrlimit() to 10000. This would only happen (I think) if at this point it's running as a non-root user. If I run traffic_server as root directly, I don't have this problem (i.e. when it calls setrlimit() as root, it works fine).
> This is using a standard installation in /usr/local.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (TS-88) traffic_server not able to setrlimit >
1024 on FDs
Posted by "George Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-88?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806601#action_12806601 ]
George Paul commented on TS-88:
-------------------------------
Looking into this the best way going forward to solve this on Linux is to use the capabilities (7) facility that exist in the Linux kernels >= 2.6.24.
Minimum requirements are: (see http://www.friedhoff.org/posixfilecaps.html)
- a kernel >= 2.6.24
- a filesystem that supports extended attributes like ext3
- File POSIX Capabilities enabled
- libcap2 package >=2.08
- libcap2 utilities (setcap, getcap, etc)
Basically after installation one needs to do the following: e.g.
sudo setcap cap_sys_resource=ep /usr/local/bin/traffic_server
The 'trafficserver' script can then be used to bring up the process stack as the configured user.
Tested on Ubunt-9.04 and Fedora 11 w/ libcap2 pkgs installed and default latest kernels.
These procedures/steps need to be documented in the installation notes/guide.
-George
> traffic_server not able to setrlimit > 1024 on FDs
> --------------------------------------------------
>
> Key: TS-88
> URL: https://issues.apache.org/jira/browse/TS-88
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0.0a
> Reporter: Leif Hedstrom
> Assignee: George Paul
>
> When starting TS, i get a warning / error like this:
> [example_alarm_bin.sh] sent alarm: loki.ogre.com [Tue Dec 15 20:33:55 2009] connection throttle too high, 10000 (throttle) + 192 (internal use) > 8192 (file descriptor limit), using throttle of 8000
> stracing traffic_manager, I see:
> root@loki 332/1 # strace -f -e trace=setrlimit ./bin/traffic_manager 2>&1 | grep setrlimit
> [pid 24114] setrlimit(RLIMIT_CPU, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_FSIZE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
> [pid 24126] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
> [pid 24126] setrlimit(RLIMIT_NOFILE, {rlim_cur=10000, rlim_max=10000}) = -1 EPERM (Operation not permitted)
> The last call is from traffic_server, indicating that it's not able to setrlimit() to 10000. This would only happen (I think) if at this point it's running as a non-root user. If I run traffic_server as root directly, I don't have this problem (i.e. when it calls setrlimit() as root, it works fine).
> This is using a standard installation in /usr/local.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (TS-88) traffic_server not able to setrlimit >
1024 on FDs
Posted by "George Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
George Paul reassigned TS-88:
-----------------------------
Assignee: George Paul
> traffic_server not able to setrlimit > 1024 on FDs
> --------------------------------------------------
>
> Key: TS-88
> URL: https://issues.apache.org/jira/browse/TS-88
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0.0a
> Reporter: Leif Hedstrom
> Assignee: George Paul
>
> When starting TS, i get a warning / error like this:
> [example_alarm_bin.sh] sent alarm: loki.ogre.com [Tue Dec 15 20:33:55 2009] connection throttle too high, 10000 (throttle) + 192 (internal use) > 8192 (file descriptor limit), using throttle of 8000
> stracing traffic_manager, I see:
> root@loki 332/1 # strace -f -e trace=setrlimit ./bin/traffic_manager 2>&1 | grep setrlimit
> [pid 24114] setrlimit(RLIMIT_CPU, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_FSIZE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
> [pid 24126] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
> [pid 24126] setrlimit(RLIMIT_NOFILE, {rlim_cur=10000, rlim_max=10000}) = -1 EPERM (Operation not permitted)
> The last call is from traffic_server, indicating that it's not able to setrlimit() to 10000. This would only happen (I think) if at this point it's running as a non-root user. If I run traffic_server as root directly, I don't have this problem (i.e. when it calls setrlimit() as root, it works fine).
> This is using a standard installation in /usr/local.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (TS-88) traffic_server not able to setrlimit > 1024
on FDs
Posted by "George Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
George Paul updated TS-88:
--------------------------
Attachment: TS88_patch1.diff
This patch 'TS88_patch1.diff' sets the process limits like 'RLIMIT_NOFILE' in Traffic Manager when run as root before the TM changes to the configured user&group.
Please review and let me know if there are any issues.
Tested on Ubuntu904 and Fedora11.
-George
> traffic_server not able to setrlimit > 1024 on FDs
> --------------------------------------------------
>
> Key: TS-88
> URL: https://issues.apache.org/jira/browse/TS-88
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0.0a
> Reporter: Leif Hedstrom
> Assignee: George Paul
> Fix For: 2.0.0a
>
> Attachments: TS88_patch1.diff
>
>
> When starting TS, i get a warning / error like this:
> [example_alarm_bin.sh] sent alarm: loki.ogre.com [Tue Dec 15 20:33:55 2009] connection throttle too high, 10000 (throttle) + 192 (internal use) > 8192 (file descriptor limit), using throttle of 8000
> stracing traffic_manager, I see:
> root@loki 332/1 # strace -f -e trace=setrlimit ./bin/traffic_manager 2>&1 | grep setrlimit
> [pid 24114] setrlimit(RLIMIT_CPU, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_FSIZE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
> [pid 24126] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
> [pid 24126] setrlimit(RLIMIT_NOFILE, {rlim_cur=10000, rlim_max=10000}) = -1 EPERM (Operation not permitted)
> The last call is from traffic_server, indicating that it's not able to setrlimit() to 10000. This would only happen (I think) if at this point it's running as a non-root user. If I run traffic_server as root directly, I don't have this problem (i.e. when it calls setrlimit() as root, it works fine).
> This is using a standard installation in /usr/local.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (TS-88) traffic_server not able to setrlimit >
1024 on FDs
Posted by "Leif Hedstrom (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-88?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806780#action_12806780 ]
Leif Hedstrom commented on TS-88:
---------------------------------
This sounds fine as an "optional" solution, but we need something much more portable and generic than this. All the requirements above are way above what we can expect from many Linux systems, and certainly not for anything outside linux. So, my suggestion is to implement a solution that is still based on calling setrlimit() as root before starting up traffic_server, and optionally implement the above. The above would be disabled by default, and enabled with appropriate --enable configure options.
> traffic_server not able to setrlimit > 1024 on FDs
> --------------------------------------------------
>
> Key: TS-88
> URL: https://issues.apache.org/jira/browse/TS-88
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0.0a
> Reporter: Leif Hedstrom
> Assignee: George Paul
>
> When starting TS, i get a warning / error like this:
> [example_alarm_bin.sh] sent alarm: loki.ogre.com [Tue Dec 15 20:33:55 2009] connection throttle too high, 10000 (throttle) + 192 (internal use) > 8192 (file descriptor limit), using throttle of 8000
> stracing traffic_manager, I see:
> root@loki 332/1 # strace -f -e trace=setrlimit ./bin/traffic_manager 2>&1 | grep setrlimit
> [pid 24114] setrlimit(RLIMIT_CPU, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_FSIZE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
> [pid 24126] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
> [pid 24126] setrlimit(RLIMIT_NOFILE, {rlim_cur=10000, rlim_max=10000}) = -1 EPERM (Operation not permitted)
> The last call is from traffic_server, indicating that it's not able to setrlimit() to 10000. This would only happen (I think) if at this point it's running as a non-root user. If I run traffic_server as root directly, I don't have this problem (i.e. when it calls setrlimit() as root, it works fine).
> This is using a standard installation in /usr/local.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (TS-88) traffic_server not able to setrlimit > 1024
on FDs
Posted by "Leif Hedstrom (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TS-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Leif Hedstrom updated TS-88:
----------------------------
Fix Version/s: 2.0.0a
> traffic_server not able to setrlimit > 1024 on FDs
> --------------------------------------------------
>
> Key: TS-88
> URL: https://issues.apache.org/jira/browse/TS-88
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0.0a
> Reporter: Leif Hedstrom
> Assignee: George Paul
> Fix For: 2.0.0a
>
>
> When starting TS, i get a warning / error like this:
> [example_alarm_bin.sh] sent alarm: loki.ogre.com [Tue Dec 15 20:33:55 2009] connection throttle too high, 10000 (throttle) + 192 (internal use) > 8192 (file descriptor limit), using throttle of 8000
> stracing traffic_manager, I see:
> root@loki 332/1 # strace -f -e trace=setrlimit ./bin/traffic_manager 2>&1 | grep setrlimit
> [pid 24114] setrlimit(RLIMIT_CPU, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_FSIZE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> [pid 24114] setrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
> [pid 24126] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
> [pid 24126] setrlimit(RLIMIT_NOFILE, {rlim_cur=10000, rlim_max=10000}) = -1 EPERM (Operation not permitted)
> The last call is from traffic_server, indicating that it's not able to setrlimit() to 10000. This would only happen (I think) if at this point it's running as a non-root user. If I run traffic_server as root directly, I don't have this problem (i.e. when it calls setrlimit() as root, it works fine).
> This is using a standard installation in /usr/local.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.