Setting fds_limit.

["Chou, Peter" <pb...@labs.att.com>]

Hi,

In adj_system_settings() in proxy/Main.cc, we have (paraphrasing here) --

if throttle > (current - head_room) then
    cur = max = throttle
    setrlimit()...

Would it make more sense to set --

    cur = max = throttle + head_room

Otherwise, our users see the warning, try to compensate, and see the warning again...

Default of 30000:

WARNING: connection throttle too high, 30000 (throttle) + 192 (internal use) > 30000 (file descriptor limit), using throttle of 29808
Changed to 29800:
WARNING: connection throttle too high, 29800 (throttle) + 192 (internal use) > 29800 (file descriptor limit), using throttle of 29608

We can make the warnings go away by setting the file limit higher in the systemd file for the service, but the above behavior can cause some confusion.

Thanks,
Peter

RE: Setting fds_limit.

["Chou, Peter" <pb...@labs.att.com>]

Oknet,

The proposed change below is NOT directly related to setting the ATS internal soft and hard limits (fds_throttle and fds_limit respectively). Instead, I am proposing to modify the way that we currently change the OS limits (done by setting lim.rlim_cur and lim.rlim_max and then calling the setrlimit() system function). If you look inside the adjust_sys_settings() function, we first attempt to set the OS limits to 90% of maxfiles (proxy.config.system.file_max_pct), and then we attempt to set the OS limits to fds_throttle (proxy.config.net.connections_throttle) if the current OS limits are not enough to satisfy the needs of fds_throttle + THROTTLE_FD_HEADROOM. The change I am proposing is that if we are changing the OS limits anway, then it seems not to make sense to set the OS limit to just fds_throttle. Instead, we should set the OS limit to fds_throttle + THROTTLE_FD_HEADROOM so that the fds_throttle is not forced lower again later in check_fd_limit().

Thanks,
Peter

-----Original Message-----
From: Chao Xu [mailto:oknet@apache.org] 
Sent: Saturday, June 24, 2017 5:43 AM
To: dev@trafficserver.apache.org
Subject: Re: Setting fds_limit.

Hi Peter,

The `fds_limit` is hard limit of fds. The 192 fds are reserved for ats
internal use (epoll fd , EThread signal, etc ...).
The throttle is soft limit of fds and (fds_limit - 192) is the max value of
throttle.

We can not and must not change the hard limit since it is set by operating
system.

Oknet Xu

2017-06-24 3:10 GMT+08:00 Chou, Peter <pb...@labs.att.com>:

> Hi,
>
> In adj_system_settings() in proxy/Main.cc, we have (paraphrasing here) --
>
> if throttle > (current - head_room) then
>     cur = max = throttle
>     setrlimit()...
>
> Would it make more sense to set --
>
>     cur = max = throttle + head_room
>
> Otherwise, our users see the warning, try to compensate, and see the
> warning again...
>
> Default of 30000:
>
> WARNING: connection throttle too high, 30000 (throttle) + 192 (internal
> use) > 30000 (file descriptor limit), using throttle of 29808
> Changed to 29800:
> WARNING: connection throttle too high, 29800 (throttle) + 192 (internal
> use) > 29800 (file descriptor limit), using throttle of 29608
>
> We can make the warnings go away by setting the file limit higher in the
> systemd file for the service, but the above behavior can cause some
> confusion.
>
> Thanks,
> Peter
>

Re: Setting fds_limit.

[Chao Xu <ok...@apache.org>]

Hi Peter,

The `fds_limit` is hard limit of fds. The 192 fds are reserved for ats
internal use (epoll fd , EThread signal, etc ...).
The throttle is soft limit of fds and (fds_limit - 192) is the max value of
throttle.

We can not and must not change the hard limit since it is set by operating
system.

Oknet Xu

2017-06-24 3:10 GMT+08:00 Chou, Peter <pb...@labs.att.com>:

> Hi,
>
> In adj_system_settings() in proxy/Main.cc, we have (paraphrasing here) --
>
> if throttle > (current - head_room) then
>     cur = max = throttle
>     setrlimit()...
>
> Would it make more sense to set --
>
>     cur = max = throttle + head_room
>
> Otherwise, our users see the warning, try to compensate, and see the
> warning again...
>
> Default of 30000:
>
> WARNING: connection throttle too high, 30000 (throttle) + 192 (internal
> use) > 30000 (file descriptor limit), using throttle of 29808
> Changed to 29800:
> WARNING: connection throttle too high, 29800 (throttle) + 192 (internal
> use) > 29800 (file descriptor limit), using throttle of 29608
>
> We can make the warnings go away by setting the file limit higher in the
> systemd file for the service, but the above behavior can cause some
> confusion.
>
> Thanks,
> Peter
>