You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Sudheer Vinukonda <su...@yahoo.com> on 2020/05/13 00:17:36 UTC
Connection and Timeout Tuning, config name changes in 9.x
9.x (re)introduces a feature (it was accidentally "lost" during some unrelated refactoring) that will allow to configure limits on max "active" connections that can be allowed at any given time (https://github.com/apache/trafficserver/pull/6754)
Given that this feature is based on tracking "active connections", it should work very well with HTTP/1.1 traffic (where, 1 active connection = 1 request). However, with HTTP/2 and stream multiplexing, this is no longer true and it isn't nearly as effective. One can still use an average number of concurrent streams/connection to configure the limits, but, ideally, what would work for both HTTP/1.1 and HTTP/2 (and going forward with QUIC/HTTP/3) would be to track and (rate) limit request level concurrency. There is some ongoing work on this and to align better with that work, we are proposing to make the below changes to existing configs with 9.x
1) Rename "proxy.config.net.max_connections_in" to "proxy.config.net.max_requests_in"
2) Rename "proxy.config.net.max_active_connections_in" to "proxy.config.net.max_active_requests_in"
3) proxy.config.net.connections_throttle - This config will stay put to limit the max number of open connections (FD, rlimits etc)
More context on the new changes -
The idea is to tune active connections that can be handled (based on the available resources/capacity (CPU, Memory, Network bandwidth) and minimize/remove dependency on having to tune connection timeouts (active/inactive/keep-alive etc) which are very hard to tune.
The primary goal is to limit the max active connections allowed at any given instant. The resource requirement for an idle/inactive vs active connections are completely different - For e.g an idle connection really only consumes memory resource, while an active connection consumes CPU, network besides memory. And allowing to tune/cap the max active connections based on the deployed capacity for the resources available, would make timeout tuning almost redundant and no op. Otherwise, we'd have to tune the timeouts to estimate throughput which is very hard as it's hard to justify how large or small we want the active timeout to be or keep alive timeout to be. For e.g in a non-peak hour, we could let the active timeout be much higher than the default, while during peak hour, we'd want to limit it to ensure we are not starving resources on one connection.
Note: there's one little TODO item here. PluginVC's connections are not tracked in the NetVC's active queue because these are bogus internal connections. However, for some users, internal traffic is significant (e.g sideways calls, or background fetches etc) and does consume plenty of resources. Since these internal requests don't impact ingress network, and have a slightly different resource requirements than client originated requests, it might be better to track these using a separate config for internal requests. Will follow that part up with a separate PR.
Re: Connection and Timeout Tuning, config name changes in 9.x
Posted by Sudheer Vinukonda <su...@yahoo.com>.
Thanks, Bryan! Based on the discussion with Bryan on slack, below is the newer proposed config changes:
1) Rename "proxy.config.net.max_active_connections_in" to "proxy.config.net.max_requests_in"
2) Rename "proxy.config.http.server_max_connections" to "proxy.config.net.max_connections_out"
3) "proxy.config.net.max_connections_in" - This config will stay put to limit the max number of in-flight requests (concurrency) + idle (socket) connections
4) "proxy.config.net.connections_throttle" - This config will stay put to limit the max number of open connections (FD, rlimits etc), but eventually will be deprecated and replaced with {{ "proxy.config.net.max_connections_in" + "proxy.config.net.max_connections_out" }}
Please provide any feedback/comments/concerns.
Thanks,
Sudheer
On Friday, May 15, 2020, 11:11:53 AM PDT, Bryan Call <bc...@apache.org> wrote:
After seeing the GitHub PR I think we should keep proxy.config.net.max_connections_in since it is the number of socket connections (active + idle) and then rename proxy.config.net.max_active_connections_in, as you suggested in your email, to proxy.config.net.max_active_requests_in or proxy.config.net.max_requests_in (as Sudheer suggested on Slack).
I would like to see us get rid of proxy.config.net.connections_throttle in the long term and use max_connections_in/out if possible.
-Bryan
> On May 12, 2020, at 5:17 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
>
> 9.x (re)introduces a feature (it was accidentally "lost" during some unrelated refactoring) that will allow to configure limits on max "active" connections that can be allowed at any given time (https://github.com/apache/trafficserver/pull/6754)
> Given that this feature is based on tracking "active connections", it should work very well with HTTP/1.1 traffic (where, 1 active connection = 1 request). However, with HTTP/2 and stream multiplexing, this is no longer true and it isn't nearly as effective. One can still use an average number of concurrent streams/connection to configure the limits, but, ideally, what would work for both HTTP/1.1 and HTTP/2 (and going forward with QUIC/HTTP/3) would be to track and (rate) limit request level concurrency. There is some ongoing work on this and to align better with that work, we are proposing to make the below changes to existing configs with 9.x
>
> 1) Rename "proxy.config.net.max_connections_in" to "proxy.config.net.max_requests_in"
> 2) Rename "proxy.config.net.max_active_connections_in" to "proxy.config.net.max_active_requests_in"
> 3) proxy.config.net.connections_throttle - This config will stay put to limit the max number of open connections (FD, rlimits etc)
>
> More context on the new changes -
>
>
> The idea is to tune active connections that can be handled (based on the available resources/capacity (CPU, Memory, Network bandwidth) and minimize/remove dependency on having to tune connection timeouts (active/inactive/keep-alive etc) which are very hard to tune.
>
> The primary goal is to limit the max active connections allowed at any given instant. The resource requirement for an idle/inactive vs active connections are completely different - For e.g an idle connection really only consumes memory resource, while an active connection consumes CPU, network besides memory. And allowing to tune/cap the max active connections based on the deployed capacity for the resources available, would make timeout tuning almost redundant and no op. Otherwise, we'd have to tune the timeouts to estimate throughput which is very hard as it's hard to justify how large or small we want the active timeout to be or keep alive timeout to be. For e.g in a non-peak hour, we could let the active timeout be much higher than the default, while during peak hour, we'd want to limit it to ensure we are not starving resources on one connection.
> Note: there's one little TODO item here. PluginVC's connections are not tracked in the NetVC's active queue because these are bogus internal connections. However, for some users, internal traffic is significant (e.g sideways calls, or background fetches etc) and does consume plenty of resources. Since these internal requests don't impact ingress network, and have a slightly different resource requirements than client originated requests, it might be better to track these using a separate config for internal requests. Will follow that part up with a separate PR.
>
>
Re: Connection and Timeout Tuning, config name changes in 9.x
Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.
Thanks, Bryan! Based on the discussion with Bryan on slack, below is the newer proposed config changes:
1) Rename "proxy.config.net.max_active_connections_in" to "proxy.config.net.max_requests_in"
2) Rename "proxy.config.http.server_max_connections" to "proxy.config.net.max_connections_out"
3) "proxy.config.net.max_connections_in" - This config will stay put to limit the max number of in-flight requests (concurrency) + idle (socket) connections
4) "proxy.config.net.connections_throttle" - This config will stay put to limit the max number of open connections (FD, rlimits etc), but eventually will be deprecated and replaced with {{ "proxy.config.net.max_connections_in" + "proxy.config.net.max_connections_out" }}
Please provide any feedback/comments/concerns.
Thanks,
Sudheer
On Friday, May 15, 2020, 11:11:53 AM PDT, Bryan Call <bc...@apache.org> wrote:
After seeing the GitHub PR I think we should keep proxy.config.net.max_connections_in since it is the number of socket connections (active + idle) and then rename proxy.config.net.max_active_connections_in, as you suggested in your email, to proxy.config.net.max_active_requests_in or proxy.config.net.max_requests_in (as Sudheer suggested on Slack).
I would like to see us get rid of proxy.config.net.connections_throttle in the long term and use max_connections_in/out if possible.
-Bryan
> On May 12, 2020, at 5:17 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
>
> 9.x (re)introduces a feature (it was accidentally "lost" during some unrelated refactoring) that will allow to configure limits on max "active" connections that can be allowed at any given time (https://github.com/apache/trafficserver/pull/6754)
> Given that this feature is based on tracking "active connections", it should work very well with HTTP/1.1 traffic (where, 1 active connection = 1 request). However, with HTTP/2 and stream multiplexing, this is no longer true and it isn't nearly as effective. One can still use an average number of concurrent streams/connection to configure the limits, but, ideally, what would work for both HTTP/1.1 and HTTP/2 (and going forward with QUIC/HTTP/3) would be to track and (rate) limit request level concurrency. There is some ongoing work on this and to align better with that work, we are proposing to make the below changes to existing configs with 9.x
>
> 1) Rename "proxy.config.net.max_connections_in" to "proxy.config.net.max_requests_in"
> 2) Rename "proxy.config.net.max_active_connections_in" to "proxy.config.net.max_active_requests_in"
> 3) proxy.config.net.connections_throttle - This config will stay put to limit the max number of open connections (FD, rlimits etc)
>
> More context on the new changes -
>
>
> The idea is to tune active connections that can be handled (based on the available resources/capacity (CPU, Memory, Network bandwidth) and minimize/remove dependency on having to tune connection timeouts (active/inactive/keep-alive etc) which are very hard to tune.
>
> The primary goal is to limit the max active connections allowed at any given instant. The resource requirement for an idle/inactive vs active connections are completely different - For e.g an idle connection really only consumes memory resource, while an active connection consumes CPU, network besides memory. And allowing to tune/cap the max active connections based on the deployed capacity for the resources available, would make timeout tuning almost redundant and no op. Otherwise, we'd have to tune the timeouts to estimate throughput which is very hard as it's hard to justify how large or small we want the active timeout to be or keep alive timeout to be. For e.g in a non-peak hour, we could let the active timeout be much higher than the default, while during peak hour, we'd want to limit it to ensure we are not starving resources on one connection.
> Note: there's one little TODO item here. PluginVC's connections are not tracked in the NetVC's active queue because these are bogus internal connections. However, for some users, internal traffic is significant (e.g sideways calls, or background fetches etc) and does consume plenty of resources. Since these internal requests don't impact ingress network, and have a slightly different resource requirements than client originated requests, it might be better to track these using a separate config for internal requests. Will follow that part up with a separate PR.
>
>
Re: Connection and Timeout Tuning, config name changes in 9.x
Posted by Bryan Call <bc...@apache.org>.
After seeing the GitHub PR I think we should keep proxy.config.net.max_connections_in since it is the number of socket connections (active + idle) and then rename proxy.config.net.max_active_connections_in, as you suggested in your email, to proxy.config.net.max_active_requests_in or proxy.config.net.max_requests_in (as Sudheer suggested on Slack).
I would like to see us get rid of proxy.config.net.connections_throttle in the long term and use max_connections_in/out if possible.
-Bryan
> On May 12, 2020, at 5:17 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
>
> 9.x (re)introduces a feature (it was accidentally "lost" during some unrelated refactoring) that will allow to configure limits on max "active" connections that can be allowed at any given time (https://github.com/apache/trafficserver/pull/6754)
> Given that this feature is based on tracking "active connections", it should work very well with HTTP/1.1 traffic (where, 1 active connection = 1 request). However, with HTTP/2 and stream multiplexing, this is no longer true and it isn't nearly as effective. One can still use an average number of concurrent streams/connection to configure the limits, but, ideally, what would work for both HTTP/1.1 and HTTP/2 (and going forward with QUIC/HTTP/3) would be to track and (rate) limit request level concurrency. There is some ongoing work on this and to align better with that work, we are proposing to make the below changes to existing configs with 9.x
>
> 1) Rename "proxy.config.net.max_connections_in" to "proxy.config.net.max_requests_in"
> 2) Rename "proxy.config.net.max_active_connections_in" to "proxy.config.net.max_active_requests_in"
> 3) proxy.config.net.connections_throttle - This config will stay put to limit the max number of open connections (FD, rlimits etc)
>
> More context on the new changes -
>
>
> The idea is to tune active connections that can be handled (based on the available resources/capacity (CPU, Memory, Network bandwidth) and minimize/remove dependency on having to tune connection timeouts (active/inactive/keep-alive etc) which are very hard to tune.
>
> The primary goal is to limit the max active connections allowed at any given instant. The resource requirement for an idle/inactive vs active connections are completely different - For e.g an idle connection really only consumes memory resource, while an active connection consumes CPU, network besides memory. And allowing to tune/cap the max active connections based on the deployed capacity for the resources available, would make timeout tuning almost redundant and no op. Otherwise, we'd have to tune the timeouts to estimate throughput which is very hard as it's hard to justify how large or small we want the active timeout to be or keep alive timeout to be. For e.g in a non-peak hour, we could let the active timeout be much higher than the default, while during peak hour, we'd want to limit it to ensure we are not starving resources on one connection.
> Note: there's one little TODO item here. PluginVC's connections are not tracked in the NetVC's active queue because these are bogus internal connections. However, for some users, internal traffic is significant (e.g sideways calls, or background fetches etc) and does consume plenty of resources. Since these internal requests don't impact ingress network, and have a slightly different resource requirements than client originated requests, it might be better to track these using a separate config for internal requests. Will follow that part up with a separate PR.
>
>
Re: Connection and Timeout Tuning, config name changes in 9.x
Posted by Bryan Call <bc...@apache.org>.
After seeing the GitHub PR I think we should keep proxy.config.net.max_connections_in since it is the number of socket connections (active + idle) and then rename proxy.config.net.max_active_connections_in, as you suggested in your email, to proxy.config.net.max_active_requests_in or proxy.config.net.max_requests_in (as Sudheer suggested on Slack).
I would like to see us get rid of proxy.config.net.connections_throttle in the long term and use max_connections_in/out if possible.
-Bryan
> On May 12, 2020, at 5:17 PM, Sudheer Vinukonda <su...@yahoo.com.INVALID> wrote:
>
> 9.x (re)introduces a feature (it was accidentally "lost" during some unrelated refactoring) that will allow to configure limits on max "active" connections that can be allowed at any given time (https://github.com/apache/trafficserver/pull/6754)
> Given that this feature is based on tracking "active connections", it should work very well with HTTP/1.1 traffic (where, 1 active connection = 1 request). However, with HTTP/2 and stream multiplexing, this is no longer true and it isn't nearly as effective. One can still use an average number of concurrent streams/connection to configure the limits, but, ideally, what would work for both HTTP/1.1 and HTTP/2 (and going forward with QUIC/HTTP/3) would be to track and (rate) limit request level concurrency. There is some ongoing work on this and to align better with that work, we are proposing to make the below changes to existing configs with 9.x
>
> 1) Rename "proxy.config.net.max_connections_in" to "proxy.config.net.max_requests_in"
> 2) Rename "proxy.config.net.max_active_connections_in" to "proxy.config.net.max_active_requests_in"
> 3) proxy.config.net.connections_throttle - This config will stay put to limit the max number of open connections (FD, rlimits etc)
>
> More context on the new changes -
>
>
> The idea is to tune active connections that can be handled (based on the available resources/capacity (CPU, Memory, Network bandwidth) and minimize/remove dependency on having to tune connection timeouts (active/inactive/keep-alive etc) which are very hard to tune.
>
> The primary goal is to limit the max active connections allowed at any given instant. The resource requirement for an idle/inactive vs active connections are completely different - For e.g an idle connection really only consumes memory resource, while an active connection consumes CPU, network besides memory. And allowing to tune/cap the max active connections based on the deployed capacity for the resources available, would make timeout tuning almost redundant and no op. Otherwise, we'd have to tune the timeouts to estimate throughput which is very hard as it's hard to justify how large or small we want the active timeout to be or keep alive timeout to be. For e.g in a non-peak hour, we could let the active timeout be much higher than the default, while during peak hour, we'd want to limit it to ensure we are not starving resources on one connection.
> Note: there's one little TODO item here. PluginVC's connections are not tracked in the NetVC's active queue because these are bogus internal connections. However, for some users, internal traffic is significant (e.g sideways calls, or background fetches etc) and does consume plenty of resources. Since these internal requests don't impact ingress network, and have a slightly different resource requirements than client originated requests, it might be better to track these using a separate config for internal requests. Will follow that part up with a separate PR.
>
>