You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by "David Carlin (JIRA)" <ji...@apache.org> on 2013/10/08 17:11:44 UTC
[jira] [Comment Edited] (TS-2267) Leaking file descriptors - no
inactivity timeout set
[ https://issues.apache.org/jira/browse/TS-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13789261#comment-13789261 ]
David Carlin edited comment on TS-2267 at 10/8/13 3:10 PM:
-----------------------------------------------------------
I built master (commit 7ba121c9ac87a90e874c726ddd134a45d10ebfe3) and applied Bryan's patch - I still see the same problem. Bryan's patch adds this debug message, and it occurs constantly:
{noformat}
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc: 0x2aeea8028360 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc: 0x2aee600a9d20 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc: 0x2aee600479a0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc: 0x2aee600464a0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee6003c720 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee60058e00 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee60057ba0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee6005baa0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee681df510 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee60054f00 inactivity timeout not set, setting a default of 5 minutes
{noformat}
was (Author: dcarlin):
I built master (commit 7ba121c9ac87a90e874c726ddd134a45d10ebfe3) and still see the same problem. Bryan's patch adds this debug message, and it occurs constantly:
{noformat}
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc: 0x2aeea8028360 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc: 0x2aee600a9d20 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc: 0x2aee600479a0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc: 0x2aee600464a0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee6003c720 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee60058e00 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee60057ba0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee6005baa0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee681df510 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc: 0x2aee60054f00 inactivity timeout not set, setting a default of 5 minutes
{noformat}
> Leaking file descriptors - no inactivity timeout set
> ----------------------------------------------------
>
> Key: TS-2267
> URL: https://issues.apache.org/jira/browse/TS-2267
> Project: Traffic Server
> Issue Type: Bug
> Components: Core, Network
> Affects Versions: 4.0.1
> Environment: RHEL 6.4
> Reporter: David Carlin
>
> We noticed a problem on two of our *forward proxy* hosts where ATS would run out of file descriptors. Even after a host was removed from a VIP for hours, ATS would report:
> proxy.process.http.current_server_connections = 0
> proxy.process.http.current_active_client_connections = 43559
> # ss -s
> Total: 43622 (kernel 43688)
> TCP: 43632 (estab 2, closed 39553, orphaned 0, synrecv 0, timewait 87/0), ports 55
> Note that ATS reports 43559 active connections, but there are only 2 established connections on the host. On this particular host, proxy.config.net.connections_throttle was set to 50000 so we first noticed the problem because ATS started throttling and never recovered - the host kept flapping as the connection count stayed near the throttling limit.
> Bryan added some debugging to inactivity cop to create a log entry every time there was a connection with no inactivity timeout set - this helped confirm the issue. He then made a patch to set a fixed 5 min timeout if there is no inactivity timeout specified - this helped stabilize the issue.
> The only zero timeout in records.config was "proxy.config.http.transaction_active_timeout_out = 0" which is the default. Changing this to something else did not alleviate the problem - we could still see in the debug logs that connections had no inactivity timeout set.
> We downgraded from 4.0.1 to 3.2.0 and the problem still occured in 3.2.0
> Thanks to Bryan Call for troubleshooting/patching!
--
This message was sent by Atlassian JIRA
(v6.1#6144)