[jira] [Created] (TS-2267) Leaking file descriptors - no inactivity timeout set

["David Carlin (JIRA)" <ji...@apache.org>]

David Carlin created TS-2267:
--------------------------------

             Summary: Leaking file descriptors - no inactivity timeout set
                 Key: TS-2267
                 URL: https://issues.apache.org/jira/browse/TS-2267
             Project: Traffic Server
          Issue Type: Bug
          Components: Core, Network
            Reporter: David Carlin


We noticed a problem on two of our *forward proxy* hosts where ATS would run out of file descriptors.  Even after a host was removed from a VIP for hours, ATS would report:

proxy.process.http.current_server_connections = 0
proxy.process.http.current_active_client_connections = 43559

# ss -s
Total: 43622 (kernel 43688)
TCP:   43632 (estab 2, closed 39553, orphaned 0, synrecv 0, timewait 87/0), ports 55

Note that ATS reports 43559 active connections, but there are only 2 established connections on the host.  On this particular host, proxy.config.net.connections_throttle was set to 50000 so we first noticed the problem because ATS started throttling and never recovered - the host kept flapping as the connection count stayed near the throttling limit. 

Bryan added some debugging to inactivity cop to create a log entry every time there was a connection with no inactivity timeout set - this helped confirm the issue.  He then made a patch to set a fixed 5 min timeout if there is no inactivity timeout specified - this helped stabilize the issue. 

The only zero timeout in records.config was "proxy.config.http.transaction_active_timeout_out = 0" which is the default.  Changing this to something else did not alleviate the problem - we could still see in the debug logs that connections had no inactivity timeout set.

We downgraded from 4.0.1 to 3.2.0 and the problem still occured in 3.2.0

Thanks to Bryan Call for troubleshooting/patching!



--
This message was sent by Atlassian JIRA
(v6.1#6144)