You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "Oleg Kalnichevski (JIRA)" <ji...@apache.org> on 2018/04/27 21:54:00 UTC
[jira] [Resolved] (HTTPCLIENT-1918) CLOSE_WAIT issue in HttpClient
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleg Kalnichevski resolved HTTPCLIENT-1918.
-------------------------------------------
Resolution: Invalid
As I suspected the connection with id 10699 got leased (there is 'Connection leased' log entry) but never released back to the pool (there is no corresponding 'Connection released' log entry). HttpClient assumes it is still being used and never attempts to close it.
Please fix connection leak in your code.
Oleg
{noformat}
Apr27 19:30:27,271 DEBUG PoolingHttpClientConnectionManager:265 - Connection request: [route: {}->http://localhost:9092][total kept alive: 0; route allocated: 14 of 300; total allocated: 14 of 300]
Apr27 19:30:27,271 DEBUG PoolingHttpClientConnectionManager:309 - Connection leased: [id: 10699][route: {}->http://localhost:9092][total kept alive: 0; route allocated: 15 of 300; total allocated: 15 of 300]
Apr27 19:30:27,271 DEBUG DefaultHttpClientConnectionOperator:139 - Connecting to localhost/127.0.0.1:9092
Apr27 19:30:27,272 DEBUG DefaultHttpClientConnectionOperator:146 - Connection established 127.0.0.1:39003<->127.0.0.1:9092
Apr27 19:30:27,272 DEBUG DefaultManagedHttpClientConnection:88 - http-outgoing-10699: set socket timeout to 4000
Apr27 19:30:27,272 DEBUG wire:73 - http-outgoing-10699 >> "PUT /<<conextroot>>/portal/scripts/NA/runs/5142742271/log HTTP/1.1[\r][\n]"
Apr27 19:30:27,273 DEBUG wire:73 - http-outgoing-10699 >> "Content-Length: 313[\r][\n]"
Apr27 19:30:27,273 DEBUG wire:73 - http-outgoing-10699 >> "Content-Type: application/json; charset=ISO-8859-1[\r][\n]"
Apr27 19:30:27,273 DEBUG wire:73 - http-outgoing-10699 >> "Host: localhost:9092[\r][\n]"
Apr27 19:30:27,273 DEBUG wire:73 - http-outgoing-10699 >> "Connection: Keep-Alive[\r][\n]"
Apr27 19:30:27,273 DEBUG wire:73 - http-outgoing-10699 >> "User-Agent: Apache-HttpClient/4.5.5 (Java/1.7.0_80)[\r][\n]"
Apr27 19:30:27,274 DEBUG wire:73 - http-outgoing-10699 >> "Accept-Encoding: gzip,deflate[\r][\n]"
Apr27 19:30:27,274 DEBUG wire:73 - http-outgoing-10699 >> "Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==[\r][\n]"
Apr27 19:30:27,274 DEBUG wire:73 - http-outgoing-10699 >> "[\r][\n]"
Apr27 19:30:27,274 DEBUG wire:87 - http-outgoing-10699 >> "{"message":"(new JavaException(\"org.mozilla.javascript.WrappedException: Wrapped com.xxxx.yyyy.script.thread.ScriptLockTimeoutException: Script lock timeout by owner: device
{noformat}
> CLOSE_WAIT issue in HttpClient
> ------------------------------
>
> Key: HTTPCLIENT-1918
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1918
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (async)
> Affects Versions: 4.5.5
> Reporter: Santhosh Sheshasayanan
> Priority: Major
> Attachments: Capture-0.PNG, Capture-1.PNG, Capture.PNG, IdleConnectionMonitorThread.java, close_wait.log, frt-acs03-netstat.log
>
>
> We are facing CLOSE_WAIT issue in our production system on heavy load. We are using Apache's httpClient 4.5.5 and httpcore 4.4.9 in our code. Sometime the server sends half of the data; sometime server sends no data and close the connection at their end. Now, httpclient puts the socket connection to CLOSE_WAIT indefinitely.
> We tried the following but nothing helped us.
> * Set TTL to 1 minutes in PoolingHttpClientConnectionManager
> * Used the following methods
> ** evictExpiredConnections()
> ** evictIdleConnections(5, TimeUnit._SECONDS_)
> ** [https://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html|https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhc.apache.org%2Fhttpcomponents-client-ga%2Ftutorial%2Fhtml%2Fconnmgmt.html&data=02%7C01%7Ckumaran.ae%40hcl.com%7Ce88009e18e4145e27ebc08d5aa2ae38c%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C636602023532727032&sdata=2s0L63M%2F%2F7slLF3Uhx2Ly2SHSiknz0hA4fF8wDvxbS0%3D&reserved=0]
> *** Topic "2.5. Connection eviction policy"
> *** Topic "2.6. Connection keep alive strategy"
> Later, we found the method "setValidateAfterInactivity". Its description says "This check helps detect connections that have become stale (half-closed) while kept inactive in the pool". So we thought it will help us to detect and close the socket which are in "CLOSE_WAIT" state. But it did not help us.
> Problem here, The HTTP client puts the socket into "CLOSE_WAIT" state and waiting for the signal from server to close the socket. But the server closed the connection already. Please suggest us how to handle/resolve the issue?
>
> +*Scala Code Snippet*+
> private val clientConnectionManager =
> { val clientConnectionManager = new PoolingHttpClientConnectionManager(1, java.util.concurrent.TimeUnit.MINUTES); clientConnectionManager.setMaxTotal(cfg.connectionPoolSize) clientConnectionManager.setDefaultMaxPerRoute(cfg.connectionPoolSize) clientConnectionManager.setValidateAfterInactivity(30000);
> clientConnectionManager }
> *val monitorThread = new IdleConnectionMonitorThread(clientConnectionManager);*
> *monitorThread.start();*
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org