[jira] Updated: (THRIFT-970) Under heavy load, THttpClient may fail with "too many open files"

["Mathias Herberts (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/THRIFT-970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mathias Herberts updated THRIFT-970:
------------------------------------

    Attachment: THRIFT-970-1

This patch introduces the THttpTransport class which intends to be a drop in replacement for THttpClient (apart from the constructor which differs slightly).

In order to quickly release resources associated with connections, response content is stored in a ByteArrayInputStream before being made available to the enclosing TProtocol.

This means that the memory footprint might be double that of the response (the THttpTransport buffer and the TBase being filled).

> Under heavy load, THttpClient may fail with "too many open files"
> -----------------------------------------------------------------
>
>                 Key: THRIFT-970
>                 URL: https://issues.apache.org/jira/browse/THRIFT-970
>             Project: Thrift
>          Issue Type: Bug
>          Components: Java - Library
>    Affects Versions: 0.2, 0.3, 0.4, 0.5
>         Environment: All
>            Reporter: Mathias Herberts
>         Attachments: THRIFT-970-1
>
>
> THttpClient uses URL.openConnection which returns a HttpUrlConnection instance for each message transmission.
> HttpUrlConnection supposedly pools connections to the server. While stress testing an application, we've noticed that after several thousands requests, THttpClient would fail with a "Too many open files" error thrown in java.net.Socket.createImpl called from THttpClient.flush.
> As the underlying connection to the server is internally handled by HttpUrlConnection and the JVM, there is unfortunately not much that can be done to remedy this problem while still using HttpUrlConnection.
> I propose a new implementation of THttpClient which uses Apache HttpClient from Http Components instead of HttpUrlConnection.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.