You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Simon Kulessa <ku...@flexsecure.de> on 2013/12/13 11:50:37 UTC
HttpAsyncClient resource consumption
Hi,
when a HttpAsyncClient is created I observe that for each client Java
creates 3 Threads (see sample code below) and opens 6 TCP connection
endpoints (this I observed by using TCPView from sysinternals) .
These tcp connections seemed to be only used for local communication.
If a request is send via the execute method another TCP connection is
opened.
I tracked those down to be created by the
PoolingNHttpClientConnectionManager class. Is there a way to use a
HttpAsyncClient without doing any pooling?
For my purposes one client should only use 1 tcp connection endpoint
(instead of 7).
Regards,
Simon Kulessa
-----------------------------
Sample Code:
public static CloseableHttpAsyncClient createSimpleClient() {
CloseableHttpAsyncClient httpclient =
HttpAsyncClients.custom()
.build();
return httpclient;
}
public static void main(String[] args) throws IOException {
List<CloseableHttpAsyncClient> clients = new
ArrayList<CloseableHttpAsyncClient>();
for (int i = 0; i < 10; i++) {
System.out.println(i + "\t" + Thread.activeCount());
CloseableHttpAsyncClient client = createSimpleClient();
client.start();
clients.add(client);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
System.out.println(Thread.activeCount());
}
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HttpAsyncClient resource consumption
Posted by Simon Kulessa <ku...@flexsecure.de>.
Hello Oleg,
thanks for your fast reply.
I verified what you said, by running my sample under linux as well.
It seems you are right that this seems to be something windows specific.
> You can force the pool to have one connection only but I doubt that your
> problem has anything to do with connection pooling.
If you refer to setMaxTotal(1) on the
PoolingNHttpClientConnectionManager that I already tried but to no avail.
Two out of those 6 TCP endpoints, are created by just building the
client. The Threads and the other 4 TCP endpoint are coming from
calling the start method.
I also executed 'Selector selector = Selector.open();'
by myself and could verify that this call indeed seems to open up 2 TCP
endpoints under Windows 7.
Regards,
Simon Kulessa.
Am 13.12.2013 12:18, schrieb Oleg Kalnichevski:
> On Fri, 2013-12-13 at 11:50 +0100, Simon Kulessa wrote:
>> Hi,
>>
>> when a HttpAsyncClient is created I observe that for each client Java
>> creates 3 Threads (see sample code below) and opens 6 TCP connection
>> endpoints (this I observed by using TCPView from sysinternals) .
>>
>
> Simon,
> By default HttpAsyncClient creates 1+N selectors and the equal number of
> threads (one connector thread and N i/o dispatch threads), where N is
> the number of CPU cores. I do not know how exactly NIO selector are
> implemented in Windows, but HttpAsyncClient does not explicitly open any
> outgoing connections by default and the connection pool is always empty
> on startup. Under Ubuntu Linux I only see one endpoint open by the
> process and that endpoint is actually open by the IDE (Idea Intellij in
> my case). I see no endpoints open by HttpAsyncClient until it actually
> executes a request and establishes an outgoing connection.
>
> You can force HttpAsyncClient to use only one I/O dispatch thread, but
> you will still end up with one connector thread and one i/o dispatch
> thread (and two NIO selectors).
>
>
>> These tcp connections seemed to be only used for local communication.
>> If a request is send via the execute method another TCP connection is
>> opened.
>>
>> I tracked those down to be created by the
>> PoolingNHttpClientConnectionManager class. Is there a way to use a
>> HttpAsyncClient without doing any pooling?
>>
>> For my purposes one client should only use 1 tcp connection endpoint
>> (instead of 7).
>>
>
> You can force the pool to have one connection only but I doubt that your
> problem has anything to do with connection pooling.
>
> Oleg
>
>
>> Regards,
>> Simon Kulessa
>>
>> -----------------------------
>> Sample Code:
>>
>> public static CloseableHttpAsyncClient createSimpleClient() {
>>
>> CloseableHttpAsyncClient httpclient =
>> HttpAsyncClients.custom()
>> .build();
>>
>> return httpclient;
>> }
>>
>> public static void main(String[] args) throws IOException {
>>
>> List<CloseableHttpAsyncClient> clients = new
>> ArrayList<CloseableHttpAsyncClient>();
>>
>> for (int i = 0; i < 10; i++) {
>> System.out.println(i + "\t" + Thread.activeCount());
>>
>> CloseableHttpAsyncClient client = createSimpleClient();
>> client.start();
>> clients.add(client);
>>
>> try {
>> Thread.sleep(1000);
>> } catch (InterruptedException e) {
>> e.printStackTrace();
>> }
>> }
>>
>> System.out.println(Thread.activeCount());
>> }
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HttpAsyncClient resource consumption
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Fri, 2013-12-13 at 11:50 +0100, Simon Kulessa wrote:
> Hi,
>
> when a HttpAsyncClient is created I observe that for each client Java
> creates 3 Threads (see sample code below) and opens 6 TCP connection
> endpoints (this I observed by using TCPView from sysinternals) .
>
Simon,
By default HttpAsyncClient creates 1+N selectors and the equal number of
threads (one connector thread and N i/o dispatch threads), where N is
the number of CPU cores. I do not know how exactly NIO selector are
implemented in Windows, but HttpAsyncClient does not explicitly open any
outgoing connections by default and the connection pool is always empty
on startup. Under Ubuntu Linux I only see one endpoint open by the
process and that endpoint is actually open by the IDE (Idea Intellij in
my case). I see no endpoints open by HttpAsyncClient until it actually
executes a request and establishes an outgoing connection.
You can force HttpAsyncClient to use only one I/O dispatch thread, but
you will still end up with one connector thread and one i/o dispatch
thread (and two NIO selectors).
> These tcp connections seemed to be only used for local communication.
> If a request is send via the execute method another TCP connection is
> opened.
>
> I tracked those down to be created by the
> PoolingNHttpClientConnectionManager class. Is there a way to use a
> HttpAsyncClient without doing any pooling?
>
> For my purposes one client should only use 1 tcp connection endpoint
> (instead of 7).
>
You can force the pool to have one connection only but I doubt that your
problem has anything to do with connection pooling.
Oleg
> Regards,
> Simon Kulessa
>
> -----------------------------
> Sample Code:
>
> public static CloseableHttpAsyncClient createSimpleClient() {
>
> CloseableHttpAsyncClient httpclient =
> HttpAsyncClients.custom()
> .build();
>
> return httpclient;
> }
>
> public static void main(String[] args) throws IOException {
>
> List<CloseableHttpAsyncClient> clients = new
> ArrayList<CloseableHttpAsyncClient>();
>
> for (int i = 0; i < 10; i++) {
> System.out.println(i + "\t" + Thread.activeCount());
>
> CloseableHttpAsyncClient client = createSimpleClient();
> client.start();
> clients.add(client);
>
> try {
> Thread.sleep(1000);
> } catch (InterruptedException e) {
> e.printStackTrace();
> }
> }
>
> System.out.println(Thread.activeCount());
> }
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org