You are viewing a plain text version of this content. The canonical link for it is here.

Posted to httpclient-users@hc.apache.org by Joan Balagueró <jo...@grupoventus.com> on 2018/10/19 13:08:41 UTC

Migration from Async 4.1.3 to HttpClient 5

Hello,

 

We are in the process of migrating to HttpClient5 from AsyncClient4.1.3, and
we have some quiestions:

 

1.       this.rc = RequestConfig.custom().setAuthenticationEnabled(false).

 
setConnectionRequestTimeout(Timeout.ofMillis(this.poolTimeout)).

 
setConnectionRequestTimeout(this.poolTimeout, TimeUnit.MILLISECONDS).

 
setConnectionTimeout(Timeout.ofMillis(this.connectionTimeout)).

 
setConnectTimeout(this.connectionTimeout, TimeUnit.MILLISECONDS).

 

                I wrote here the 4 possible methods setConnectionxxxxx.
The two first have the same name and different parameters, two ways to do
the same.

But the next two methods seem to be the same, but the method names are
different. Shouldnt the fourth method be setConnectionTimeout as well?

                

 

2.       The
PoolingAsyncClientConnectionManagerBuilder.setConnectionTimeToLive() is the
keep alive to set to connections in this pool?

(similar to what we do in HttpAsyncClients.custom().setKeepAliveStrategy()?)

 

3.       In previous AsyncClient, we set the setSocketTimeout at
RequestConfig level, so on each request I could copy this base rc and
modify the response timeout.

But this method does no longer exist in RequestConfig. Where can I set the
response timeout now for each request?

 

4.       I imagine the PoolConcurrencyPolicy cannot be modified after
creating the pool 

 

5.       Before I had this method to set buffer sizes (and modify them on
the fly), getting the ConnectionConfig from the pool:

 

public void setBufferInInterface(int bufferInInterface) { 

 
this.phccm.setDefaultConnectionConfig(ConnectionConfig.custom().setBufferSiz
e(bufferInInterface).setFragmentSizeHint(bufferInInterface).build()); 

}

 

How can I do this now?

 

 

Thanks,

 

Joan.

Re: RV: Migration from Async 4.1.3 to HttpClient 5

Posted by Gary Gregory <ga...@gmail.com>.

On Fri, Nov 9, 2018 at 7:01 AM Oleg Kalnichevski <ol...@apache.org> wrote:

> On Fri, 2018-11-09 at 13:39 +0100, Joan Balagueró wrote:
> > Thanks Oleg. One more thing about the max connections with lax/strict
> > pool. Our code to modify the number of max connections is:
> >
> > public void setMaxConnections(int maxConnections)
> >  {
> >   this.phccm.setMaxTotal(maxConnections);
> >   this.phccm.setDefaultMaxPerRoute(maxConnections);
> > }
> >
> > If I modify (on the fly) the max connections in a strict pool it
> > works. For example I set a very low value and I start to receive
> > DeadlineTimeoutException, when I set a higher value the error
> > disappears. If I print the pool.getMaxTotal() I get the right value.
> >
> > But this does not work with a lax pool. I set up a lax pool with max
> > connections = 1, and no DeadlineTimeoutException is thrown (with the
> > same load). When I print the maxTotal and defaultMaxPerRoute I get 0
> > and 1 (instead of 1 and 1).
> >
> > Is this a bug or am I missing something?
> >
>
> Max total is not enforced by the lax pool, only max per route.
>

Hi,

Can you make sure this is Javadoc'd in the right places if it is not
already?

Gary

>
> Oleg
>
>
> > Thanks,
> >
> > Joan.
> >
> >
> > -----Mensaje original-----
> > De: Oleg Kalnichevski [mailto:olegk@apache.org]
> > Enviado el: jueves, 8 de noviembre de 2018 11:04
> > Para: HttpClient User Discussion
> > Asunto: Re: RV: Migration from Async 4.1.3 to HttpClient 5
> >
> > On Wed, 2018-11-07 at 19:30 +0100, Joan Balagueró wrote:
> > > Hello Oleg,
> > >
> > > We are finishing the migration and have the last questions:
> > >
> > > 1. If a connection is kept-alive for 30s at second 0, and after 10s
> > > is
> > > reused, this connection will die at second 30 or will survive
> > > until
> > > second 40?
> >
> > Keep-alive value is always relative to the last connection release.
> > If you want to limit the absolute connection life time please use set
> > a finite TTL (total time to live) value.
> >
> > >
> > > 2. Regarding the RetryHandler, below the method inherited from http
> > > 4.5 and modified to work with http5:
> > >
> >
> > I would recommend using DefaultHttpRequestRetryHandler shipped with
> > the library unless you have some application specific requirements.
> >
> >
>
> https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultHttpRequestRetryHandler.java#L160
> >
> > > public boolean retryRequest(HttpRequest request, IOException
> > > exception, int executionCount, HttpContext context) {
> > >   // Don't retry if max retries are reached.
> > >   if (executionCount > this.maxExecutionCount) return false;
> > >
> > >   // Don't retry if any of these exceptions occur.
> > >   if (exception instanceof InterruptedIOException || exception
> > > instanceof UnknownHostException || exception instanceof
> > > ConnectException || exception instanceof SSLException) return
> > > false;
> > >
> > >   // Retry of if this request is considered 'idempotent'.
> > >   return (!(request instanceof HttpEntityEnclosingRequest)); }
> > >
> > > I understand the first two conditions are still ok (not sure if we
> > > have to add new exceptions on that list) but regarding the last
> > > condition,what would the equivalent condition be in Http5?
> > >
> >
> > I would suggest the following:
> >
> >
>
> https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultHttpRequestRetryHandler.java#L160
> >
> >
> > >
> > > 3. We have increased the response time of our backend (ip ended
> > > with
> > > '182') in order to exhaust the strict/lax pool. When this happens
> > > the
> > > pool starts to throw a DeadlineTimeoutException. At this moment
> > > the
> > > number of sockets in TIME_WAIT increases a lot until making the
> > > server
> > > unresponsive (probably exhausting the local ports):
> > >
> > >  [root@ns3103538 ~]# netstat -anp | grep TIME_WAIT | grep "179.182"
> > > |
> > > wc -l
> > > 99
> > > [root@ns3103538 ~]# netstat -anp | grep TIME_WAIT | grep "179.182"
> > > |
> > > wc -l
> > > 101
> > > [root@ns3103538 ~]# netstat -anp | grep TIME_WAIT | grep "179.182"
> > > |
> > > wc -l
> > > 98
> > > [root@ns3103538 ~]# netstat -anp | grep TIME_WAIT | grep "179.182"
> > > |
> > > wc -l
> > > 25876
> > > [root@ns3103538 ~]# netstat -anp | grep TIME_WAIT | grep "179.182"
> > > |
> > > wc -l
> > > 61507
> > > [root@ns3103538 ~]# netstat -anp | grep TIME_WAIT | grep "179.182"
> > > |
> > > wc -l
> > > 97615
> > >
> > > Is this the right behaviour? If Http5 cannot create new
> > > connections,
> > > so no new sockets are opened, why does the number of sockets in
> > > TIME_WAIT raise at those values?
> > >
> >
> > I believe it is. There is pretty good explanation of what the
> > TIME_WAIT state represents in our old wiki:
> >
> >
>
> https://wiki.apache.org/HttpComponents/FrequentlyAskedConnectionManagementQuestions
> >
> > Oleg
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>