You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by David Ayre <Da...@fvreb.bc.ca> on 2015/09/14 18:48:03 UTC
GZIP encoding, EOF detector & leaked connections
I'm using httpclient 4.4.1 with Jersey 2.19 and have spent some time troubleshooting a problem with http connections to a specific server A leased from the PoolingClientConnectionManager which are never made available again because they are never released. Our per route for A maxes out and a portion of our app locks up within the connection manger waiting for an available connection.
Connections are properly released when i connected to a different server B, so i compared the calls from both A & B and the one difference i noticed was the successful one to B was not getting a compressed response.
When connecting to server B, the EOF detector released the connection to the connection manager successfully, for A with a compressed response, the EOF detector never releases the connection.
When i enable the "contentCompressionDisabled" flag manually for A within the HttpClientBuilder, the connection to server A works as the EOF detector correctly releases the connection.
Should gzip compression affect the ability for the EOF detector to release the connection back to the pool ? If not, then i think this may be a bug.
David
RE: GZIP encoding, EOF detector & leaked connections
Posted by David Ayre <Da...@fvreb.bc.ca>.
Thank you for your help Oleg, very much appreciated.
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Tuesday, September 15, 2015 8:18 AM
To: HttpClient User Discussion
Subject: Re: GZIP encoding, EOF detector & leaked connections
On Tue, 2015-09-15 at 08:10 -0700, David Ayre wrote:
> Hi Oleg,
>
> I think you are probably right... we can read the entity fully, it's just that the connection is never released... & we have very little control over these remote servers. The only solution i can think of in this case is to disable content compression, don't accept gzip.
>
> Are there any other solutions you can think of ?
>
(1) Disable automatic content decompression, (2) add 'Accept-Encoding:
gzip' using a protocol interceptor or manually (3) decode content manually.
Oleg
> David
>
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Tuesday, September 15, 2015 12:33 AM
> To: HttpClient User Discussion
> Subject: Re: GZIP encoding, EOF detector & leaked connections
>
> On Mon, 2015-09-14 at 09:48 -0700, David Ayre wrote:
> > I'm using httpclient 4.4.1 with Jersey 2.19 and have spent some time troubleshooting a problem with http connections to a specific server A leased from the PoolingClientConnectionManager which are never made available again because they are never released. Our per route for A maxes out and a portion of our app locks up within the connection manger waiting for an available connection.
> >
> > Connections are properly released when i connected to a different server B, so i compared the calls from both A & B and the one difference i noticed was the successful one to B was not getting a compressed response.
> >
> > When connecting to server B, the EOF detector released the connection to the connection manager successfully, for A with a compressed response, the EOF detector never releases the connection.
> >
> > When i enable the "contentCompressionDisabled" flag manually for A within the HttpClientBuilder, the connection to server A works as the EOF detector correctly releases the connection.
> >
> > Should gzip compression affect the ability for the EOF detector to release the connection back to the pool ? If not, then i think this may be a bug.
> >
> > David
>
> David
>
> While this may well be a bug in HC I think it is more likely that GZIP content stream is incorrectly terminated which causes HC to assume that more data is still to come.
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: GZIP encoding, EOF detector & leaked connections
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2015-09-15 at 08:10 -0700, David Ayre wrote:
> Hi Oleg,
>
> I think you are probably right... we can read the entity fully, it's just that the connection is never released... & we have very little control over these remote servers. The only solution i can think of in this case is to disable content compression, don't accept gzip.
>
> Are there any other solutions you can think of ?
>
(1) Disable automatic content decompression, (2) add 'Accept-Encoding:
gzip' using a protocol interceptor or manually (3) decode content
manually.
Oleg
> David
>
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Tuesday, September 15, 2015 12:33 AM
> To: HttpClient User Discussion
> Subject: Re: GZIP encoding, EOF detector & leaked connections
>
> On Mon, 2015-09-14 at 09:48 -0700, David Ayre wrote:
> > I'm using httpclient 4.4.1 with Jersey 2.19 and have spent some time troubleshooting a problem with http connections to a specific server A leased from the PoolingClientConnectionManager which are never made available again because they are never released. Our per route for A maxes out and a portion of our app locks up within the connection manger waiting for an available connection.
> >
> > Connections are properly released when i connected to a different server B, so i compared the calls from both A & B and the one difference i noticed was the successful one to B was not getting a compressed response.
> >
> > When connecting to server B, the EOF detector released the connection to the connection manager successfully, for A with a compressed response, the EOF detector never releases the connection.
> >
> > When i enable the "contentCompressionDisabled" flag manually for A within the HttpClientBuilder, the connection to server A works as the EOF detector correctly releases the connection.
> >
> > Should gzip compression affect the ability for the EOF detector to release the connection back to the pool ? If not, then i think this may be a bug.
> >
> > David
>
> David
>
> While this may well be a bug in HC I think it is more likely that GZIP content stream is incorrectly terminated which causes HC to assume that more data is still to come.
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: GZIP encoding, EOF detector & leaked connections
Posted by David Ayre <Da...@fvreb.bc.ca>.
Hi Oleg,
I think you are probably right... we can read the entity fully, it's just that the connection is never released... & we have very little control over these remote servers. The only solution i can think of in this case is to disable content compression, don't accept gzip.
Are there any other solutions you can think of ?
David
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Tuesday, September 15, 2015 12:33 AM
To: HttpClient User Discussion
Subject: Re: GZIP encoding, EOF detector & leaked connections
On Mon, 2015-09-14 at 09:48 -0700, David Ayre wrote:
> I'm using httpclient 4.4.1 with Jersey 2.19 and have spent some time troubleshooting a problem with http connections to a specific server A leased from the PoolingClientConnectionManager which are never made available again because they are never released. Our per route for A maxes out and a portion of our app locks up within the connection manger waiting for an available connection.
>
> Connections are properly released when i connected to a different server B, so i compared the calls from both A & B and the one difference i noticed was the successful one to B was not getting a compressed response.
>
> When connecting to server B, the EOF detector released the connection to the connection manager successfully, for A with a compressed response, the EOF detector never releases the connection.
>
> When i enable the "contentCompressionDisabled" flag manually for A within the HttpClientBuilder, the connection to server A works as the EOF detector correctly releases the connection.
>
> Should gzip compression affect the ability for the EOF detector to release the connection back to the pool ? If not, then i think this may be a bug.
>
> David
David
While this may well be a bug in HC I think it is more likely that GZIP content stream is incorrectly terminated which causes HC to assume that more data is still to come.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: GZIP encoding, EOF detector & leaked connections
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2015-09-14 at 09:48 -0700, David Ayre wrote:
> I'm using httpclient 4.4.1 with Jersey 2.19 and have spent some time troubleshooting a problem with http connections to a specific server A leased from the PoolingClientConnectionManager which are never made available again because they are never released. Our per route for A maxes out and a portion of our app locks up within the connection manger waiting for an available connection.
>
> Connections are properly released when i connected to a different server B, so i compared the calls from both A & B and the one difference i noticed was the successful one to B was not getting a compressed response.
>
> When connecting to server B, the EOF detector released the connection to the connection manager successfully, for A with a compressed response, the EOF detector never releases the connection.
>
> When i enable the "contentCompressionDisabled" flag manually for A within the HttpClientBuilder, the connection to server A works as the EOF detector correctly releases the connection.
>
> Should gzip compression affect the ability for the EOF detector to release the connection back to the pool ? If not, then i think this may be a bug.
>
> David
David
While this may well be a bug in HC I think it is more likely that GZIP
content stream is incorrectly terminated which causes HC to assume that
more data is still to come.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org