You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Mark Johnson <mj...@gmail.com> on 2015/05/08 16:28:06 UTC
No Response to TCP RESET (???)
Hi,
I'm having intermittent hangs with a client application that uses the async HTTPClient. (Its an HTTPS connection also.) I've turned on the diagnostics logs (attached) and performed a wire capture. My guess is that the server is not responding to a TCP RESET. One other variable - I'm going through a VPN connection to get to the server.
Below is the diagnostics of a typical sequence when the failure occurs (just the last sequence). There’s a small attachment of a Wireshark capture. I see several cases where a TCP RST occurs in the full packet capture and it seems to recover until the hang. After the last TPC RESET, nothing else happens. I do have timeout settings for the response and connection.
So I'm thinking this is a problem on the server (not responding to the TCP RST). Is there any way to verify this? The version of HTTPClient is 4.3.2.
Thanks in advance,
Mark
MainClientExec | | [exchange: 62] start execution
RequestAddCookies | | CookieSpec selected: best-match
RequestAddCookies | | Cookie [version: 0][name: vcloud-token][value: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain: 192.168.70.25][path: /][expiry: null] match [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea]
RequestAuthCache | | Auth cache not set in the context
InternalHttpAsyncClient | | [exchange: 62] Request connection for {s}->https://192.168.70.25:443
lingNHttpClientConnectionManager | | Connection request: [route: {s}->https://192.168.70.25:443][total kept alive: 1; route allocated: 1 of 2; total allocated: 1 of 20]
lingNHttpClientConnectionManager | | Connection leased: [id: http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
InternalHttpAsyncClient | | [exchange: 62] Connection allocated: http-outgoing-47 [ACTIVE]
ManagedNHttpClientConnectionImpl | | http-outgoing-47 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set attribute http.nio.exchange-handler
ManagedNHttpClientConnectionImpl | | http-outgoing-47 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE] Request ready
MainClientExec | | Connection route already established
MainClientExec | | [exchange: 62] Attempt 1 to execute request
MainClientExec | | Target auth state: UNCHALLENGED
MainClientExec | | Proxy auth state: UNCHALLENGED
ManagedNHttpClientConnectionImpl | | http-outgoing-47 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set timeout 50000
headers | | http-outgoing-47 >> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
headers | | http-outgoing-47 >> Accept: application/*+xml;version=5.1
headers | | http-outgoing-47 >> Connection: close
headers | | http-outgoing-47 >> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
headers | | http-outgoing-47 >> Host: 192.168.70.25
headers | | http-outgoing-47 >> User-Agent: Apache-HttpAsyncClient/4.0.1 (java 1.5)
headers | | http-outgoing-47 >> Cookie: vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
headers | | http-outgoing-47 >> Cookie2: $Version=1
ManagedNHttpClientConnectionImpl | | http-outgoing-47 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
MainClientExec | | [exchange: 62] Request completed
ManagedNHttpClientConnectionImpl | | http-outgoing-47 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: 0 bytes written
HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]: Disconnected
Nothing after this.
Re: No Response to TCP RESET (???)
Posted by Stefan Magnus Landrø <st...@gmail.com>.
I'll try to have a look at the tcpdump tomorrow.
Stefan
Sendt fra min iPhone
> Den 12. mai 2015 kl. 00.10 skrev Mark Johnson <mj...@gmail.com>:
>
> Here is the trace messages from the HTTPClient library and a wireshark capture. As I said earlier, this doesn't happen all the time.
>
> Thanks.
>
> https://www.dropbox.com/s/j2548hgx5x1biml/trace.log?dl=0
>
> https://www.dropbox.com/s/i0wxlc5rvhvx73v/tcp-reset.pcapng?dl=0
>
>
>> On 5/11/2015 2:44 AM, Oleg Kalnichevski wrote:
>>> On Sun, 2015-05-10 at 10:10 -0500, Mark Johnson wrote:
>>> What happens at this point is we have tried to issue an HTTP GET (but
>>> nothing goes out on the wire) and there's no timeout or exception
>>> thrown. So my client application hangs.
>> I cannot see any evidence of that in the log.
>>
>> Please upgrade to HttpAsyncClient 4.1 and post a _complete_ wire /
>> context log of the session that exhibits the issue.
>>
>> Oleg
>>
>>>> On 5/10/2015 6:51 AM, Oleg Kalnichevski wrote:
>>>>> On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
>>>>> Hi,
>>>>>
>>>>> I'm having intermittent hangs with a client application that uses the
>>>>> async HTTPClient. (Its an HTTPS connection also.) I've turned on the
>>>>> diagnostics logs (attached) and performed a wire capture. My guess is
>>>>> that the server is not responding to a TCP RESET. One other variable -
>>>>> I'm going through a VPN connection to get to the server.
>>>>>
>>>>> Below is the diagnostics of a typical sequence when the failure occurs
>>>>> (just the last sequence). There’s a small attachment of a Wireshark
>>>>> capture. I see several cases where a TCP RST occurs in the full
>>>>> packet capture and it seems to recover until the hang. After the last
>>>>> TPC RESET, nothing else happens. I do have timeout settings for the
>>>>> response and connection.
>>>> What else do you expect to happen? As far as I can tell the client
>>>> closed the connection in response to server closing the connection on
>>>> its end.
>>>>
>>>> Oleg
>>>>
>>>>> So I'm thinking this is a problem on the server (not responding to
>>>>> the TCP RST). Is there any way to verify this? The version of
>>>>> HTTPClient is 4.3.2.
>>>>>
>>>>> Thanks in advance,
>>>>> Mark
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> MainClientExec | | [exchange: 62] start execution
>>>>> RequestAddCookies | | CookieSpec selected: best-match
>>>>> RequestAddCookies | | Cookie [version: 0][name:
>>>>> vcloud-token][value:
>>>>> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
>>>>> 192.168.70.25][path: /][expiry: null] match
>>>>> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea]
>>>>> RequestAuthCache | | Auth cache not set in the
>>>>> context
>>>>> InternalHttpAsyncClient | | [exchange: 62] Request
>>>>> connection for {s}->https://192.168.70.25:443
>>>>> lingNHttpClientConnectionManager | | Connection request: [route:
>>>>> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
>>>>> 1 of 2; total allocated: 1 of 20]
>>>>> lingNHttpClientConnectionManager | | Connection leased: [id:
>>>>> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
>>>>> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
>>>>> InternalHttpAsyncClient | | [exchange: 62] Connection
>>>>> allocated: http-outgoing-47 [ACTIVE]
>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set attribute http.nio.exchange-handler
>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
>>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
>>>>> Request ready
>>>>> MainClientExec | | Connection route already
>>>>> established
>>>>> MainClientExec | | [exchange: 62] Attempt 1 to
>>>>> execute request
>>>>> MainClientExec | | Target auth state: UNCHALLENGED
>>>>> MainClientExec | | Proxy auth state: UNCHALLENGED
>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set timeout 50000
>>>>> headers | | http-outgoing-47 >>
>>>>> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
>>>>> headers | | http-outgoing-47 >> Accept:
>>>>> application/*+xml;version=5.1
>>>>> headers | | http-outgoing-47 >> Connection:
>>>>> close
>>>>> headers | | http-outgoing-47 >>
>>>>> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>>>> headers | | http-outgoing-47 >> Host:
>>>>> 192.168.70.25
>>>>> headers | | http-outgoing-47 >> User-Agent:
>>>>> Apache-HttpAsyncClient/4.0.1 (java 1.5)
>>>>> headers | | http-outgoing-47 >> Cookie:
>>>>> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>>>> headers | | http-outgoing-47 >> Cookie2:
>>>>> $Version=1
>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
>>>>> MainClientExec | | [exchange: 62] Request completed
>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: 0 bytes written
>>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
>>>>> Disconnected
>>>>>
>>>>>
>>>>> Nothing after this.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: No Response to TCP RESET (???)
Posted by Mark Johnson <mj...@gmail.com>.
After upgrading to HttpAysnClient 4.1 and HttpClient 4.4.1 and testing
for the last 3 days, I have not been able to reproduce this anymore.
Thanks for pushing me in that direction.
Thanks,
Mark
On 5/12/2015 7:07 AM, Oleg Kalnichevski wrote:
> On Tue, 2015-05-12 at 07:03 -0500, Mark Johnson wrote:
>> I thought I did upgrade to 4.1. Does it still appear as 4.0.1?
>>
> Looks to me that way
>
> 2015-05-11 16:59:06,356 | DEBUG | I/O dispatcher 8 | headers
> | 96 - com.servicemesh.io - 1.0.0.0 | http-outgoing-29 >>
> GET /api/versions HTTP/1.1
> 2015-05-11 16:59:06,356 | DEBUG | I/O dispatcher 8 | headers
> | 96 - com.servicemesh.io - 1.0.0.0 | http-outgoing-29 >> Accept:
> application/*+xml;version=5.1
> 2015-05-11 16:59:06,356 | DEBUG | I/O dispatcher 8 | headers
> | 96 - com.servicemesh.io - 1.0.0.0 | http-outgoing-29 >> Host:
> 192.168.70.25
> 2015-05-11 16:59:06,356 | DEBUG | I/O dispatcher 8 | headers
> | 96 - com.servicemesh.io - 1.0.0.0 | http-outgoing-29 >> Connection:
> Keep-Alive
> 2015-05-11 16:59:06,356 | DEBUG | I/O dispatcher 8 | headers
> | 96 - com.servicemesh.io - 1.0.0.0 | http-outgoing-29 >> User-Agent:
> Apache-HttpAsyncClient/4.0.1 (java 1.5)
>
> Oleg
>
>> On 5/12/2015 2:31 AM, Oleg Kalnichevski wrote:
>>> On Mon, 2015-05-11 at 17:10 -0500, Mark Johnson wrote:
>>>> Here is the trace messages from the HTTPClient library and a wireshark
>>>> capture. As I said earlier, this doesn't happen all the time.
>>>>
>>>> Thanks.
>>>>
>>>> https://www.dropbox.com/s/j2548hgx5x1biml/trace.log?dl=0
>>>>
>>>> https://www.dropbox.com/s/i0wxlc5rvhvx73v/tcp-reset.pcapng?dl=0
>>>>
>>> Could you please try to reproduce the issue with the latest release of
>>> HttpAsyncClient, which is 4.1, instead of 4.0.1?
>>>
>>> Oleg
>>>
>>>> On 5/11/2015 2:44 AM, Oleg Kalnichevski wrote:
>>>>> On Sun, 2015-05-10 at 10:10 -0500, Mark Johnson wrote:
>>>>>> What happens at this point is we have tried to issue an HTTP GET (but
>>>>>> nothing goes out on the wire) and there's no timeout or exception
>>>>>> thrown. So my client application hangs.
>>>>>>
>>>>>>
>>>>> I cannot see any evidence of that in the log.
>>>>>
>>>>> Please upgrade to HttpAsyncClient 4.1 and post a _complete_ wire /
>>>>> context log of the session that exhibits the issue.
>>>>>
>>>>> Oleg
>>>>>
>>>>>> On 5/10/2015 6:51 AM, Oleg Kalnichevski wrote:
>>>>>>> On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I'm having intermittent hangs with a client application that uses the
>>>>>>>> async HTTPClient. (Its an HTTPS connection also.) I've turned on the
>>>>>>>> diagnostics logs (attached) and performed a wire capture. My guess is
>>>>>>>> that the server is not responding to a TCP RESET. One other variable -
>>>>>>>> I'm going through a VPN connection to get to the server.
>>>>>>>>
>>>>>>>> Below is the diagnostics of a typical sequence when the failure occurs
>>>>>>>> (just the last sequence). There’s a small attachment of a Wireshark
>>>>>>>> capture. I see several cases where a TCP RST occurs in the full
>>>>>>>> packet capture and it seems to recover until the hang. After the last
>>>>>>>> TPC RESET, nothing else happens. I do have timeout settings for the
>>>>>>>> response and connection.
>>>>>>>>
>>>>>>> What else do you expect to happen? As far as I can tell the client
>>>>>>> closed the connection in response to server closing the connection on
>>>>>>> its end.
>>>>>>>
>>>>>>> Oleg
>>>>>>>
>>>>>>>> So I'm thinking this is a problem on the server (not responding to
>>>>>>>> the TCP RST). Is there any way to verify this? The version of
>>>>>>>> HTTPClient is 4.3.2.
>>>>>>>>
>>>>>>>> Thanks in advance,
>>>>>>>> Mark
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> MainClientExec | | [exchange: 62] start execution
>>>>>>>> RequestAddCookies | | CookieSpec selected: best-match
>>>>>>>> RequestAddCookies | | Cookie [version: 0][name:
>>>>>>>> vcloud-token][value:
>>>>>>>> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
>>>>>>>> 192.168.70.25][path: /][expiry: null] match
>>>>>>>> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea]
>>>>>>>> RequestAuthCache | | Auth cache not set in the
>>>>>>>> context
>>>>>>>> InternalHttpAsyncClient | | [exchange: 62] Request
>>>>>>>> connection for {s}->https://192.168.70.25:443
>>>>>>>> lingNHttpClientConnectionManager | | Connection request: [route:
>>>>>>>> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
>>>>>>>> 1 of 2; total allocated: 1 of 20]
>>>>>>>> lingNHttpClientConnectionManager | | Connection leased: [id:
>>>>>>>> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
>>>>>>>> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
>>>>>>>> InternalHttpAsyncClient | | [exchange: 62] Connection
>>>>>>>> allocated: http-outgoing-47 [ACTIVE]
>>>>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set attribute http.nio.exchange-handler
>>>>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
>>>>>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
>>>>>>>> Request ready
>>>>>>>> MainClientExec | | Connection route already
>>>>>>>> established
>>>>>>>> MainClientExec | | [exchange: 62] Attempt 1 to
>>>>>>>> execute request
>>>>>>>> MainClientExec | | Target auth state: UNCHALLENGED
>>>>>>>> MainClientExec | | Proxy auth state: UNCHALLENGED
>>>>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set timeout 50000
>>>>>>>> headers | | http-outgoing-47 >>
>>>>>>>> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
>>>>>>>> headers | | http-outgoing-47 >> Accept:
>>>>>>>> application/*+xml;version=5.1
>>>>>>>> headers | | http-outgoing-47 >> Connection:
>>>>>>>> close
>>>>>>>> headers | | http-outgoing-47 >>
>>>>>>>> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>>>>>>> headers | | http-outgoing-47 >> Host:
>>>>>>>> 192.168.70.25
>>>>>>>> headers | | http-outgoing-47 >> User-Agent:
>>>>>>>> Apache-HttpAsyncClient/4.0.1 (java 1.5)
>>>>>>>> headers | | http-outgoing-47 >> Cookie:
>>>>>>>> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>>>>>>> headers | | http-outgoing-47 >> Cookie2:
>>>>>>>> $Version=1
>>>>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
>>>>>>>> MainClientExec | | [exchange: 62] Request completed
>>>>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: 0 bytes written
>>>>>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
>>>>>>>> Disconnected
>>>>>>>>
>>>>>>>>
>>>>>>>> Nothing after this.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: No Response to TCP RESET (???)
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2015-05-12 at 07:03 -0500, Mark Johnson wrote:
> I thought I did upgrade to 4.1. Does it still appear as 4.0.1?
>
Looks to me that way
2015-05-11 16:59:06,356 | DEBUG | I/O dispatcher 8 | headers
| 96 - com.servicemesh.io - 1.0.0.0 | http-outgoing-29 >>
GET /api/versions HTTP/1.1
2015-05-11 16:59:06,356 | DEBUG | I/O dispatcher 8 | headers
| 96 - com.servicemesh.io - 1.0.0.0 | http-outgoing-29 >> Accept:
application/*+xml;version=5.1
2015-05-11 16:59:06,356 | DEBUG | I/O dispatcher 8 | headers
| 96 - com.servicemesh.io - 1.0.0.0 | http-outgoing-29 >> Host:
192.168.70.25
2015-05-11 16:59:06,356 | DEBUG | I/O dispatcher 8 | headers
| 96 - com.servicemesh.io - 1.0.0.0 | http-outgoing-29 >> Connection:
Keep-Alive
2015-05-11 16:59:06,356 | DEBUG | I/O dispatcher 8 | headers
| 96 - com.servicemesh.io - 1.0.0.0 | http-outgoing-29 >> User-Agent:
Apache-HttpAsyncClient/4.0.1 (java 1.5)
Oleg
>
> On 5/12/2015 2:31 AM, Oleg Kalnichevski wrote:
> > On Mon, 2015-05-11 at 17:10 -0500, Mark Johnson wrote:
> >> Here is the trace messages from the HTTPClient library and a wireshark
> >> capture. As I said earlier, this doesn't happen all the time.
> >>
> >> Thanks.
> >>
> >> https://www.dropbox.com/s/j2548hgx5x1biml/trace.log?dl=0
> >>
> >> https://www.dropbox.com/s/i0wxlc5rvhvx73v/tcp-reset.pcapng?dl=0
> >>
> > Could you please try to reproduce the issue with the latest release of
> > HttpAsyncClient, which is 4.1, instead of 4.0.1?
> >
> > Oleg
> >
> >> On 5/11/2015 2:44 AM, Oleg Kalnichevski wrote:
> >>> On Sun, 2015-05-10 at 10:10 -0500, Mark Johnson wrote:
> >>>> What happens at this point is we have tried to issue an HTTP GET (but
> >>>> nothing goes out on the wire) and there's no timeout or exception
> >>>> thrown. So my client application hangs.
> >>>>
> >>>>
> >>> I cannot see any evidence of that in the log.
> >>>
> >>> Please upgrade to HttpAsyncClient 4.1 and post a _complete_ wire /
> >>> context log of the session that exhibits the issue.
> >>>
> >>> Oleg
> >>>
> >>>> On 5/10/2015 6:51 AM, Oleg Kalnichevski wrote:
> >>>>> On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> I'm having intermittent hangs with a client application that uses the
> >>>>>> async HTTPClient. (Its an HTTPS connection also.) I've turned on the
> >>>>>> diagnostics logs (attached) and performed a wire capture. My guess is
> >>>>>> that the server is not responding to a TCP RESET. One other variable -
> >>>>>> I'm going through a VPN connection to get to the server.
> >>>>>>
> >>>>>> Below is the diagnostics of a typical sequence when the failure occurs
> >>>>>> (just the last sequence). There’s a small attachment of a Wireshark
> >>>>>> capture. I see several cases where a TCP RST occurs in the full
> >>>>>> packet capture and it seems to recover until the hang. After the last
> >>>>>> TPC RESET, nothing else happens. I do have timeout settings for the
> >>>>>> response and connection.
> >>>>>>
> >>>>> What else do you expect to happen? As far as I can tell the client
> >>>>> closed the connection in response to server closing the connection on
> >>>>> its end.
> >>>>>
> >>>>> Oleg
> >>>>>
> >>>>>> So I'm thinking this is a problem on the server (not responding to
> >>>>>> the TCP RST). Is there any way to verify this? The version of
> >>>>>> HTTPClient is 4.3.2.
> >>>>>>
> >>>>>> Thanks in advance,
> >>>>>> Mark
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> MainClientExec | | [exchange: 62] start execution
> >>>>>> RequestAddCookies | | CookieSpec selected: best-match
> >>>>>> RequestAddCookies | | Cookie [version: 0][name:
> >>>>>> vcloud-token][value:
> >>>>>> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
> >>>>>> 192.168.70.25][path: /][expiry: null] match
> >>>>>> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea]
> >>>>>> RequestAuthCache | | Auth cache not set in the
> >>>>>> context
> >>>>>> InternalHttpAsyncClient | | [exchange: 62] Request
> >>>>>> connection for {s}->https://192.168.70.25:443
> >>>>>> lingNHttpClientConnectionManager | | Connection request: [route:
> >>>>>> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
> >>>>>> 1 of 2; total allocated: 1 of 20]
> >>>>>> lingNHttpClientConnectionManager | | Connection leased: [id:
> >>>>>> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
> >>>>>> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
> >>>>>> InternalHttpAsyncClient | | [exchange: 62] Connection
> >>>>>> allocated: http-outgoing-47 [ACTIVE]
> >>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set attribute http.nio.exchange-handler
> >>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
> >>>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
> >>>>>> Request ready
> >>>>>> MainClientExec | | Connection route already
> >>>>>> established
> >>>>>> MainClientExec | | [exchange: 62] Attempt 1 to
> >>>>>> execute request
> >>>>>> MainClientExec | | Target auth state: UNCHALLENGED
> >>>>>> MainClientExec | | Proxy auth state: UNCHALLENGED
> >>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set timeout 50000
> >>>>>> headers | | http-outgoing-47 >>
> >>>>>> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
> >>>>>> headers | | http-outgoing-47 >> Accept:
> >>>>>> application/*+xml;version=5.1
> >>>>>> headers | | http-outgoing-47 >> Connection:
> >>>>>> close
> >>>>>> headers | | http-outgoing-47 >>
> >>>>>> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
> >>>>>> headers | | http-outgoing-47 >> Host:
> >>>>>> 192.168.70.25
> >>>>>> headers | | http-outgoing-47 >> User-Agent:
> >>>>>> Apache-HttpAsyncClient/4.0.1 (java 1.5)
> >>>>>> headers | | http-outgoing-47 >> Cookie:
> >>>>>> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
> >>>>>> headers | | http-outgoing-47 >> Cookie2:
> >>>>>> $Version=1
> >>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
> >>>>>> MainClientExec | | [exchange: 62] Request completed
> >>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: 0 bytes written
> >>>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
> >>>>>> Disconnected
> >>>>>>
> >>>>>>
> >>>>>> Nothing after this.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: No Response to TCP RESET (???)
Posted by Mark Johnson <mj...@gmail.com>.
I thought I did upgrade to 4.1. Does it still appear as 4.0.1?
On 5/12/2015 2:31 AM, Oleg Kalnichevski wrote:
> On Mon, 2015-05-11 at 17:10 -0500, Mark Johnson wrote:
>> Here is the trace messages from the HTTPClient library and a wireshark
>> capture. As I said earlier, this doesn't happen all the time.
>>
>> Thanks.
>>
>> https://www.dropbox.com/s/j2548hgx5x1biml/trace.log?dl=0
>>
>> https://www.dropbox.com/s/i0wxlc5rvhvx73v/tcp-reset.pcapng?dl=0
>>
> Could you please try to reproduce the issue with the latest release of
> HttpAsyncClient, which is 4.1, instead of 4.0.1?
>
> Oleg
>
>> On 5/11/2015 2:44 AM, Oleg Kalnichevski wrote:
>>> On Sun, 2015-05-10 at 10:10 -0500, Mark Johnson wrote:
>>>> What happens at this point is we have tried to issue an HTTP GET (but
>>>> nothing goes out on the wire) and there's no timeout or exception
>>>> thrown. So my client application hangs.
>>>>
>>>>
>>> I cannot see any evidence of that in the log.
>>>
>>> Please upgrade to HttpAsyncClient 4.1 and post a _complete_ wire /
>>> context log of the session that exhibits the issue.
>>>
>>> Oleg
>>>
>>>> On 5/10/2015 6:51 AM, Oleg Kalnichevski wrote:
>>>>> On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I'm having intermittent hangs with a client application that uses the
>>>>>> async HTTPClient. (Its an HTTPS connection also.) I've turned on the
>>>>>> diagnostics logs (attached) and performed a wire capture. My guess is
>>>>>> that the server is not responding to a TCP RESET. One other variable -
>>>>>> I'm going through a VPN connection to get to the server.
>>>>>>
>>>>>> Below is the diagnostics of a typical sequence when the failure occurs
>>>>>> (just the last sequence). There’s a small attachment of a Wireshark
>>>>>> capture. I see several cases where a TCP RST occurs in the full
>>>>>> packet capture and it seems to recover until the hang. After the last
>>>>>> TPC RESET, nothing else happens. I do have timeout settings for the
>>>>>> response and connection.
>>>>>>
>>>>> What else do you expect to happen? As far as I can tell the client
>>>>> closed the connection in response to server closing the connection on
>>>>> its end.
>>>>>
>>>>> Oleg
>>>>>
>>>>>> So I'm thinking this is a problem on the server (not responding to
>>>>>> the TCP RST). Is there any way to verify this? The version of
>>>>>> HTTPClient is 4.3.2.
>>>>>>
>>>>>> Thanks in advance,
>>>>>> Mark
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> MainClientExec | | [exchange: 62] start execution
>>>>>> RequestAddCookies | | CookieSpec selected: best-match
>>>>>> RequestAddCookies | | Cookie [version: 0][name:
>>>>>> vcloud-token][value:
>>>>>> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
>>>>>> 192.168.70.25][path: /][expiry: null] match
>>>>>> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea]
>>>>>> RequestAuthCache | | Auth cache not set in the
>>>>>> context
>>>>>> InternalHttpAsyncClient | | [exchange: 62] Request
>>>>>> connection for {s}->https://192.168.70.25:443
>>>>>> lingNHttpClientConnectionManager | | Connection request: [route:
>>>>>> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
>>>>>> 1 of 2; total allocated: 1 of 20]
>>>>>> lingNHttpClientConnectionManager | | Connection leased: [id:
>>>>>> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
>>>>>> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
>>>>>> InternalHttpAsyncClient | | [exchange: 62] Connection
>>>>>> allocated: http-outgoing-47 [ACTIVE]
>>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set attribute http.nio.exchange-handler
>>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
>>>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
>>>>>> Request ready
>>>>>> MainClientExec | | Connection route already
>>>>>> established
>>>>>> MainClientExec | | [exchange: 62] Attempt 1 to
>>>>>> execute request
>>>>>> MainClientExec | | Target auth state: UNCHALLENGED
>>>>>> MainClientExec | | Proxy auth state: UNCHALLENGED
>>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set timeout 50000
>>>>>> headers | | http-outgoing-47 >>
>>>>>> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
>>>>>> headers | | http-outgoing-47 >> Accept:
>>>>>> application/*+xml;version=5.1
>>>>>> headers | | http-outgoing-47 >> Connection:
>>>>>> close
>>>>>> headers | | http-outgoing-47 >>
>>>>>> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>>>>> headers | | http-outgoing-47 >> Host:
>>>>>> 192.168.70.25
>>>>>> headers | | http-outgoing-47 >> User-Agent:
>>>>>> Apache-HttpAsyncClient/4.0.1 (java 1.5)
>>>>>> headers | | http-outgoing-47 >> Cookie:
>>>>>> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>>>>> headers | | http-outgoing-47 >> Cookie2:
>>>>>> $Version=1
>>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
>>>>>> MainClientExec | | [exchange: 62] Request completed
>>>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: 0 bytes written
>>>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
>>>>>> Disconnected
>>>>>>
>>>>>>
>>>>>> Nothing after this.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: No Response to TCP RESET (???)
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2015-05-11 at 17:10 -0500, Mark Johnson wrote:
> Here is the trace messages from the HTTPClient library and a wireshark
> capture. As I said earlier, this doesn't happen all the time.
>
> Thanks.
>
> https://www.dropbox.com/s/j2548hgx5x1biml/trace.log?dl=0
>
> https://www.dropbox.com/s/i0wxlc5rvhvx73v/tcp-reset.pcapng?dl=0
>
Could you please try to reproduce the issue with the latest release of
HttpAsyncClient, which is 4.1, instead of 4.0.1?
Oleg
>
> On 5/11/2015 2:44 AM, Oleg Kalnichevski wrote:
> > On Sun, 2015-05-10 at 10:10 -0500, Mark Johnson wrote:
> >> What happens at this point is we have tried to issue an HTTP GET (but
> >> nothing goes out on the wire) and there's no timeout or exception
> >> thrown. So my client application hangs.
> >>
> >>
> > I cannot see any evidence of that in the log.
> >
> > Please upgrade to HttpAsyncClient 4.1 and post a _complete_ wire /
> > context log of the session that exhibits the issue.
> >
> > Oleg
> >
> >> On 5/10/2015 6:51 AM, Oleg Kalnichevski wrote:
> >>> On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
> >>>> Hi,
> >>>>
> >>>> I'm having intermittent hangs with a client application that uses the
> >>>> async HTTPClient. (Its an HTTPS connection also.) I've turned on the
> >>>> diagnostics logs (attached) and performed a wire capture. My guess is
> >>>> that the server is not responding to a TCP RESET. One other variable -
> >>>> I'm going through a VPN connection to get to the server.
> >>>>
> >>>> Below is the diagnostics of a typical sequence when the failure occurs
> >>>> (just the last sequence). There’s a small attachment of a Wireshark
> >>>> capture. I see several cases where a TCP RST occurs in the full
> >>>> packet capture and it seems to recover until the hang. After the last
> >>>> TPC RESET, nothing else happens. I do have timeout settings for the
> >>>> response and connection.
> >>>>
> >>> What else do you expect to happen? As far as I can tell the client
> >>> closed the connection in response to server closing the connection on
> >>> its end.
> >>>
> >>> Oleg
> >>>
> >>>> So I'm thinking this is a problem on the server (not responding to
> >>>> the TCP RST). Is there any way to verify this? The version of
> >>>> HTTPClient is 4.3.2.
> >>>>
> >>>> Thanks in advance,
> >>>> Mark
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> MainClientExec | | [exchange: 62] start execution
> >>>> RequestAddCookies | | CookieSpec selected: best-match
> >>>> RequestAddCookies | | Cookie [version: 0][name:
> >>>> vcloud-token][value:
> >>>> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
> >>>> 192.168.70.25][path: /][expiry: null] match
> >>>> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea]
> >>>> RequestAuthCache | | Auth cache not set in the
> >>>> context
> >>>> InternalHttpAsyncClient | | [exchange: 62] Request
> >>>> connection for {s}->https://192.168.70.25:443
> >>>> lingNHttpClientConnectionManager | | Connection request: [route:
> >>>> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
> >>>> 1 of 2; total allocated: 1 of 20]
> >>>> lingNHttpClientConnectionManager | | Connection leased: [id:
> >>>> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
> >>>> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
> >>>> InternalHttpAsyncClient | | [exchange: 62] Connection
> >>>> allocated: http-outgoing-47 [ACTIVE]
> >>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set attribute http.nio.exchange-handler
> >>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
> >>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
> >>>> Request ready
> >>>> MainClientExec | | Connection route already
> >>>> established
> >>>> MainClientExec | | [exchange: 62] Attempt 1 to
> >>>> execute request
> >>>> MainClientExec | | Target auth state: UNCHALLENGED
> >>>> MainClientExec | | Proxy auth state: UNCHALLENGED
> >>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set timeout 50000
> >>>> headers | | http-outgoing-47 >>
> >>>> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
> >>>> headers | | http-outgoing-47 >> Accept:
> >>>> application/*+xml;version=5.1
> >>>> headers | | http-outgoing-47 >> Connection:
> >>>> close
> >>>> headers | | http-outgoing-47 >>
> >>>> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
> >>>> headers | | http-outgoing-47 >> Host:
> >>>> 192.168.70.25
> >>>> headers | | http-outgoing-47 >> User-Agent:
> >>>> Apache-HttpAsyncClient/4.0.1 (java 1.5)
> >>>> headers | | http-outgoing-47 >> Cookie:
> >>>> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
> >>>> headers | | http-outgoing-47 >> Cookie2:
> >>>> $Version=1
> >>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
> >>>> MainClientExec | | [exchange: 62] Request completed
> >>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: 0 bytes written
> >>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
> >>>> Disconnected
> >>>>
> >>>>
> >>>> Nothing after this.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: No Response to TCP RESET (???)
Posted by Mark Johnson <mj...@gmail.com>.
Here is the trace messages from the HTTPClient library and a wireshark
capture. As I said earlier, this doesn't happen all the time.
Thanks.
https://www.dropbox.com/s/j2548hgx5x1biml/trace.log?dl=0
https://www.dropbox.com/s/i0wxlc5rvhvx73v/tcp-reset.pcapng?dl=0
On 5/11/2015 2:44 AM, Oleg Kalnichevski wrote:
> On Sun, 2015-05-10 at 10:10 -0500, Mark Johnson wrote:
>> What happens at this point is we have tried to issue an HTTP GET (but
>> nothing goes out on the wire) and there's no timeout or exception
>> thrown. So my client application hangs.
>>
>>
> I cannot see any evidence of that in the log.
>
> Please upgrade to HttpAsyncClient 4.1 and post a _complete_ wire /
> context log of the session that exhibits the issue.
>
> Oleg
>
>> On 5/10/2015 6:51 AM, Oleg Kalnichevski wrote:
>>> On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
>>>> Hi,
>>>>
>>>> I'm having intermittent hangs with a client application that uses the
>>>> async HTTPClient. (Its an HTTPS connection also.) I've turned on the
>>>> diagnostics logs (attached) and performed a wire capture. My guess is
>>>> that the server is not responding to a TCP RESET. One other variable -
>>>> I'm going through a VPN connection to get to the server.
>>>>
>>>> Below is the diagnostics of a typical sequence when the failure occurs
>>>> (just the last sequence). There’s a small attachment of a Wireshark
>>>> capture. I see several cases where a TCP RST occurs in the full
>>>> packet capture and it seems to recover until the hang. After the last
>>>> TPC RESET, nothing else happens. I do have timeout settings for the
>>>> response and connection.
>>>>
>>> What else do you expect to happen? As far as I can tell the client
>>> closed the connection in response to server closing the connection on
>>> its end.
>>>
>>> Oleg
>>>
>>>> So I'm thinking this is a problem on the server (not responding to
>>>> the TCP RST). Is there any way to verify this? The version of
>>>> HTTPClient is 4.3.2.
>>>>
>>>> Thanks in advance,
>>>> Mark
>>>>
>>>>
>>>>
>>>>
>>>> MainClientExec | | [exchange: 62] start execution
>>>> RequestAddCookies | | CookieSpec selected: best-match
>>>> RequestAddCookies | | Cookie [version: 0][name:
>>>> vcloud-token][value:
>>>> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
>>>> 192.168.70.25][path: /][expiry: null] match
>>>> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea]
>>>> RequestAuthCache | | Auth cache not set in the
>>>> context
>>>> InternalHttpAsyncClient | | [exchange: 62] Request
>>>> connection for {s}->https://192.168.70.25:443
>>>> lingNHttpClientConnectionManager | | Connection request: [route:
>>>> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
>>>> 1 of 2; total allocated: 1 of 20]
>>>> lingNHttpClientConnectionManager | | Connection leased: [id:
>>>> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
>>>> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
>>>> InternalHttpAsyncClient | | [exchange: 62] Connection
>>>> allocated: http-outgoing-47 [ACTIVE]
>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set attribute http.nio.exchange-handler
>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
>>>> Request ready
>>>> MainClientExec | | Connection route already
>>>> established
>>>> MainClientExec | | [exchange: 62] Attempt 1 to
>>>> execute request
>>>> MainClientExec | | Target auth state: UNCHALLENGED
>>>> MainClientExec | | Proxy auth state: UNCHALLENGED
>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set timeout 50000
>>>> headers | | http-outgoing-47 >>
>>>> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
>>>> headers | | http-outgoing-47 >> Accept:
>>>> application/*+xml;version=5.1
>>>> headers | | http-outgoing-47 >> Connection:
>>>> close
>>>> headers | | http-outgoing-47 >>
>>>> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>>> headers | | http-outgoing-47 >> Host:
>>>> 192.168.70.25
>>>> headers | | http-outgoing-47 >> User-Agent:
>>>> Apache-HttpAsyncClient/4.0.1 (java 1.5)
>>>> headers | | http-outgoing-47 >> Cookie:
>>>> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>>> headers | | http-outgoing-47 >> Cookie2:
>>>> $Version=1
>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
>>>> MainClientExec | | [exchange: 62] Request completed
>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: 0 bytes written
>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
>>>> Disconnected
>>>>
>>>>
>>>> Nothing after this.
>>>>
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: No Response to TCP RESET (???)
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2015-05-10 at 10:10 -0500, Mark Johnson wrote:
> What happens at this point is we have tried to issue an HTTP GET (but
> nothing goes out on the wire) and there's no timeout or exception
> thrown. So my client application hangs.
>
>
I cannot see any evidence of that in the log.
Please upgrade to HttpAsyncClient 4.1 and post a _complete_ wire /
context log of the session that exhibits the issue.
Oleg
>
> On 5/10/2015 6:51 AM, Oleg Kalnichevski wrote:
> > On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
> >> Hi,
> >>
> >> I'm having intermittent hangs with a client application that uses the
> >> async HTTPClient. (Its an HTTPS connection also.) I've turned on the
> >> diagnostics logs (attached) and performed a wire capture. My guess is
> >> that the server is not responding to a TCP RESET. One other variable -
> >> I'm going through a VPN connection to get to the server.
> >>
> >> Below is the diagnostics of a typical sequence when the failure occurs
> >> (just the last sequence). There’s a small attachment of a Wireshark
> >> capture. I see several cases where a TCP RST occurs in the full
> >> packet capture and it seems to recover until the hang. After the last
> >> TPC RESET, nothing else happens. I do have timeout settings for the
> >> response and connection.
> >>
> > What else do you expect to happen? As far as I can tell the client
> > closed the connection in response to server closing the connection on
> > its end.
> >
> > Oleg
> >
> >> So I'm thinking this is a problem on the server (not responding to
> >> the TCP RST). Is there any way to verify this? The version of
> >> HTTPClient is 4.3.2.
> >>
> >> Thanks in advance,
> >> Mark
> >>
> >>
> >>
> >>
> >> MainClientExec | | [exchange: 62] start execution
> >> RequestAddCookies | | CookieSpec selected: best-match
> >> RequestAddCookies | | Cookie [version: 0][name:
> >> vcloud-token][value:
> >> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
> >> 192.168.70.25][path: /][expiry: null] match
> >> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea]
> >> RequestAuthCache | | Auth cache not set in the
> >> context
> >> InternalHttpAsyncClient | | [exchange: 62] Request
> >> connection for {s}->https://192.168.70.25:443
> >> lingNHttpClientConnectionManager | | Connection request: [route:
> >> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
> >> 1 of 2; total allocated: 1 of 20]
> >> lingNHttpClientConnectionManager | | Connection leased: [id:
> >> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
> >> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
> >> InternalHttpAsyncClient | | [exchange: 62] Connection
> >> allocated: http-outgoing-47 [ACTIVE]
> >> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set attribute http.nio.exchange-handler
> >> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
> >> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
> >> Request ready
> >> MainClientExec | | Connection route already
> >> established
> >> MainClientExec | | [exchange: 62] Attempt 1 to
> >> execute request
> >> MainClientExec | | Target auth state: UNCHALLENGED
> >> MainClientExec | | Proxy auth state: UNCHALLENGED
> >> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set timeout 50000
> >> headers | | http-outgoing-47 >>
> >> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
> >> headers | | http-outgoing-47 >> Accept:
> >> application/*+xml;version=5.1
> >> headers | | http-outgoing-47 >> Connection:
> >> close
> >> headers | | http-outgoing-47 >>
> >> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
> >> headers | | http-outgoing-47 >> Host:
> >> 192.168.70.25
> >> headers | | http-outgoing-47 >> User-Agent:
> >> Apache-HttpAsyncClient/4.0.1 (java 1.5)
> >> headers | | http-outgoing-47 >> Cookie:
> >> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
> >> headers | | http-outgoing-47 >> Cookie2:
> >> $Version=1
> >> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
> >> MainClientExec | | [exchange: 62] Request completed
> >> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> >> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: 0 bytes written
> >> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
> >> Disconnected
> >>
> >>
> >> Nothing after this.
> >>
> >>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: No Response to TCP RESET (???)
Posted by Mark Johnson <mj...@gmail.com>.
I have not done a thread dump but have a trace and wireshark capture.
However I'm trying to reproduce this using the 4.1 async client. I'll
post some files when I complete that.
On 5/12/2015 7:41 AM, Stefan Magnus Landrø wrote:
> You say the app hangs - do you know where it hangs? do you have a thread
> dump you could provide? (use jstack or jcmd)
>
> 2015-05-10 17:10 GMT+02:00 Mark Johnson <mj...@gmail.com>:
>
>> What happens at this point is we have tried to issue an HTTP GET (but
>> nothing goes out on the wire) and there's no timeout or exception thrown.
>> So my client application hangs.
>>
>>
>>
>>
>> On 5/10/2015 6:51 AM, Oleg Kalnichevski wrote:
>>
>>> On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
>>>
>>>> Hi,
>>>>
>>>> I’'m having intermittent hangs with a client application that uses the
>>>> async HTTPClient. (It’s an HTTPS connection also.) I'’ve turned on the
>>>> diagnostics logs (attached) and performed a wire capture. My guess is
>>>> that the server is not responding to a TCP RESET. One other variable -
>>>> I'm going through a VPN connection to get to the server.
>>>>
>>>> Below is the diagnostics of a typical sequence when the failure occurs
>>>> (just the last sequence). There’s a small attachment of a Wireshark
>>>> capture. I see several cases where a TCP RST occurs in the full
>>>> packet capture and it seems to recover until the hang. After the last
>>>> TPC RESET, nothing else happens. I do have timeout settings for the
>>>> response and connection.
>>>>
>>>> What else do you expect to happen? As far as I can tell the client
>>> closed the connection in response to server closing the connection on
>>> its end.
>>>
>>> Oleg
>>>
>>> So I'’m thinking this is a problem on the server (not responding to
>>>> the TCP RST). Is there any way to verify this? The version of
>>>> HTTPClient is 4.3.2.
>>>>
>>>> Thanks in advance,
>>>> Mark
>>>>
>>>>
>>>>
>>>>
>>>> MainClientExec | | [exchange: 62] start execution
>>>> RequestAddCookies | | CookieSpec selected: best-match
>>>> RequestAddCookies | | Cookie [version: 0][name:
>>>> vcloud-token][value:
>>>> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
>>>> 192.168.70.25][path: /][expiry: null] match
>>>> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea
>>>> ]
>>>> RequestAuthCache | | Auth cache not set in the
>>>> context
>>>> InternalHttpAsyncClient | | [exchange: 62] Request
>>>> connection for {s}->https://192.168.70.25:443
>>>> lingNHttpClientConnectionManager | | Connection request: [route:
>>>> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
>>>> 1 of 2; total allocated: 1 of 20]
>>>> lingNHttpClientConnectionManager | | Connection leased: [id:
>>>> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
>>>> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
>>>> InternalHttpAsyncClient | | [exchange: 62] Connection
>>>> allocated: http-outgoing-47 [ACTIVE]
>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound
>>>> done][][outbound done][][0][0][0][0]: Set attribute
>>>> http.nio.exchange-handler
>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound
>>>> done][][outbound done][][0][0][0][0]: Event set [w]
>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
>>>> Request ready
>>>> MainClientExec | | Connection route already
>>>> established
>>>> MainClientExec | | [exchange: 62] Attempt 1 to
>>>> execute request
>>>> MainClientExec | | Target auth state: UNCHALLENGED
>>>> MainClientExec | | Proxy auth state: UNCHALLENGED
>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound
>>>> done][][outbound done][][0][0][0][0]: Set timeout 50000
>>>> headers | | http-outgoing-47 >>
>>>> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
>>>> headers | | http-outgoing-47 >> Accept:
>>>> application/*+xml;version=5.1
>>>> headers | | http-outgoing-47 >> Connection:
>>>> close
>>>> headers | | http-outgoing-47 >>
>>>> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>>> headers | | http-outgoing-47 >> Host:
>>>> 192.168.70.25
>>>> headers | | http-outgoing-47 >> User-Agent:
>>>> Apache-HttpAsyncClient/4.0.1 (java 1.5)
>>>> headers | | http-outgoing-47 >> Cookie:
>>>> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>>> headers | | http-outgoing-47 >> Cookie2:
>>>> $Version=1
>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound
>>>> done][][outbound done][][0][0][0][0]: Event set [w]
>>>> MainClientExec | | [exchange: 62] Request completed
>>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound
>>>> done][][outbound done][][0][0][0][0]: 0 bytes written
>>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
>>>> Disconnected
>>>>
>>>>
>>>> Nothing after this.
>>>>
>>>>
>>>>
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: No Response to TCP RESET (???)
Posted by Stefan Magnus Landrø <st...@gmail.com>.
You say the app hangs - do you know where it hangs? do you have a thread
dump you could provide? (use jstack or jcmd)
2015-05-10 17:10 GMT+02:00 Mark Johnson <mj...@gmail.com>:
> What happens at this point is we have tried to issue an HTTP GET (but
> nothing goes out on the wire) and there's no timeout or exception thrown.
> So my client application hangs.
>
>
>
>
> On 5/10/2015 6:51 AM, Oleg Kalnichevski wrote:
>
>> On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
>>
>>> Hi,
>>>
>>> I’'m having intermittent hangs with a client application that uses the
>>> async HTTPClient. (It’s an HTTPS connection also.) I'’ve turned on the
>>> diagnostics logs (attached) and performed a wire capture. My guess is
>>> that the server is not responding to a TCP RESET. One other variable -
>>> I'm going through a VPN connection to get to the server.
>>>
>>> Below is the diagnostics of a typical sequence when the failure occurs
>>> (just the last sequence). There’s a small attachment of a Wireshark
>>> capture. I see several cases where a TCP RST occurs in the full
>>> packet capture and it seems to recover until the hang. After the last
>>> TPC RESET, nothing else happens. I do have timeout settings for the
>>> response and connection.
>>>
>>> What else do you expect to happen? As far as I can tell the client
>> closed the connection in response to server closing the connection on
>> its end.
>>
>> Oleg
>>
>> So I'’m thinking this is a problem on the server (not responding to
>>> the TCP RST). Is there any way to verify this? The version of
>>> HTTPClient is 4.3.2.
>>>
>>> Thanks in advance,
>>> Mark
>>>
>>>
>>>
>>>
>>> MainClientExec | | [exchange: 62] start execution
>>> RequestAddCookies | | CookieSpec selected: best-match
>>> RequestAddCookies | | Cookie [version: 0][name:
>>> vcloud-token][value:
>>> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
>>> 192.168.70.25][path: /][expiry: null] match
>>> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea
>>> ]
>>> RequestAuthCache | | Auth cache not set in the
>>> context
>>> InternalHttpAsyncClient | | [exchange: 62] Request
>>> connection for {s}->https://192.168.70.25:443
>>> lingNHttpClientConnectionManager | | Connection request: [route:
>>> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
>>> 1 of 2; total allocated: 1 of 20]
>>> lingNHttpClientConnectionManager | | Connection leased: [id:
>>> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
>>> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
>>> InternalHttpAsyncClient | | [exchange: 62] Connection
>>> allocated: http-outgoing-47 [ACTIVE]
>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound
>>> done][][outbound done][][0][0][0][0]: Set attribute
>>> http.nio.exchange-handler
>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound
>>> done][][outbound done][][0][0][0][0]: Event set [w]
>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
>>> Request ready
>>> MainClientExec | | Connection route already
>>> established
>>> MainClientExec | | [exchange: 62] Attempt 1 to
>>> execute request
>>> MainClientExec | | Target auth state: UNCHALLENGED
>>> MainClientExec | | Proxy auth state: UNCHALLENGED
>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound
>>> done][][outbound done][][0][0][0][0]: Set timeout 50000
>>> headers | | http-outgoing-47 >>
>>> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
>>> headers | | http-outgoing-47 >> Accept:
>>> application/*+xml;version=5.1
>>> headers | | http-outgoing-47 >> Connection:
>>> close
>>> headers | | http-outgoing-47 >>
>>> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>> headers | | http-outgoing-47 >> Host:
>>> 192.168.70.25
>>> headers | | http-outgoing-47 >> User-Agent:
>>> Apache-HttpAsyncClient/4.0.1 (java 1.5)
>>> headers | | http-outgoing-47 >> Cookie:
>>> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>>> headers | | http-outgoing-47 >> Cookie2:
>>> $Version=1
>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound
>>> done][][outbound done][][0][0][0][0]: Event set [w]
>>> MainClientExec | | [exchange: 62] Request completed
>>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound
>>> done][][outbound done][][0][0][0][0]: 0 bytes written
>>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
>>> Disconnected
>>>
>>>
>>> Nothing after this.
>>>
>>>
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
--
BEKK Open
http://open.bekk.no
TesTcl - a unit test framework for iRules
http://testcl.com
Re: No Response to TCP RESET (???)
Posted by Mark Johnson <mj...@gmail.com>.
What happens at this point is we have tried to issue an HTTP GET (but
nothing goes out on the wire) and there's no timeout or exception
thrown. So my client application hangs.
On 5/10/2015 6:51 AM, Oleg Kalnichevski wrote:
> On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
>> Hi,
>>
>> I'm having intermittent hangs with a client application that uses the
>> async HTTPClient. (Its an HTTPS connection also.) I've turned on the
>> diagnostics logs (attached) and performed a wire capture. My guess is
>> that the server is not responding to a TCP RESET. One other variable -
>> I'm going through a VPN connection to get to the server.
>>
>> Below is the diagnostics of a typical sequence when the failure occurs
>> (just the last sequence). There’s a small attachment of a Wireshark
>> capture. I see several cases where a TCP RST occurs in the full
>> packet capture and it seems to recover until the hang. After the last
>> TPC RESET, nothing else happens. I do have timeout settings for the
>> response and connection.
>>
> What else do you expect to happen? As far as I can tell the client
> closed the connection in response to server closing the connection on
> its end.
>
> Oleg
>
>> So I'm thinking this is a problem on the server (not responding to
>> the TCP RST). Is there any way to verify this? The version of
>> HTTPClient is 4.3.2.
>>
>> Thanks in advance,
>> Mark
>>
>>
>>
>>
>> MainClientExec | | [exchange: 62] start execution
>> RequestAddCookies | | CookieSpec selected: best-match
>> RequestAddCookies | | Cookie [version: 0][name:
>> vcloud-token][value:
>> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
>> 192.168.70.25][path: /][expiry: null] match
>> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea]
>> RequestAuthCache | | Auth cache not set in the
>> context
>> InternalHttpAsyncClient | | [exchange: 62] Request
>> connection for {s}->https://192.168.70.25:443
>> lingNHttpClientConnectionManager | | Connection request: [route:
>> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
>> 1 of 2; total allocated: 1 of 20]
>> lingNHttpClientConnectionManager | | Connection leased: [id:
>> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
>> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
>> InternalHttpAsyncClient | | [exchange: 62] Connection
>> allocated: http-outgoing-47 [ACTIVE]
>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set attribute http.nio.exchange-handler
>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
>> Request ready
>> MainClientExec | | Connection route already
>> established
>> MainClientExec | | [exchange: 62] Attempt 1 to
>> execute request
>> MainClientExec | | Target auth state: UNCHALLENGED
>> MainClientExec | | Proxy auth state: UNCHALLENGED
>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set timeout 50000
>> headers | | http-outgoing-47 >>
>> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
>> headers | | http-outgoing-47 >> Accept:
>> application/*+xml;version=5.1
>> headers | | http-outgoing-47 >> Connection:
>> close
>> headers | | http-outgoing-47 >>
>> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>> headers | | http-outgoing-47 >> Host:
>> 192.168.70.25
>> headers | | http-outgoing-47 >> User-Agent:
>> Apache-HttpAsyncClient/4.0.1 (java 1.5)
>> headers | | http-outgoing-47 >> Cookie:
>> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
>> headers | | http-outgoing-47 >> Cookie2:
>> $Version=1
>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
>> MainClientExec | | [exchange: 62] Request completed
>> ManagedNHttpClientConnectionImpl | | http-outgoing-47
>> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: 0 bytes written
>> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
>> Disconnected
>>
>>
>> Nothing after this.
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: No Response to TCP RESET (???)
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Fri, 2015-05-08 at 09:28 -0500, Mark Johnson wrote:
> Hi,
>
> I'm having intermittent hangs with a client application that uses the
> async HTTPClient. (Its an HTTPS connection also.) I've turned on the
> diagnostics logs (attached) and performed a wire capture. My guess is
> that the server is not responding to a TCP RESET. One other variable -
> I'm going through a VPN connection to get to the server.
>
> Below is the diagnostics of a typical sequence when the failure occurs
> (just the last sequence). There’s a small attachment of a Wireshark
> capture. I see several cases where a TCP RST occurs in the full
> packet capture and it seems to recover until the hang. After the last
> TPC RESET, nothing else happens. I do have timeout settings for the
> response and connection.
>
What else do you expect to happen? As far as I can tell the client
closed the connection in response to server closing the connection on
its end.
Oleg
> So I'm thinking this is a problem on the server (not responding to
> the TCP RST). Is there any way to verify this? The version of
> HTTPClient is 4.3.2.
>
> Thanks in advance,
> Mark
>
>
>
>
> MainClientExec | | [exchange: 62] start execution
> RequestAddCookies | | CookieSpec selected: best-match
> RequestAddCookies | | Cookie [version: 0][name:
> vcloud-token][value:
> Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=][domain:
> 192.168.70.25][path: /][expiry: null] match
> [(secure)192.168.70.25:443/api/task/b6d66a53-8d67-459f-871b-f024021606ea]
> RequestAuthCache | | Auth cache not set in the
> context
> InternalHttpAsyncClient | | [exchange: 62] Request
> connection for {s}->https://192.168.70.25:443
> lingNHttpClientConnectionManager | | Connection request: [route:
> {s}->https://192.168.70.25:443][total kept alive: 1; route allocated:
> 1 of 2; total allocated: 1 of 20]
> lingNHttpClientConnectionManager | | Connection leased: [id:
> http-outgoing-47][route: {s}->https://192.168.70.25:443][total kept
> alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
> InternalHttpAsyncClient | | [exchange: 62] Connection
> allocated: http-outgoing-47 [ACTIVE]
> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set attribute http.nio.exchange-handler
> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]
> Request ready
> MainClientExec | | Connection route already
> established
> MainClientExec | | [exchange: 62] Attempt 1 to
> execute request
> MainClientExec | | Target auth state: UNCHALLENGED
> MainClientExec | | Proxy auth state: UNCHALLENGED
> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Set timeout 50000
> headers | | http-outgoing-47 >>
> GET /api/task/b6d66a53-8d67-459f-871b-f024021606ea HTTP/1.1
> headers | | http-outgoing-47 >> Accept:
> application/*+xml;version=5.1
> headers | | http-outgoing-47 >> Connection:
> close
> headers | | http-outgoing-47 >>
> x-vcloud-authorization: Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
> headers | | http-outgoing-47 >> Host:
> 192.168.70.25
> headers | | http-outgoing-47 >> User-Agent:
> Apache-HttpAsyncClient/4.0.1 (java 1.5)
> headers | | http-outgoing-47 >> Cookie:
> vcloud-token=Trsj1Hs4btsBVK5jfY1xhf07tNadC0zDS8jkVv6pHPw=
> headers | | http-outgoing-47 >> Cookie2:
> $Version=1
> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: Event set [w]
> MainClientExec | | [exchange: 62] Request completed
> ManagedNHttpClientConnectionImpl | | http-outgoing-47
> 192.168.14.105:6028<->192.168.70.25:443[ACTIVE][rw:w][CLOSED][rw][NOT_HANDSHAKING][inbound done][][outbound done][][0][0][0][0]: 0 bytes written
> HttpAsyncRequestExecutor | | http-outgoing-47 [ACTIVE]:
> Disconnected
>
>
> Nothing after this.
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org