You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Jayanga Dissanayake <js...@gmail.com> on 2014/10/09 13:13:17 UTC
Connection termination issue (sometimes connections are not closed
after receiving [FIN, ACK] from server)
Hi All,
We are using WSO2ESB [1], and we want to communicate with a web server via
HTTPS. While we were trying to communicate with the web service, we
observed some weired behavior in the client side, when we analyzed the
tcpdump, found that, client sometimes try to send application data after
the [FIN, ACK] is received from the server, and server sends a [RST] after
we send application data. (Here, in this case, my end point is a HIS server)
As I know, according to the TCP spec, its allowed to send data via a half
closed connection. But in this case I want to stop sending data,
immediately after detecting the half closed connection. So, tried to put
debug points at endOfInput(), in all the implementation classes found in
synapse.transport and apache.http, and see whether I can detect the
connection close from the server end. The break points were hit at the
times when the connection termination happens properly. But not at the
times when application messages were sent after the [FIN, ACK].
It seems like the connection termination from the remote end is not
detected. Please read the below for more information.
Once we create the connection with the back end service, response header
looks,
[2014-10-09 13:52:09,516] DEBUG - wire >> "HTTP/1.1 200 OK[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "Date: Thu, 09 Oct 2014 08:22:07
GMT[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "X-Powered-By:
Servlet/3.0[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "SOAPAction: ""[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "Accept: text/xml[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "Keep-Alive: timeout=10,
max=100[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "Connection: Keep-Alive[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "Transfer-Encoding:
chunked[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "Content-Type: text/xml[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "Content-Language: en-US[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "[\r][\n]"
[2014-10-09 13:52:09,516] DEBUG - wire >> "20b[\r][\n]"
As indicated in the Keep-Alive field, connection termination initiated from
the server end, after idle time of 10 seconds,
1206 72.626148 [BackEndServerIP] [MyIP] TCP 60 https > 59949 [FIN, ACK]
Seq=5230 Ack=2024 Win=19024 Len=0
As a response it sent an ACK to the server,
1207 72.665189 [MyIP] [BackEndServerIP] TCP 54 59949 > https [ACK] Seq=2024
Ack=5231 Win=24840 Len=0
After that it sends application data to the server,
1940 120.137122 [MyIP] [BackEndServerIP] TLSv1 443 Application Data
1941 120.137419 [MyIP] [BackEndServerIP] TLSv1 1275 Application Data
Then server sends a RST,
1949 120.182599 [BackEndServerIP] [MyIP] TCP 60 https > 59949 [RST]
Seq=5231 Win=0 Len=0
Note: there is a considerable time gap between ACK and first application
data packet (72.665189 – 120.137122)
1. My first questions is, why sometimes the client end sends just a ACK for
the server side [FIN, ACK]. If client can send a [FIN, ACK] and gracefully
terminate the connection, my problem is solved.
2. As we have problem (1), Is there anyway to detect partially closed
connection, so that we can call close() or shutdown() to close the client
end.
Appreciate if someone can answer my questions.
[1] http://wso2.com/products/enterprise-service-bus/
Thanks,
Jayanga.
Re: Connection termination issue (sometimes connections are not
closed after receiving [FIN, ACK] from server)
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Fri, 2014-10-10 at 11:39 +0530, Jayanga Dissanayake wrote:
> Hi Oleg,
>
> Thanks for the reply.
>
> I have another question related to this behavior. Why some times client
> replies to the server with a [FIN,ACK] then close the connection as
> expected and some times just send an [ACK] and try to keep the connection.
> I think, its acceptable to send an [ACK] and try to send application data
> in the same connection, if the those application data is ready by the time,
> when the [FIN,ACK] is received from the server (considering half-closed
> scenario). But if you see the time-stamps of the last [ACK] and application
> data, (72.665189 – 120.137122), you will see there is a considerable time
> gap.
>
> Could you please explain, what would cause this behavior?
>
This is likely happening to due to client trying to terminate SSL
session gracefully and send out SSL handshake data. One can however
force the session to get dropped immediately by calling #shutdown method
instead of #close from the #endOfInput() event.
Oleg
> Thanks
> Jayanga.
>
> On Thu, Oct 9, 2014 at 6:29 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> > On Thu, 2014-10-09 at 16:43 +0530, Jayanga Dissanayake wrote:
> > > Hi All,
> > >
> > > We are using WSO2ESB [1], and we want to communicate with a web server
> > via
> > > HTTPS. While we were trying to communicate with the web service, we
> > > observed some weired behavior in the client side, when we analyzed the
> > > tcpdump, found that, client sometimes try to send application data after
> > > the [FIN, ACK] is received from the server, and server sends a [RST]
> > after
> > > we send application data. (Here, in this case, my end point is a HIS
> > server)
> > >
> > > As I know, according to the TCP spec, its allowed to send data via a half
> > > closed connection. But in this case I want to stop sending data,
> > > immediately after detecting the half closed connection. So, tried to put
> > > debug points at endOfInput(), in all the implementation classes found in
> > > synapse.transport and apache.http, and see whether I can detect the
> > > connection close from the server end. The break points were hit at the
> > > times when the connection termination happens properly. But not at the
> > > times when application messages were sent after the [FIN, ACK].
> > >
> > > It seems like the connection termination from the remote end is not
> > > detected. Please read the below for more information.
> > >
> > > Once we create the connection with the back end service, response header
> > > looks,
> > >
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "HTTP/1.1 200 OK[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "Date: Thu, 09 Oct 2014
> > 08:22:07
> > > GMT[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "X-Powered-By:
> > > Servlet/3.0[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "SOAPAction: ""[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "Accept: text/xml[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "Keep-Alive: timeout=10,
> > > max=100[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "Connection:
> > Keep-Alive[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "Transfer-Encoding:
> > > chunked[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "Content-Type:
> > text/xml[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "Content-Language:
> > en-US[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "[\r][\n]"
> > > [2014-10-09 13:52:09,516] DEBUG - wire >> "20b[\r][\n]"
> > >
> > > As indicated in the Keep-Alive field, connection termination initiated
> > from
> > > the server end, after idle time of 10 seconds,
> > >
> > > 1206 72.626148 [BackEndServerIP] [MyIP] TCP 60 https > 59949 [FIN, ACK]
> > > Seq=5230 Ack=2024 Win=19024 Len=0
> > >
> > > As a response it sent an ACK to the server,
> > >
> > > 1207 72.665189 [MyIP] [BackEndServerIP] TCP 54 59949 > https [ACK]
> > Seq=2024
> > > Ack=5231 Win=24840 Len=0
> > >
> > > After that it sends application data to the server,
> > >
> > > 1940 120.137122 [MyIP] [BackEndServerIP] TLSv1 443 Application Data
> > > 1941 120.137419 [MyIP] [BackEndServerIP] TLSv1 1275 Application Data
> > >
> > > Then server sends a RST,
> > >
> > > 1949 120.182599 [BackEndServerIP] [MyIP] TCP 60 https > 59949 [RST]
> > > Seq=5231 Win=0 Len=0
> > >
> > > Note: there is a considerable time gap between ACK and first application
> > > data packet (72.665189 – 120.137122)
> > >
> > > 1. My first questions is, why sometimes the client end sends just a ACK
> > for
> > > the server side [FIN, ACK]. If client can send a [FIN, ACK] and
> > gracefully
> > > terminate the connection, my problem is solved.
> > > 2. As we have problem (1), Is there anyway to detect partially closed
> > > connection, so that we can call close() or shutdown() to close the client
> > > end.
> > >
> >
> > Jayanga
> >
> > Unfortunately to my knowledge there is simply no way to control TCP
> > signals from Java. TCP signals are received and sent by JRE and the way
> > it is done is entirely out of application control. #endOfInput() should
> > always get fired when the channel returns -1 on a read operation. If it
> > does not, there is really not much one can do.
> >
> > Oleg
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Connection termination issue (sometimes connections are not
closed after receiving [FIN, ACK] from server)
Posted by Jayanga Dissanayake <js...@gmail.com>.
Hi Oleg,
Thanks for the reply.
I have another question related to this behavior. Why some times client
replies to the server with a [FIN,ACK] then close the connection as
expected and some times just send an [ACK] and try to keep the connection.
I think, its acceptable to send an [ACK] and try to send application data
in the same connection, if the those application data is ready by the time,
when the [FIN,ACK] is received from the server (considering half-closed
scenario). But if you see the time-stamps of the last [ACK] and application
data, (72.665189 – 120.137122), you will see there is a considerable time
gap.
Could you please explain, what would cause this behavior?
Thanks
Jayanga.
On Thu, Oct 9, 2014 at 6:29 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Thu, 2014-10-09 at 16:43 +0530, Jayanga Dissanayake wrote:
> > Hi All,
> >
> > We are using WSO2ESB [1], and we want to communicate with a web server
> via
> > HTTPS. While we were trying to communicate with the web service, we
> > observed some weired behavior in the client side, when we analyzed the
> > tcpdump, found that, client sometimes try to send application data after
> > the [FIN, ACK] is received from the server, and server sends a [RST]
> after
> > we send application data. (Here, in this case, my end point is a HIS
> server)
> >
> > As I know, according to the TCP spec, its allowed to send data via a half
> > closed connection. But in this case I want to stop sending data,
> > immediately after detecting the half closed connection. So, tried to put
> > debug points at endOfInput(), in all the implementation classes found in
> > synapse.transport and apache.http, and see whether I can detect the
> > connection close from the server end. The break points were hit at the
> > times when the connection termination happens properly. But not at the
> > times when application messages were sent after the [FIN, ACK].
> >
> > It seems like the connection termination from the remote end is not
> > detected. Please read the below for more information.
> >
> > Once we create the connection with the back end service, response header
> > looks,
> >
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "HTTP/1.1 200 OK[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "Date: Thu, 09 Oct 2014
> 08:22:07
> > GMT[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "X-Powered-By:
> > Servlet/3.0[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "SOAPAction: ""[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "Accept: text/xml[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "Keep-Alive: timeout=10,
> > max=100[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "Connection:
> Keep-Alive[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "Transfer-Encoding:
> > chunked[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "Content-Type:
> text/xml[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "Content-Language:
> en-US[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "[\r][\n]"
> > [2014-10-09 13:52:09,516] DEBUG - wire >> "20b[\r][\n]"
> >
> > As indicated in the Keep-Alive field, connection termination initiated
> from
> > the server end, after idle time of 10 seconds,
> >
> > 1206 72.626148 [BackEndServerIP] [MyIP] TCP 60 https > 59949 [FIN, ACK]
> > Seq=5230 Ack=2024 Win=19024 Len=0
> >
> > As a response it sent an ACK to the server,
> >
> > 1207 72.665189 [MyIP] [BackEndServerIP] TCP 54 59949 > https [ACK]
> Seq=2024
> > Ack=5231 Win=24840 Len=0
> >
> > After that it sends application data to the server,
> >
> > 1940 120.137122 [MyIP] [BackEndServerIP] TLSv1 443 Application Data
> > 1941 120.137419 [MyIP] [BackEndServerIP] TLSv1 1275 Application Data
> >
> > Then server sends a RST,
> >
> > 1949 120.182599 [BackEndServerIP] [MyIP] TCP 60 https > 59949 [RST]
> > Seq=5231 Win=0 Len=0
> >
> > Note: there is a considerable time gap between ACK and first application
> > data packet (72.665189 – 120.137122)
> >
> > 1. My first questions is, why sometimes the client end sends just a ACK
> for
> > the server side [FIN, ACK]. If client can send a [FIN, ACK] and
> gracefully
> > terminate the connection, my problem is solved.
> > 2. As we have problem (1), Is there anyway to detect partially closed
> > connection, so that we can call close() or shutdown() to close the client
> > end.
> >
>
> Jayanga
>
> Unfortunately to my knowledge there is simply no way to control TCP
> signals from Java. TCP signals are received and sent by JRE and the way
> it is done is entirely out of application control. #endOfInput() should
> always get fired when the channel returns -1 on a read operation. If it
> does not, there is really not much one can do.
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
Re: Connection termination issue (sometimes connections are not
closed after receiving [FIN, ACK] from server)
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2014-10-09 at 16:43 +0530, Jayanga Dissanayake wrote:
> Hi All,
>
> We are using WSO2ESB [1], and we want to communicate with a web server via
> HTTPS. While we were trying to communicate with the web service, we
> observed some weired behavior in the client side, when we analyzed the
> tcpdump, found that, client sometimes try to send application data after
> the [FIN, ACK] is received from the server, and server sends a [RST] after
> we send application data. (Here, in this case, my end point is a HIS server)
>
> As I know, according to the TCP spec, its allowed to send data via a half
> closed connection. But in this case I want to stop sending data,
> immediately after detecting the half closed connection. So, tried to put
> debug points at endOfInput(), in all the implementation classes found in
> synapse.transport and apache.http, and see whether I can detect the
> connection close from the server end. The break points were hit at the
> times when the connection termination happens properly. But not at the
> times when application messages were sent after the [FIN, ACK].
>
> It seems like the connection termination from the remote end is not
> detected. Please read the below for more information.
>
> Once we create the connection with the back end service, response header
> looks,
>
> [2014-10-09 13:52:09,516] DEBUG - wire >> "HTTP/1.1 200 OK[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "Date: Thu, 09 Oct 2014 08:22:07
> GMT[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "X-Powered-By:
> Servlet/3.0[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "SOAPAction: ""[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "Accept: text/xml[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "Keep-Alive: timeout=10,
> max=100[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "Connection: Keep-Alive[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "Transfer-Encoding:
> chunked[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "Content-Type: text/xml[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "Content-Language: en-US[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "[\r][\n]"
> [2014-10-09 13:52:09,516] DEBUG - wire >> "20b[\r][\n]"
>
> As indicated in the Keep-Alive field, connection termination initiated from
> the server end, after idle time of 10 seconds,
>
> 1206 72.626148 [BackEndServerIP] [MyIP] TCP 60 https > 59949 [FIN, ACK]
> Seq=5230 Ack=2024 Win=19024 Len=0
>
> As a response it sent an ACK to the server,
>
> 1207 72.665189 [MyIP] [BackEndServerIP] TCP 54 59949 > https [ACK] Seq=2024
> Ack=5231 Win=24840 Len=0
>
> After that it sends application data to the server,
>
> 1940 120.137122 [MyIP] [BackEndServerIP] TLSv1 443 Application Data
> 1941 120.137419 [MyIP] [BackEndServerIP] TLSv1 1275 Application Data
>
> Then server sends a RST,
>
> 1949 120.182599 [BackEndServerIP] [MyIP] TCP 60 https > 59949 [RST]
> Seq=5231 Win=0 Len=0
>
> Note: there is a considerable time gap between ACK and first application
> data packet (72.665189 – 120.137122)
>
> 1. My first questions is, why sometimes the client end sends just a ACK for
> the server side [FIN, ACK]. If client can send a [FIN, ACK] and gracefully
> terminate the connection, my problem is solved.
> 2. As we have problem (1), Is there anyway to detect partially closed
> connection, so that we can call close() or shutdown() to close the client
> end.
>
Jayanga
Unfortunately to my knowledge there is simply no way to control TCP
signals from Java. TCP signals are received and sent by JRE and the way
it is done is entirely out of application control. #endOfInput() should
always get fired when the channel returns -1 on a read operation. If it
does not, there is really not much one can do.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org