You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Ephemeris Lappis <ep...@gmail.com> on 2005/12/14 09:22:39 UTC
[HTTP Client][2.0.2] Incorrect response reading on connections closed by server
Hello.
This issue has already bean discussed in the past, but I have not found any
actual solution to solve it. So, if someone has a new idea, it's welcome!
The problem occurs with pooled connections, using a
MultiThreadedHttpConnectionManager. Connections are created according to the
client requests load, and kept in pool. After a given delay, about 15 or 20
seconds on Tomcat or Apache, the server dismisses old connections. At the
other end, the http-client doesn't seem to detect the unilateral server's
closing action, and send the new request: I don't understand how this can
work ! Then, as from the server's point of view the connection is closed,
nothing comes back, and the http-client fails reading the response. It
throws a HttpRecoverableException, with the well known specific message:
Error in parsing the status line from the response: unable to find line
starting with "HTTP". Activating the stale connections checking fixes the
problem, but induces an unacceptable performance down: requests that are
processed in less than 1 ms take over 20-30 ms with checking. Another
possible solution is not managing connections at all, using HTTP 1.0 and no
connection manager, but this also leads to slower processing and, above all,
excessive TCP/IP system resources waste.
Note this occurs with JDK 1.3, 1.4 or 5.0.
Is there any way to make the http-client handle such server's close, and
avoid using a doubtful socket?
I think an "acceptable" solution should be to limit the reuse of pooled
connections giving them a maximum life-time (from the creation time) or
inactivity delay (since last request): when the manager gets a connection
from the pool, it should check the connection age, and discard it if too
old.
I've had a look to the library's sources to evaluate such a solution, but
its design disallow extension since all the connection pool and adapter
classes, and instantiation and selection policies are all private inside the
manager code.
Your comment and ideas are all welcome !
Thanks.
--
Ephemeris Lappis
Re: [HTTP Client][2.0.2] Incorrect response reading on connections closed by server
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, Dec 14, 2005 at 12:01:05PM +0100, Ephemeris Lappis wrote:
> Hello.
>
> Thanks for the early answer!
>
> Ok for the new 3.0. I'll have a look to it, but I'm afraid we can't migrate
> to the new version immediately for our current project.
>
> I'm a bit amazed by the current behaviour that lets the http-client send a
> request over a half-closed socket, and logically fail reading the response.
> If actually no way exists to check the socket's TCP state, a feature request
> should be addressed to the Java specification responsible!...
>
Ephemeris,
It is a small comfort to me if this feature is made available sometime
in Java 7. HttpClient 3.x series of releases must be Java 1.2 compatible
and HttpClient 4.0 is planned to require Java 1.4 only
> I have had a look at the current implementation of the stale checking, and
> I'm also surprised by the poor performance it induces...
>
The trouble is the socket timeout granularity on some platforms/JREs is
approximately 15-20ms. There's not much we can do about it
Hope this explains it
Oleg
> --
> Ephemeris Lappis
>
> -----Message d'origine-----
> De?: Oleg Kalnichevski [mailto:olegk@apache.org]
> Envoy??: mercredi 14 d?cembre 2005 10:26
> ??: HttpClient User Discussion
> Objet?: Re: [HTTP Client][2.0.2] Incorrect response reading on connections
> closed by server
>
> On Wed, Dec 14, 2005 at 09:22:39AM +0100, Ephemeris Lappis wrote:
> > Hello.
> >
> >
> >
> > This issue has already bean discussed in the past, but I have not found
> any
> > actual solution to solve it. So, if someone has a new idea, it's welcome!
> >
> >
> >
> > The problem occurs with pooled connections, using a
> > MultiThreadedHttpConnectionManager. Connections are created according to
> the
> > client requests load, and kept in pool. After a given delay, about 15 or
> 20
> > seconds on Tomcat or Apache, the server dismisses old connections. At the
> > other end, the http-client doesn't seem to detect the unilateral server's
> > closing action, and send the new request: I don't understand how this can
> > work !
>
> The trouble is there is no way (I personally know of) to detect this
> event in Java. The only reliable to way to test if a socket is still
> valid is to perform a read operation on it. This is essentially what the
> stale connection check is all about
>
>
> Then, as from the server's point of view the connection is closed,
> > nothing comes back, and the http-client fails reading the response. It
> > throws a HttpRecoverableException, with the well known specific message:
> > Error in parsing the status line from the response: unable to find line
> > starting with "HTTP". Activating the stale connections checking fixes the
> > problem, but induces an unacceptable performance down: requests that are
> > processed in less than 1 ms take over 20-30 ms with checking. Another
> > possible solution is not managing connections at all, using HTTP 1.0 and
> no
> > connection manager, but this also leads to slower processing and, above
> all,
> > excessive TCP/IP system resources waste.
> >
> >
> >
> > Note this occurs with JDK 1.3, 1.4 or 5.0.
> >
> >
> >
> > Is there any way to make the http-client handle such server's close, and
> > avoid using a doubtful socket?
> >
> >
> >
> > I think an "acceptable" solution should be to limit the reuse of pooled
> > connections giving them a maximum life-time (from the creation time) or
> > inactivity delay (since last request): when the manager gets a connection
> > from the pool, it should check the connection age, and discard it if too
> > old.
> >
> >
>
> This feature has been implemented in HttpClient 3.0. Consider upgrading.
>
>
> >
> > I've had a look to the library's sources to evaluate such a solution, but
> > its design disallow extension since all the connection pool and adapter
> > classes, and instantiation and selection policies are all private inside
> the
> > manager code.
> >
> >
>
> I assume you have been looking at the HttpClient 2.0.2 source code?
>
> Oleg
>
> >
> > Your comment and ideas are all welcome !
> >
> >
> >
> > Thanks.
> >
> >
> >
> > --
> >
> > Ephemeris Lappis
> >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
RE: [HTTP Client][2.0.2] Incorrect response reading on connections closed by server
Posted by Ephemeris Lappis <ep...@gmail.com>.
Hello.
Thanks for the early answer!
Ok for the new 3.0. I'll have a look to it, but I'm afraid we can't migrate
to the new version immediately for our current project.
I'm a bit amazed by the current behaviour that lets the http-client send a
request over a half-closed socket, and logically fail reading the response.
If actually no way exists to check the socket's TCP state, a feature request
should be addressed to the Java specification responsible!...
I have had a look at the current implementation of the stale checking, and
I'm also surprised by the poor performance it induces...
--
Ephemeris Lappis
-----Message d'origine-----
De : Oleg Kalnichevski [mailto:olegk@apache.org]
Envoyé : mercredi 14 décembre 2005 10:26
À : HttpClient User Discussion
Objet : Re: [HTTP Client][2.0.2] Incorrect response reading on connections
closed by server
On Wed, Dec 14, 2005 at 09:22:39AM +0100, Ephemeris Lappis wrote:
> Hello.
>
>
>
> This issue has already bean discussed in the past, but I have not found
any
> actual solution to solve it. So, if someone has a new idea, it's welcome!
>
>
>
> The problem occurs with pooled connections, using a
> MultiThreadedHttpConnectionManager. Connections are created according to
the
> client requests load, and kept in pool. After a given delay, about 15 or
20
> seconds on Tomcat or Apache, the server dismisses old connections. At the
> other end, the http-client doesn't seem to detect the unilateral server's
> closing action, and send the new request: I don't understand how this can
> work !
The trouble is there is no way (I personally know of) to detect this
event in Java. The only reliable to way to test if a socket is still
valid is to perform a read operation on it. This is essentially what the
stale connection check is all about
Then, as from the server's point of view the connection is closed,
> nothing comes back, and the http-client fails reading the response. It
> throws a HttpRecoverableException, with the well known specific message:
> Error in parsing the status line from the response: unable to find line
> starting with "HTTP". Activating the stale connections checking fixes the
> problem, but induces an unacceptable performance down: requests that are
> processed in less than 1 ms take over 20-30 ms with checking. Another
> possible solution is not managing connections at all, using HTTP 1.0 and
no
> connection manager, but this also leads to slower processing and, above
all,
> excessive TCP/IP system resources waste.
>
>
>
> Note this occurs with JDK 1.3, 1.4 or 5.0.
>
>
>
> Is there any way to make the http-client handle such server's close, and
> avoid using a doubtful socket?
>
>
>
> I think an "acceptable" solution should be to limit the reuse of pooled
> connections giving them a maximum life-time (from the creation time) or
> inactivity delay (since last request): when the manager gets a connection
> from the pool, it should check the connection age, and discard it if too
> old.
>
>
This feature has been implemented in HttpClient 3.0. Consider upgrading.
>
> I've had a look to the library's sources to evaluate such a solution, but
> its design disallow extension since all the connection pool and adapter
> classes, and instantiation and selection policies are all private inside
the
> manager code.
>
>
I assume you have been looking at the HttpClient 2.0.2 source code?
Oleg
>
> Your comment and ideas are all welcome !
>
>
>
> Thanks.
>
>
>
> --
>
> Ephemeris Lappis
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
Re: [HTTP Client][2.0.2] Incorrect response reading on connections closed by server
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, Dec 14, 2005 at 09:22:39AM +0100, Ephemeris Lappis wrote:
> Hello.
>
>
>
> This issue has already bean discussed in the past, but I have not found any
> actual solution to solve it. So, if someone has a new idea, it's welcome!
>
>
>
> The problem occurs with pooled connections, using a
> MultiThreadedHttpConnectionManager. Connections are created according to the
> client requests load, and kept in pool. After a given delay, about 15 or 20
> seconds on Tomcat or Apache, the server dismisses old connections. At the
> other end, the http-client doesn't seem to detect the unilateral server's
> closing action, and send the new request: I don't understand how this can
> work !
The trouble is there is no way (I personally know of) to detect this
event in Java. The only reliable to way to test if a socket is still
valid is to perform a read operation on it. This is essentially what the
stale connection check is all about
Then, as from the server's point of view the connection is closed,
> nothing comes back, and the http-client fails reading the response. It
> throws a HttpRecoverableException, with the well known specific message:
> Error in parsing the status line from the response: unable to find line
> starting with "HTTP". Activating the stale connections checking fixes the
> problem, but induces an unacceptable performance down: requests that are
> processed in less than 1 ms take over 20-30 ms with checking. Another
> possible solution is not managing connections at all, using HTTP 1.0 and no
> connection manager, but this also leads to slower processing and, above all,
> excessive TCP/IP system resources waste.
>
>
>
> Note this occurs with JDK 1.3, 1.4 or 5.0.
>
>
>
> Is there any way to make the http-client handle such server's close, and
> avoid using a doubtful socket?
>
>
>
> I think an "acceptable" solution should be to limit the reuse of pooled
> connections giving them a maximum life-time (from the creation time) or
> inactivity delay (since last request): when the manager gets a connection
> from the pool, it should check the connection age, and discard it if too
> old.
>
>
This feature has been implemented in HttpClient 3.0. Consider upgrading.
>
> I've had a look to the library's sources to evaluate such a solution, but
> its design disallow extension since all the connection pool and adapter
> classes, and instantiation and selection policies are all private inside the
> manager code.
>
>
I assume you have been looking at the HttpClient 2.0.2 source code?
Oleg
>
> Your comment and ideas are all welcome !
>
>
>
> Thanks.
>
>
>
> --
>
> Ephemeris Lappis
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org