You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by agherardi <gi...@git.apache.org> on 2018/01/10 02:28:30 UTC
[GitHub] httpcomponents-core pull request #56: Improve connection reuse logic
GitHub user agherardi opened a pull request:
https://github.com/apache/httpcomponents-core/pull/56
Improve connection reuse logic
This is still a work-in-progress, at this point I'm just soliciting feedback.
The purpose of this change is to improve connection reuse by detecting that the content stream of an HTTP response entity is already at EOF before the stream's read() method returns -1. This is useful when the application does not want to call the close() method of the content stream, since that method can potentially block.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/agherardi/httpcomponents-core eof
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/httpcomponents-core/pull/56.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #56
----
commit cc6f3649ee1962e954b26ef686e92335f1637890
Author: alessandro.gherardi <al...@...>
Date: 2018-01-10T02:14:06Z
First commit
commit dbc8aea63788ff287c0caef28562cadca824ffa6
Author: alessandro.gherardi <al...@...>
Date: 2018-01-10T02:16:16Z
Comment
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[GitHub] httpcomponents-core pull request #56: Improve connection reuse logic
Posted by agherardi <gi...@git.apache.org>.
Github user agherardi closed the pull request at:
https://github.com/apache/httpcomponents-core/pull/56
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[GitHub] httpcomponents-core issue #56: Improve connection reuse logic
Posted by agherardi <gi...@git.apache.org>.
Github user agherardi commented on the issue:
https://github.com/apache/httpcomponents-core/pull/56
After running some tests, I realized that there isn't a bullet-proof way to detect EOF without blocking when the server uses chunked encoding. The solution I came up with only works if the SessionInputBuffer happens to have the chunk header with size=0 already buffered, and that's not always the case.
Therefore, I believe the best option is for the app - and in make case the Jersey Apache connector https://github.com/jersey/jersey/pull/3752 - to close the entity stream before closing the response.
I'm closing this PR. Thank you for your help.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[GitHub] httpcomponents-core issue #56: Improve connection reuse logic
Posted by agherardi <gi...@git.apache.org>.
Github user agherardi commented on the issue:
https://github.com/apache/httpcomponents-core/pull/56
> There are cases when clients might want to consume the payload which has some sort of format delineation and stop reading without getting -1 (end of stream) result that would trigger release of the underlying connection.
Yes, that's exactly the point.
> Please let me know if you want me to merge the changes or you still intent to do more work on it
Now that I know that this is acceptable, please give me some time to do more work and add some unit tests. Thanks!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[GitHub] httpcomponents-core issue #56: Improve connection reuse logic
Posted by ok2c <gi...@git.apache.org>.
Github user ok2c commented on the issue:
https://github.com/apache/httpcomponents-core/pull/56
@agherardi OK. I see. While it is not pretty I understand the rationale for the work-around. There are cases when clients might want to consume the payload which has some sort of format delineation and stop reading without getting -1 (end of stream) result that would trigger release of the underlying connection.
Please let me know if you want me to merge the changes or you still intent to do more work on it
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[GitHub] httpcomponents-core issue #56: Improve connection reuse logic
Posted by agherardi <gi...@git.apache.org>.
Github user agherardi commented on the issue:
https://github.com/apache/httpcomponents-core/pull/56
My HTTP stack consists of Jersey https://github.com/jersey/jersey with the Apache httpclient connector and the JacksonJsonProvider. I'm using the stack to talk to a REST-ful server that returns JSON responses, and have Jersey + JacksonJsonProvider automatically deserialize JSON responses to Java POJOs. The approach is similar to the one described here https://dennis-xlc.gitbooks.io/restful-java-with-jax-rs-2-0-en/cn/part1/chapter8/building_and_invoking_requests.html
While testing, I noticed that HTTP and HTTPS connections are not reused - i.e., the stack opens and closes the HTTP/HTTPS connection for every REST call. The underlying problem is that the Jersey httpclient connector closes the CloseableHttpResponse BEFORE closing the response's content input stream.
I can think of a couple of ways to solve this issue. One is to reverse the order in which the Jersey connector closes things - i,e,, first close the input stream, then close the CloseableHttpResponse. I have a merge request out to the Jersey maintainers https://github.com/jersey/jersey/pull/3752 for that.
The other possibility is to modify httpclient along the lines described in this merge request. The idea is to make EofSensorInputStream#checkEOF more proactive in detecting when a response has been consumed and release the connection to the pool without waiting for a close().
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[GitHub] httpcomponents-core issue #56: Improve connection reuse logic
Posted by ok2c <gi...@git.apache.org>.
Github user ok2c commented on the issue:
https://github.com/apache/httpcomponents-core/pull/56
@agherardi Could you please help me better understand what you are trying to achieve? Why would you want to avoid calling `#close` method when it is a part of `InputStream` contract? I also do not quite understand why blocking in `#close` is a problem given that `InputStream` is inherently blocking.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org