You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2020/10/16 20:33:19 UTC
[GitHub] [helix] pkuwm opened a new pull request #1472: Fix connection leakage in CustomRestClient
pkuwm opened a new pull request #1472:
URL: https://github.com/apache/helix/pull/1472
### Issues
- [ ] My PR addresses the following Helix issues and references them in the PR description:
Fixes #1471
### Description
- [ ] Here are some details about my PR, including screenshots of any UI changes:
There is connection leakage in `CustomRestClientImpl` and causes timeout waiting for connection.
Fix this issue by consuming entity and releasing stream and connection.
### Tests
- [ ] The following tests are written for this issue:
Manually reproduce the issue locally. After the fix, the issue is gone.
- [ ] The following is the result of the "mvn test" command on the appropriate module:
(Before CI test pass, please copy & paste the result of "mvn test")
### Documentation (Optional)
- In case of new functionality, my PR adds documentation in the following wiki page:
(Link the GitHub wiki you added)
### Commits
- My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Code Quality
- My diff has been formatted using helix-style.xml
(helix-style-intellij.xml if IntelliJ IDE is used)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #1472: Fix connection leakage in CustomRestClient
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #1472:
URL: https://github.com/apache/helix/pull/1472#discussion_r506746581
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/client/CustomRestClientImpl.java
##########
@@ -116,25 +116,36 @@ protected JsonNode getJsonObject(HttpResponse httpResponse) throws IOException {
LOG.info("Expected HttpResponse statusCode: {}", HttpStatus.SC_OK);
return jsonConverter.convert(getJsonObject(httpResponse));
} else {
+ // Ensure entity is fully consumed so stream is closed.
+ EntityUtils.consumeQuietly(httpResponse.getEntity());
throw new ClientProtocolException("Unexpected response status: " + status + ", reason: "
+ httpResponse.getStatusLine().getReasonPhrase());
}
}
@VisibleForTesting
protected HttpResponse post(String url, Map<String, String> payloads) throws IOException {
+ HttpPost postRequest = new HttpPost(url);
try {
- HttpPost postRequest = new HttpPost(url);
postRequest.setHeader("Accept", ACCEPT_CONTENT_TYPE);
StringEntity entity = new StringEntity(OBJECT_MAPPER.writeValueAsString(payloads),
ContentType.APPLICATION_JSON);
postRequest.setEntity(entity);
LOG.info("Executing request: {}, headers: {}, entity: {}", postRequest.getRequestLine(),
postRequest.getAllHeaders(), postRequest.getEntity());
- return _httpClient.execute(postRequest);
+
+ HttpResponse response = _httpClient.execute(postRequest);
Review comment:
I tried that. If we move to `handleResponse`, we'll have to change more code, eg. method signature of `handleResponse` to have an extra parameter `payloads`, and we need to change the callers as well. That sounds more work to me and it doesn't look better than this. I prefer this way so it is cleaner.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] kaisun2000 commented on a change in pull request #1472: Fix connection leakage in CustomRestClient
Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #1472:
URL: https://github.com/apache/helix/pull/1472#discussion_r506742290
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/client/CustomRestClientImpl.java
##########
@@ -40,7 +40,7 @@
import org.slf4j.LoggerFactory;
class CustomRestClientImpl implements CustomRestClient {
- private static final Logger LOG = LoggerFactory.getLogger(CustomRestClient.class);
Review comment:
Good catch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on pull request #1472: Fix connection leakage in CustomRestClient
Posted by GitBox <gi...@apache.org>.
pkuwm commented on pull request #1472:
URL: https://github.com/apache/helix/pull/1472#issuecomment-710682396
This PR is ready to be merged, approved by @kaisun2000
There is connection leakage in CustomRestClientImpl and causes timeout waiting for connection.
Fix this issue by consuming entity and releasing the content stream and connection.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] kaisun2000 commented on pull request #1472: Fix connection leakage in CustomRestClient
Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on pull request #1472:
URL: https://github.com/apache/helix/pull/1472#issuecomment-710679055
@pkuwm, yes, this should fix the issue I found which can be urgent. So without a unit test, it is fine for now.
Can you also add an issue to add a unit test?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm merged pull request #1472: Fix connection leakage in CustomRestClient
Posted by GitBox <gi...@apache.org>.
pkuwm merged pull request #1472:
URL: https://github.com/apache/helix/pull/1472
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] kaisun2000 commented on a change in pull request #1472: Fix connection leakage in CustomRestClient
Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #1472:
URL: https://github.com/apache/helix/pull/1472#discussion_r506745498
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/client/CustomRestClientImpl.java
##########
@@ -116,25 +116,36 @@ protected JsonNode getJsonObject(HttpResponse httpResponse) throws IOException {
LOG.info("Expected HttpResponse statusCode: {}", HttpStatus.SC_OK);
return jsonConverter.convert(getJsonObject(httpResponse));
} else {
+ // Ensure entity is fully consumed so stream is closed.
+ EntityUtils.consumeQuietly(httpResponse.getEntity());
throw new ClientProtocolException("Unexpected response status: " + status + ", reason: "
+ httpResponse.getStatusLine().getReasonPhrase());
}
}
@VisibleForTesting
protected HttpResponse post(String url, Map<String, String> payloads) throws IOException {
+ HttpPost postRequest = new HttpPost(url);
try {
- HttpPost postRequest = new HttpPost(url);
postRequest.setHeader("Accept", ACCEPT_CONTENT_TYPE);
StringEntity entity = new StringEntity(OBJECT_MAPPER.writeValueAsString(payloads),
ContentType.APPLICATION_JSON);
postRequest.setEntity(entity);
LOG.info("Executing request: {}, headers: {}, entity: {}", postRequest.getRequestLine(),
postRequest.getAllHeaders(), postRequest.getEntity());
- return _httpClient.execute(postRequest);
+
+ HttpResponse response = _httpClient.execute(postRequest);
Review comment:
handleResponse() handles the result from post() assuming post has not IO exception. That is the original author's thinking. So looking at the response code previously is in handleResponse(). If sticking to the same line of thinking, we can add this warning log to handleResponse, instead in post() here.
But in reality, it does not really matter. More like a style of personal choice.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org