You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/03/09 19:46:48 UTC

[GitHub] [pinot] walterddr opened a new pull request #8329: using HttpClient

walterddr opened a new pull request #8329:
URL: https://github.com/apache/pinot/pull/8329


   alternative solution to #8293 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: using HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (035e67f) into [master](https://codecov.io/gh/apache/pinot/commit/8ef4dbacfb42596590d08c7c6f14f88093dfe7d5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8ef4dba) will **decrease** coverage by `9.71%`.
   > The diff coverage is `41.47%`.
   
   > :exclamation: Current head 035e67f differs from pull request most recent head 001e1b9. Consider uploading reports for the commit 001e1b9 to get more accurate results
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8329/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8329      +/-   ##
   ============================================
   - Coverage     70.80%   61.08%   -9.72%     
   + Complexity     4258     4174      -84     
   ============================================
     Files          1636     1626      -10     
     Lines         85804    85532     -272     
     Branches      12920    12887      -33     
   ============================================
   - Hits          60755    52250    -8505     
   - Misses        20858    29444    +8586     
   + Partials       4191     3838     -353     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `27.57% <36.86%> (+0.05%)` | :arrow_up: |
   | unittests1 | `66.87% <23.29%> (-0.08%)` | :arrow_down: |
   | unittests2 | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...inot/controller/helix/ControllerRequestClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9Db250cm9sbGVyUmVxdWVzdENsaWVudC5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `0.00% <0.00%> (-74.61%)` | :arrow_down: |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `38.33% <ø> (-38.34%)` | :arrow_down: |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `13.48% <0.00%> (ø)` | |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `45.45% <45.45%> (ø)` | |
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `56.93% <60.00%> (-6.08%)` | :arrow_down: |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `76.40% <60.00%> (-1.25%)` | :arrow_down: |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `72.41% <100.00%> (ø)` | |
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `48.57% <100.00%> (-2.86%)` | :arrow_down: |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `69.16% <100.00%> (-20.84%)` | :arrow_down: |
   | ... and [299 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8ef4dba...001e1b9](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] yupeng9 commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r828188238



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,371 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendGetRequest(URI uri)

Review comment:
       why do we add deprecated method, given this is a new file

##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/ControllerRequestClient.java
##########
@@ -0,0 +1,117 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.controller.helix;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.net.URL;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.http.HttpClient;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.tenant.Tenant;
+import org.apache.pinot.spi.config.tenant.TenantRole;
+import org.apache.pinot.spi.data.Schema;
+
+
+public class ControllerRequestClient {

Review comment:
       need docs
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (58eaef1) into [master](https://codecov.io/gh/apache/pinot/commit/98d781092136eeb3a80e9e54fc8736f4c3d270f5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (98d7810) will **increase** coverage by `2.85%`.
   > The diff coverage is `17.88%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8329      +/-   ##
   ============================================
   + Coverage     64.08%   66.94%   +2.85%     
   + Complexity     4254     4186      -68     
   ============================================
     Files          1592     1238     -354     
     Lines         83916    62624   -21292     
     Branches      12706     9739    -2967     
   ============================================
   - Hits          53779    41923   -11856     
   + Misses        26261    17718    -8543     
   + Partials       3876     2983     -893     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests1 | `66.94% <17.88%> (-0.02%)` | :arrow_down: |
   | unittests2 | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `6.66% <0.00%> (ø)` | |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `13.48% <0.00%> (ø)` | |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `14.46% <14.46%> (ø)` | |
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `18.07% <22.85%> (+1.96%)` | :arrow_up: |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `4.21% <36.36%> (+4.21%)` | :arrow_up: |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `72.41% <100.00%> (ø)` | |
   | [.../aggregation/function/ModeAggregationFunction.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9hZ2dyZWdhdGlvbi9mdW5jdGlvbi9Nb2RlQWdncmVnYXRpb25GdW5jdGlvbi5qYXZh) | `87.56% <0.00%> (-1.09%)` | :arrow_down: |
   | [...ot/segment/spi/creator/SegmentGeneratorConfig.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1zcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3NlZ21lbnQvc3BpL2NyZWF0b3IvU2VnbWVudEdlbmVyYXRvckNvbmZpZy5qYXZh) | `82.08% <0.00%> (-0.98%)` | :arrow_down: |
   | [...he/pinot/segment/local/utils/TableConfigUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9UYWJsZUNvbmZpZ1V0aWxzLmphdmE=) | `65.34% <0.00%> (-0.80%)` | :arrow_down: |
   | [...g/apache/pinot/common/metrics/ControllerGauge.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Db250cm9sbGVyR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | ... and [363 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [98d7810...58eaef1](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: using HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e288a0b) into [master](https://codecov.io/gh/apache/pinot/commit/8ef4dbacfb42596590d08c7c6f14f88093dfe7d5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8ef4dba) will **decrease** coverage by `1.26%`.
   > The diff coverage is `41.70%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8329/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8329      +/-   ##
   ============================================
   - Coverage     70.80%   69.53%   -1.27%     
   + Complexity     4258     4255       -3     
   ============================================
     Files          1636     1638       +2     
     Lines         85804    85899      +95     
     Branches      12920    12927       +7     
   ============================================
   - Hits          60755    59734    -1021     
   - Misses        20858    22000    +1142     
   + Partials       4191     4165      -26     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `27.65% <34.78%> (+0.14%)` | :arrow_up: |
   | unittests1 | `66.85% <22.04%> (-0.10%)` | :arrow_down: |
   | unittests2 | `14.17% <3.40%> (-0.01%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...not/ingestion/common/DefaultControllerRestApi.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1iYXRjaC1pbmdlc3Rpb24vdjBfZGVwcmVjYXRlZC9waW5vdC1pbmdlc3Rpb24tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9pbmdlc3Rpb24vY29tbW9uL0RlZmF1bHRDb250cm9sbGVyUmVzdEFwaS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.17% <0.00%> (-71.43%)` | :arrow_down: |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `38.33% <ø> (-38.34%)` | :arrow_down: |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `13.48% <0.00%> (ø)` | |
   | [...inot/controller/helix/ControllerRequestClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9Db250cm9sbGVyUmVxdWVzdENsaWVudC5qYXZh) | `19.04% <19.04%> (ø)` | |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `41.66% <41.66%> (ø)` | |
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `56.93% <60.00%> (-6.08%)` | :arrow_down: |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `76.40% <60.00%> (-1.25%)` | :arrow_down: |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `72.41% <100.00%> (ø)` | |
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `48.57% <100.00%> (-2.86%)` | :arrow_down: |
   | ... and [118 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8ef4dba...e288a0b](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: using HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e288a0b) into [master](https://codecov.io/gh/apache/pinot/commit/8ef4dbacfb42596590d08c7c6f14f88093dfe7d5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8ef4dba) will **decrease** coverage by `56.62%`.
   > The diff coverage is `3.40%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8329/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #8329       +/-   ##
   =============================================
   - Coverage     70.80%   14.17%   -56.63%     
   + Complexity     4258       81     -4177     
   =============================================
     Files          1636     1593       -43     
     Lines         85804    84015     -1789     
     Branches      12920    12723      -197     
   =============================================
   - Hits          60755    11912    -48843     
   - Misses        20858    71207    +50349     
   + Partials       4191      896     -3295     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `?` | |
   | unittests2 | `14.17% <3.40%> (-0.01%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `0.00% <0.00%> (-63.01%)` | :arrow_down: |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `0.00% <0.00%> (-77.65%)` | :arrow_down: |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `0.00% <0.00%> (-72.42%)` | :arrow_down: |
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `0.00% <0.00%> (-51.43%)` | :arrow_down: |
   | [...not/ingestion/common/DefaultControllerRestApi.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1iYXRjaC1pbmdlc3Rpb24vdjBfZGVwcmVjYXRlZC9waW5vdC1pbmdlc3Rpb24tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9pbmdlc3Rpb24vY29tbW9uL0RlZmF1bHRDb250cm9sbGVyUmVzdEFwaS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.33% <0.00%> (-86.67%)` | :arrow_down: |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.17% <0.00%> (-71.43%)` | :arrow_down: |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `0.00% <ø> (-76.67%)` | :arrow_down: |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `0.00% <0.00%> (-13.49%)` | :arrow_down: |
   | ... and [1313 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8ef4dba...e288a0b](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: using HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6f2d43d) into [master](https://codecov.io/gh/apache/pinot/commit/8ef4dbacfb42596590d08c7c6f14f88093dfe7d5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8ef4dba) will **decrease** coverage by `6.78%`.
   > The diff coverage is `26.79%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8329/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8329      +/-   ##
   ============================================
   - Coverage     70.80%   64.02%   -6.79%     
   + Complexity     4258     4253       -5     
   ============================================
     Files          1636     1592      -44     
     Lines         85804    83933    -1871     
     Branches      12920    12715     -205     
   ============================================
   - Hits          60755    53737    -7018     
   - Misses        20858    26324    +5466     
   + Partials       4191     3872     -319     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `66.90% <28.08%> (-0.05%)` | :arrow_down: |
   | unittests2 | `14.18% <0.00%> (+<0.01%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `6.66% <0.00%> (-44.77%)` | :arrow_down: |
   | [...not/ingestion/common/DefaultControllerRestApi.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1iYXRjaC1pbmdlc3Rpb24vdjBfZGVwcmVjYXRlZC9waW5vdC1pbmdlc3Rpb24tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9pbmdlc3Rpb24vY29tbW9uL0RlZmF1bHRDb250cm9sbGVyUmVzdEFwaS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.33% <0.00%> (-86.67%)` | :arrow_down: |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.17% <0.00%> (-71.43%)` | :arrow_down: |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `0.00% <ø> (-76.67%)` | :arrow_down: |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `13.48% <0.00%> (ø)` | |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `1.12% <20.00%> (-76.53%)` | :arrow_down: |
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `14.85% <27.27%> (-48.16%)` | :arrow_down: |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `30.00% <30.00%> (ø)` | |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `72.41% <100.00%> (ø)` | |
   | ... and [386 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8ef4dba...6f2d43d](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang merged pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang merged pull request #8329:
URL: https://github.com/apache/pinot/pull/8329


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] walterddr commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829633997



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       yeah. this is what I meant previously (less the `OVERRIDE` part which brilliantly solve the tls keystore init issue). thanks @richardstartin !




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0e789c4) into [master](https://codecov.io/gh/apache/pinot/commit/98d781092136eeb3a80e9e54fc8736f4c3d270f5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (98d7810) will **decrease** coverage by `33.35%`.
   > The diff coverage is `38.86%`.
   
   > :exclamation: Current head 0e789c4 differs from pull request most recent head 2b865bf. Consider uploading reports for the commit 2b865bf to get more accurate results
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #8329       +/-   ##
   =============================================
   - Coverage     64.08%   30.73%   -33.36%     
   =============================================
     Files          1592     1627       +35     
     Lines         83916    85553     +1637     
     Branches      12706    12880      +174     
   =============================================
   - Hits          53779    26291    -27488     
   - Misses        26261    56888    +30627     
   + Partials       3876     2374     -1502     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `28.93% <35.62%> (?)` | |
   | integration2 | `27.55% <34.41%> (?)` | |
   | unittests1 | `?` | |
   | unittests2 | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...inot/controller/helix/ControllerRequestClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9Db250cm9sbGVyUmVxdWVzdENsaWVudC5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `76.66% <ø> (+76.66%)` | :arrow_up: |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `0.00% <0.00%> (-13.49%)` | :arrow_down: |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `39.37% <39.37%> (ø)` | |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `76.40% <60.00%> (+76.40%)` | :arrow_up: |
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `56.52% <61.29%> (+40.40%)` | :arrow_up: |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `65.51% <100.00%> (-6.90%)` | :arrow_down: |
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `58.09% <100.00%> (+51.42%)` | :arrow_up: |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `90.00% <100.00%> (+86.66%)` | :arrow_up: |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `74.60% <100.00%> (+71.42%)` | :arrow_up: |
   | ... and [1356 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [98d7810...2b865bf](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter commented on pull request #8329: using HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (07d412a) into [master](https://codecov.io/gh/apache/pinot/commit/1a701b7d52c5d571689d4b2f8159f8c0cdda0acd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1a701b7) will **decrease** coverage by `41.88%`.
   > The diff coverage is `56.77%`.
   
   > :exclamation: Current head 07d412a differs from pull request most recent head 2e401e6. Consider uploading reports for the commit 2e401e6 to get more accurate results
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8329/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #8329       +/-   ##
   =============================================
   - Coverage     70.86%   28.97%   -41.89%     
   =============================================
     Files          1635     1624       -11     
     Lines         85756    85417      -339     
     Branches      12912    12872       -40     
   =============================================
   - Hits          60771    24751    -36020     
   - Misses        20793    58404    +37611     
   + Partials       4192     2262     -1930     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `28.97% <56.77%> (-0.01%)` | :arrow_down: |
   | integration2 | `?` | |
   | unittests1 | `?` | |
   | unittests2 | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...inot/common/utils/fetcher/HttpsSegmentFetcher.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvZmV0Y2hlci9IdHRwc1NlZ21lbnRGZXRjaGVyLmphdmE=) | `0.00% <0.00%> (-58.34%)` | :arrow_down: |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `0.00% <0.00%> (-13.49%)` | :arrow_down: |
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `50.00% <49.09%> (-13.74%)` | :arrow_down: |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `58.44% <58.44%> (ø)` | |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `25.84% <60.00%> (-51.81%)` | :arrow_down: |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `65.51% <100.00%> (-6.90%)` | :arrow_down: |
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `58.49% <100.00%> (+0.39%)` | :arrow_up: |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `90.00% <100.00%> (ø)` | |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `74.60% <100.00%> (ø)` | |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `77.77% <100.00%> (+1.11%)` | :arrow_up: |
   | ... and [1188 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1a701b7...2e401e6](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] kkrugler commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
kkrugler commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829494766



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -60,6 +61,8 @@
   private static final String TRUSTSTORE_PASSWORD = "truststore.password";
   private static final String SSL_PROVIDER = "ssl.provider";
 
+  private static SSLContext _sslContext = null;

Review comment:
       Just curious about the naming convention for static variables - I'd thought they were always upper-cased.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] yupeng9 commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r828189443



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,371 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);

Review comment:
       make `Authorization ` a constant

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,371 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      String controllerHost = null;
+      String controllerVersion = null;
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+      }
+      if (controllerHost != null) {
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,

Review comment:
       shall this be extended to talk to server/broker?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: using HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (da1ce32) into [master](https://codecov.io/gh/apache/pinot/commit/8ef4dbacfb42596590d08c7c6f14f88093dfe7d5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8ef4dba) will **decrease** coverage by `41.92%`.
   > The diff coverage is `56.08%`.
   
   > :exclamation: Current head da1ce32 differs from pull request most recent head 6f2d43d. Consider uploading reports for the commit 6f2d43d to get more accurate results
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8329/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #8329       +/-   ##
   =============================================
   - Coverage     70.80%   28.88%   -41.93%     
   =============================================
     Files          1636     1625       -11     
     Lines         85804    85463      -341     
     Branches      12920    12881       -39     
   =============================================
   - Hits          60755    24687    -36068     
   - Misses        20858    58491    +37633     
   + Partials       4191     2285     -1906     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `28.88% <56.08%> (+<0.01%)` | :arrow_up: |
   | integration2 | `?` | |
   | unittests1 | `?` | |
   | unittests2 | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `76.66% <ø> (ø)` | |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `0.00% <0.00%> (-13.49%)` | :arrow_down: |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `23.59% <20.00%> (-54.06%)` | :arrow_down: |
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `51.48% <54.54%> (-11.52%)` | :arrow_down: |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `58.75% <58.75%> (ø)` | |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `65.51% <100.00%> (-6.90%)` | :arrow_down: |
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `51.42% <100.00%> (ø)` | |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `90.00% <100.00%> (ø)` | |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `74.60% <100.00%> (ø)` | |
   | [.../java/org/apache/pinot/spi/utils/BooleanUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdXRpbHMvQm9vbGVhblV0aWxzLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | ... and [1185 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8ef4dba...6f2d43d](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] walterddr commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r828433324



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,371 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendGetRequest(URI uri)

Review comment:
       since this is a generic HTTPClient. I think we can remove the `@Deprecated` annotation yes @Jackie-Jiang ^




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829533085



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       I suggest putting it in a holder class:
   
   ```java
   private static final AtomicReference<SSLContext> OVERRIDE = new AtomicReference<>();
   
   public static void registerOverride(SSLContext override) {
       if (!OVERRIDE.compareAndSet(null, override)) {
          // warn that something else beat the caller here
       }
   }
   
   private static final class Holder {
       static final SSLContext SSL_CONTEXT = OVERRIDE.get() == null ? SSLContexts.createDefault() : OVERRIDE.get();
   }
   ```
   
   And then reference it via `Holder.SSL_CONTEXT` everywhere. This has a few advantages:
   
   * registration is thread safe
   * mutable until first use
   * synchronization, at most once initialisation guaranteed by the classloader
   * after initialisation, the `SSLContext` is constant which can drive optimisations like constant folding, which may or may not be important

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,

Review comment:
       Yes, please do not use `String.format` - it's ridiculously expensive. Most of the time we use it, it is when something has gone wrong so I suppose it is ok but it should never be used on the happy path.

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,

Review comment:
       In this particular case, since there are 3 parameters a varargs array will be allocated for every log statement which will always be logged since it's at INFO, so it would be better just to concatenate the strings to minimise logging overhead.

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +262,38 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return SSLContextHolder.SSL_CONTEXT;
+  }
+
+  public static void setSslContext(SSLContext sslContext) {
+    registerOverride(sslContext);
+  }
+
+  private static void registerOverride(SSLContext override) {
+    if (!SSL_CONTEXT_REF.compareAndSet(null, override)) {
+      LOGGER.warn("SSL Context has already been set.");
+      // warn that something else beat the caller here
+    }
+  }

Review comment:
       oh I'd just inline this into `setSslContext`, `registerOverride` was just a name




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] walterddr commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r828435386



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,371 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      String controllerHost = null;
+      String controllerVersion = null;
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+      }
+      if (controllerHost != null) {
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,

Review comment:
       this is a logging only usage. only Controller response attaches special HTTP response header. but yes we can add others later if necessary




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: using HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e288a0b) into [master](https://codecov.io/gh/apache/pinot/commit/8ef4dbacfb42596590d08c7c6f14f88093dfe7d5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8ef4dba) will **decrease** coverage by `0.09%`.
   > The diff coverage is `45.95%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8329/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8329      +/-   ##
   ============================================
   - Coverage     70.80%   70.70%   -0.10%     
   + Complexity     4258     4255       -3     
   ============================================
     Files          1636     1638       +2     
     Lines         85804    85899      +95     
     Branches      12920    12927       +7     
   ============================================
   - Hits          60755    60739      -16     
   - Misses        20858    20969     +111     
     Partials       4191     4191              
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `28.87% <36.08%> (-0.01%)` | :arrow_down: |
   | integration2 | `27.65% <34.78%> (+0.14%)` | :arrow_up: |
   | unittests1 | `66.85% <22.04%> (-0.10%)` | :arrow_down: |
   | unittests2 | `14.17% <3.40%> (-0.01%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...not/ingestion/common/DefaultControllerRestApi.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1iYXRjaC1pbmdlc3Rpb24vdjBfZGVwcmVjYXRlZC9waW5vdC1pbmdlc3Rpb24tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9pbmdlc3Rpb24vY29tbW9uL0RlZmF1bHRDb250cm9sbGVyUmVzdEFwaS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `76.66% <ø> (ø)` | |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `13.48% <0.00%> (ø)` | |
   | [...inot/controller/helix/ControllerRequestClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9Db250cm9sbGVyUmVxdWVzdENsaWVudC5qYXZh) | `19.04% <19.04%> (ø)` | |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `41.66% <41.66%> (ø)` | |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `76.40% <60.00%> (-1.25%)` | :arrow_down: |
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `64.35% <76.36%> (+1.35%)` | :arrow_up: |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `72.41% <100.00%> (ø)` | |
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `51.42% <100.00%> (ø)` | |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `90.00% <100.00%> (ø)` | |
   | ... and [25 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8ef4dba...e288a0b](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: using HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e288a0b) into [master](https://codecov.io/gh/apache/pinot/commit/8ef4dbacfb42596590d08c7c6f14f88093dfe7d5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8ef4dba) will **decrease** coverage by `6.84%`.
   > The diff coverage is `20.85%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8329/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8329      +/-   ##
   ============================================
   - Coverage     70.80%   63.96%   -6.85%     
   + Complexity     4258     4255       -3     
   ============================================
     Files          1636     1593      -43     
     Lines         85804    84015    -1789     
     Branches      12920    12723     -197     
   ============================================
   - Hits          60755    53741    -7014     
   - Misses        20858    26403    +5545     
   + Partials       4191     3871     -320     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `66.85% <22.04%> (-0.10%)` | :arrow_down: |
   | unittests2 | `14.17% <3.40%> (-0.01%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `6.66% <0.00%> (-44.77%)` | :arrow_down: |
   | [...not/ingestion/common/DefaultControllerRestApi.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1iYXRjaC1pbmdlc3Rpb24vdjBfZGVwcmVjYXRlZC9waW5vdC1pbmdlc3Rpb24tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9pbmdlc3Rpb24vY29tbW9uL0RlZmF1bHRDb250cm9sbGVyUmVzdEFwaS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.33% <0.00%> (-86.67%)` | :arrow_down: |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.17% <0.00%> (-71.43%)` | :arrow_down: |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `0.00% <ø> (-76.67%)` | :arrow_down: |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `13.48% <0.00%> (ø)` | |
   | [...inot/controller/helix/ControllerRequestClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9Db250cm9sbGVyUmVxdWVzdENsaWVudC5qYXZh) | `19.04% <19.04%> (ø)` | |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `1.12% <20.00%> (-76.53%)` | :arrow_down: |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `20.00% <20.00%> (ø)` | |
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `14.85% <27.27%> (-48.16%)` | :arrow_down: |
   | ... and [385 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8ef4dba...e288a0b](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] walterddr commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829216994



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       i dont know whether this needs to be synchronized as it is only generated once.
   
   the only way to modify it is via setSslContext, which should also be statically accessed when the secure environment constructs at startup time). 
   
   CC the original author @apucher ^




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829289735



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,371 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendGetRequest(URI uri)

Review comment:
       Yes, we don't need to make it deprecated




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829798759



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +262,38 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return SSLContextHolder.SSL_CONTEXT;
+  }
+
+  public static void setSslContext(SSLContext sslContext) {
+    registerOverride(sslContext);
+  }
+
+  private static void registerOverride(SSLContext override) {
+    if (!SSL_CONTEXT_REF.compareAndSet(null, override)) {
+      LOGGER.warn("SSL Context has already been set.");
+      // warn that something else beat the caller here
+    }
+  }

Review comment:
       oh I'd just inline this into `setSslContext`, `registerOverride` was just a name




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829294078



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());

Review comment:
       This can be simplified to avoid duplicate code, same for other methods
   ```suggestion
       return sendGetRequest(uri, null);
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829535449



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,

Review comment:
       Yes, please do not use `String.format` - it's ridiculously expensive. Most of the time we use it, it is when something has gone wrong so I suppose it is ok but it should never be used on the happy path.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829533085



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       I suggest putting it in a holder class:
   
   ```java
   private static final AtomicReference<SSLContext> OVERRIDE = new AtomicReference<>();
   
   public static void registerOverride(SSLContext override) {
       if (!OVERRIDE.compareAndSet(null, override)) {
          // warn that something else beat the caller here
       }
   }
   
   private static final class Holder {
       static final SSLContext SSL_CONTEXT = OVERRIDE.get() == null ? SSLContexts.createDefault() : OVERRIDE.get();
   }
   ```
   
   And then reference it via `Holder.SSL_CONTEXT` everywhere. This has a few advantages:
   
   * registration is thread safe
   * mutable until first use
   * synchronization, at most once initialisation guaranteed by the classloader
   * after initialisation, the `SSLContext` is constant which can drive optimisations like constant folding, which may or may not be important




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] walterddr commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829638811



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)

Review comment:
       yeah weirdly we have some post method with empty body :-/

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,
+                controllerVersion));
+      }
+      int statusCode = response.getStatusLine().getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(request, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  public CloseableHttpResponse execute(HttpUriRequest request)
+      throws IOException {
+    return _httpClient.execute(request);
+  }
+
+  // --------------------------------------------------------------------------
+  // Multi-part post/put APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendMultipartPostRequest(String url, String body)
+      throws IOException {
+    HttpPost post = new HttpPost(url);
+    // our handlers ignore key...so we can put anything here
+    MultipartEntityBuilder builder = MultipartEntityBuilder.create();
+    builder.addTextBody("body", body);
+    post.setEntity(builder.build());

Review comment:
       for now i dont see it be useful. so i would prefer to add it when it is used. 
   
   (also tried to use sendPostRequest it had some weird test errors)

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)

Review comment:
       this would be a major refactoring. should we do this in a separate PR?

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -232,8 +235,7 @@ public static void installDefaultSSLSocketFactory(String keyStoreType, String ke
       Protocol.registerProtocol("https",
           new Protocol(CommonConstants.HTTPS_PROTOCOL, new PinotProtocolSocketFactory(sc.getSocketFactory()), 443));
 
-      // FileUploadDownloadClient
-      FileUploadDownloadClient.installDefaultSSLContext(sc);
+      _sslContext = sc;

Review comment:
       yup. i guess it aligns with other comments will address it accordingly

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -60,6 +61,8 @@
   private static final String TRUSTSTORE_PASSWORD = "truststore.password";
   private static final String SSL_PROVIDER = "ssl.provider";
 
+  private static SSLContext _sslContext = null;

Review comment:
       seems it only apply to private static final variables




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] walterddr commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829216994



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       i dont know whether this needs to be synchronized as it is only generated once.
   
   the only way to modify it is via setSslContext, which should also be statically accessed when the secure environment constructs at startup time). 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] yupeng9 commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r828188208



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,371 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendGetRequest(URI uri)

Review comment:
       why do we add deprecated method, given this is a new file




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] walterddr commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829216994



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       i dont know whether this needs to be synchronized as it is only generated once.
   
   the only way to modify it is via setSslContext, which should also be statically accessed when the secure environment constructs at startup time). 

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       i dont know whether this needs to be synchronized as it is only generated once.
   
   the only way to modify it is via setSslContext, which should also be statically accessed when the secure environment constructs at startup time). 
   
   CC the original author @apucher ^

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       yeah I was thinking about using lazy static initialization similar to https://stackoverflow.com/a/7420714 since it is static thread safe. 
   
   However the current way how TLSUtils install the TLSSpec is non-static. so it might not be possible to acquire the TLSSpecs at jvm start up time. 

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       yeah. this is what I meant previously (less the `OVERRIDE` part which brilliantly solve the tls keystore init issue). thanks @richardstartin !

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)

Review comment:
       yeah weirdly we have some post method with empty body :-/

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,
+                controllerVersion));
+      }
+      int statusCode = response.getStatusLine().getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(request, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  public CloseableHttpResponse execute(HttpUriRequest request)
+      throws IOException {
+    return _httpClient.execute(request);
+  }
+
+  // --------------------------------------------------------------------------
+  // Multi-part post/put APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendMultipartPostRequest(String url, String body)
+      throws IOException {
+    HttpPost post = new HttpPost(url);
+    // our handlers ignore key...so we can put anything here
+    MultipartEntityBuilder builder = MultipartEntityBuilder.create();
+    builder.addTextBody("body", body);
+    post.setEntity(builder.build());

Review comment:
       for now i dont see it be useful. so i would prefer to add it when it is used. 
   
   (also tried to use sendPostRequest it had some weird test errors)

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)

Review comment:
       this would be a major refactoring. should we do this in a separate PR?

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -232,8 +235,7 @@ public static void installDefaultSSLSocketFactory(String keyStoreType, String ke
       Protocol.registerProtocol("https",
           new Protocol(CommonConstants.HTTPS_PROTOCOL, new PinotProtocolSocketFactory(sc.getSocketFactory()), 443));
 
-      // FileUploadDownloadClient
-      FileUploadDownloadClient.installDefaultSSLContext(sc);
+      _sslContext = sc;

Review comment:
       yup. i guess it aligns with other comments will address it accordingly

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -60,6 +61,8 @@
   private static final String TRUSTSTORE_PASSWORD = "truststore.password";
   private static final String SSL_PROVIDER = "ssl.provider";
 
+  private static SSLContext _sslContext = null;

Review comment:
       seems it only apply to private static final variables




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r830247106



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)

Review comment:
       Then we should use the regular post request




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: using HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5463ff4) into [master](https://codecov.io/gh/apache/pinot/commit/8ef4dbacfb42596590d08c7c6f14f88093dfe7d5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8ef4dba) will **decrease** coverage by `56.59%`.
   > The diff coverage is `3.13%`.
   
   > :exclamation: Current head 5463ff4 differs from pull request most recent head 5b585ee. Consider uploading reports for the commit 5b585ee to get more accurate results
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8329/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #8329       +/-   ##
   =============================================
   - Coverage     70.80%   14.20%   -56.60%     
   + Complexity     4258       81     -4177     
   =============================================
     Files          1636     1593       -43     
     Lines         85804    84033     -1771     
     Branches      12920    12726      -194     
   =============================================
   - Hits          60755    11939    -48816     
   - Misses        20858    71189    +50331     
   + Partials       4191      905     -3286     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `?` | |
   | unittests2 | `14.20% <3.13%> (+0.02%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `0.00% <0.00%> (-63.01%)` | :arrow_down: |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `0.00% <0.00%> (-77.65%)` | :arrow_down: |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `0.00% <0.00%> (-72.42%)` | :arrow_down: |
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `0.00% <0.00%> (-51.43%)` | :arrow_down: |
   | [...not/ingestion/common/DefaultControllerRestApi.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1iYXRjaC1pbmdlc3Rpb24vdjBfZGVwcmVjYXRlZC9waW5vdC1pbmdlc3Rpb24tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9pbmdlc3Rpb24vY29tbW9uL0RlZmF1bHRDb250cm9sbGVyUmVzdEFwaS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.33% <0.00%> (-86.67%)` | :arrow_down: |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.17% <0.00%> (-71.43%)` | :arrow_down: |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `0.00% <ø> (-76.67%)` | :arrow_down: |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `0.00% <0.00%> (-13.49%)` | :arrow_down: |
   | ... and [1314 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8ef4dba...5b585ee](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] kkrugler commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
kkrugler commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829485698



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,

Review comment:
       Just for my own edification, I thought that since we were using slf4j, the approved approach for formatted log msgs was to use slf4j's support for `{}` parameters. E.g.
   ``` java
   LOGGER.info("Sending request: {} to controller: {}, version: {}", request.getURI(), controllerHost, controllerVersion);
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829538546



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,

Review comment:
       In this particular case, since there are 3 parameters a varargs array will be allocated for every log statement which will always be logged since it's at INFO, so it would be better just to concatenate the strings to minimise logging overhead.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829300127



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)

Review comment:
       Should `jsonRequestBody` be `Nullable`?

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,
+                controllerVersion));
+      }
+      int statusCode = response.getStatusLine().getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(request, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  public CloseableHttpResponse execute(HttpUriRequest request)
+      throws IOException {
+    return _httpClient.execute(request);
+  }
+
+  // --------------------------------------------------------------------------
+  // Multi-part post/put APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendMultipartPostRequest(String url, String body)
+      throws IOException {
+    HttpPost post = new HttpPost(url);
+    // our handlers ignore key...so we can put anything here
+    MultipartEntityBuilder builder = MultipartEntityBuilder.create();
+    builder.addTextBody("body", body);
+    post.setEntity(builder.build());
+    try (CloseableHttpResponse response = _httpClient.execute(post)) {
+      StatusLine statusLine = response.getStatusLine();
+      int statusCode = statusLine.getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(post, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  public SimpleHttpResponse sendMultipartPutRequest(String url, String body)
+      throws IOException {
+    HttpPut put = new HttpPut(url);
+    // our handlers ignore key...so we can put anything here
+    MultipartEntityBuilder builder = MultipartEntityBuilder.create();
+    builder.addTextBody("body", body);
+    put.setEntity(builder.build());
+    try (CloseableHttpResponse response = _httpClient.execute(put)) {
+      StatusLine statusLine = response.getStatusLine();
+      int statusCode = statusLine.getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(put, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  // --------------------------------------------------------------------------
+  // Static utility for dealing with lower-level API responses.
+  // --------------------------------------------------------------------------
+
+  public static SimpleHttpResponse wrapAndThrowHttpException(SimpleHttpResponse resp)
+      throws HttpErrorStatusException {
+    if (resp.getStatusCode() >= 300) {
+      throw new HttpErrorStatusException(resp.getResponse(), resp.getStatusCode());
+    } else {
+      return resp;
+    }
+  }
+
+  public static String getErrorMessage(HttpUriRequest request, CloseableHttpResponse response) {

Review comment:
       Make it `private`

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)

Review comment:
       Consider adding parameter for `headers`, `parameters`, `socketTimeoutMs` so that it is flexible. `sendGetRequest(uri)` will call `sendGetRequest(uri, null, null, null, DEFAULT_GET_REQUEST_TIMEOUT_MS)`. Same for other methods

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)

Review comment:
       Annotate `payload` and `headers` as `Nullable`, same for other methods

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)

Review comment:
       We should also include the `authToken`

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)

Review comment:
       Renaming to `sendJsonPostRequest` to be consistent? (e.g. we have `sendMultipartPostRequest`)

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,
+                controllerVersion));
+      }
+      int statusCode = response.getStatusLine().getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(request, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  public CloseableHttpResponse execute(HttpUriRequest request)

Review comment:
       Seems this is only used in `FileUploadDownloadClient.downloadFile()`, I'd suggest moving that method here as well. We should try to avoid exposing this API as caller needs to close the response

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,
+                controllerVersion));
+      }
+      int statusCode = response.getStatusLine().getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(request, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  public CloseableHttpResponse execute(HttpUriRequest request)
+      throws IOException {
+    return _httpClient.execute(request);
+  }
+
+  // --------------------------------------------------------------------------
+  // Multi-part post/put APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendMultipartPostRequest(String url, String body)
+      throws IOException {
+    HttpPost post = new HttpPost(url);
+    // our handlers ignore key...so we can put anything here
+    MultipartEntityBuilder builder = MultipartEntityBuilder.create();
+    builder.addTextBody("body", body);
+    post.setEntity(builder.build());

Review comment:
       Can we reuse `sendPostRequest`?
   Also should we add `headers` and `authToken` support?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (58eaef1) into [master](https://codecov.io/gh/apache/pinot/commit/98d781092136eeb3a80e9e54fc8736f4c3d270f5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (98d7810) will **decrease** coverage by `0.04%`.
   > The diff coverage is `17.44%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8329      +/-   ##
   ============================================
   - Coverage     64.08%   64.04%   -0.05%     
   - Complexity     4254     4267      +13     
   ============================================
     Files          1592     1596       +4     
     Lines         83916    84156     +240     
     Branches      12706    12730      +24     
   ============================================
   + Hits          53779    53898     +119     
   - Misses        26261    26371     +110     
   - Partials       3876     3887      +11     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests1 | `66.94% <17.88%> (-0.02%)` | :arrow_down: |
   | unittests2 | `14.22% <2.68%> (+0.02%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `6.66% <0.00%> (ø)` | |
   | [...not/ingestion/common/DefaultControllerRestApi.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1iYXRjaC1pbmdlc3Rpb24vdjBfZGVwcmVjYXRlZC9waW5vdC1pbmdlc3Rpb24tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9pbmdlc3Rpb24vY29tbW9uL0RlZmF1bHRDb250cm9sbGVyUmVzdEFwaS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.33% <0.00%> (ø)` | |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.17% <0.00%> (ø)` | |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `0.00% <ø> (ø)` | |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `13.48% <0.00%> (ø)` | |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `14.46% <14.46%> (ø)` | |
   | [...inot/controller/helix/ControllerRequestClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9Db250cm9sbGVyUmVxdWVzdENsaWVudC5qYXZh) | `17.77% <17.77%> (ø)` | |
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `18.07% <22.85%> (+1.96%)` | :arrow_up: |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `4.21% <36.36%> (+4.21%)` | :arrow_up: |
   | ... and [23 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [98d7810...58eaef1](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829288558



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       Also CC @richardstartin to give suggestions on how to set static variables

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,371 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendGetRequest(URI uri)

Review comment:
       Yes, we don't need to make it deprecated

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());

Review comment:
       This can be simplified to avoid duplicate code, same for other methods
   ```suggestion
       return sendGetRequest(uri, null);
   ```

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)

Review comment:
       Should `jsonRequestBody` be `Nullable`?

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,
+                controllerVersion));
+      }
+      int statusCode = response.getStatusLine().getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(request, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  public CloseableHttpResponse execute(HttpUriRequest request)
+      throws IOException {
+    return _httpClient.execute(request);
+  }
+
+  // --------------------------------------------------------------------------
+  // Multi-part post/put APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendMultipartPostRequest(String url, String body)
+      throws IOException {
+    HttpPost post = new HttpPost(url);
+    // our handlers ignore key...so we can put anything here
+    MultipartEntityBuilder builder = MultipartEntityBuilder.create();
+    builder.addTextBody("body", body);
+    post.setEntity(builder.build());
+    try (CloseableHttpResponse response = _httpClient.execute(post)) {
+      StatusLine statusLine = response.getStatusLine();
+      int statusCode = statusLine.getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(post, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  public SimpleHttpResponse sendMultipartPutRequest(String url, String body)
+      throws IOException {
+    HttpPut put = new HttpPut(url);
+    // our handlers ignore key...so we can put anything here
+    MultipartEntityBuilder builder = MultipartEntityBuilder.create();
+    builder.addTextBody("body", body);
+    put.setEntity(builder.build());
+    try (CloseableHttpResponse response = _httpClient.execute(put)) {
+      StatusLine statusLine = response.getStatusLine();
+      int statusCode = statusLine.getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(put, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  // --------------------------------------------------------------------------
+  // Static utility for dealing with lower-level API responses.
+  // --------------------------------------------------------------------------
+
+  public static SimpleHttpResponse wrapAndThrowHttpException(SimpleHttpResponse resp)
+      throws HttpErrorStatusException {
+    if (resp.getStatusCode() >= 300) {
+      throw new HttpErrorStatusException(resp.getResponse(), resp.getStatusCode());
+    } else {
+      return resp;
+    }
+  }
+
+  public static String getErrorMessage(HttpUriRequest request, CloseableHttpResponse response) {

Review comment:
       Make it `private`

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)

Review comment:
       Consider adding parameter for `headers`, `parameters`, `socketTimeoutMs` so that it is flexible. `sendGetRequest(uri)` will call `sendGetRequest(uri, null, null, null, DEFAULT_GET_REQUEST_TIMEOUT_MS)`. Same for other methods

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)

Review comment:
       Annotate `payload` and `headers` as `Nullable`, same for other methods

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)

Review comment:
       We should also include the `authToken`

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)

Review comment:
       Renaming to `sendJsonPostRequest` to be consistent? (e.g. we have `sendMultipartPostRequest`)

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,
+                controllerVersion));
+      }
+      int statusCode = response.getStatusLine().getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(request, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  public CloseableHttpResponse execute(HttpUriRequest request)

Review comment:
       Seems this is only used in `FileUploadDownloadClient.downloadFile()`, I'd suggest moving that method here as well. We should try to avoid exposing this API as caller needs to close the response

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,
+                controllerVersion));
+      }
+      int statusCode = response.getStatusLine().getStatusCode();
+      if (statusCode >= 300) {
+        return new SimpleHttpResponse(statusCode, getErrorMessage(request, response));
+      }
+      return new SimpleHttpResponse(statusCode, EntityUtils.toString(response.getEntity()));
+    }
+  }
+
+  public CloseableHttpResponse execute(HttpUriRequest request)
+      throws IOException {
+    return _httpClient.execute(request);
+  }
+
+  // --------------------------------------------------------------------------
+  // Multi-part post/put APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendMultipartPostRequest(String url, String body)
+      throws IOException {
+    HttpPost post = new HttpPost(url);
+    // our handlers ignore key...so we can put anything here
+    MultipartEntityBuilder builder = MultipartEntityBuilder.create();
+    builder.addTextBody("body", body);
+    post.setEntity(builder.build());

Review comment:
       Can we reuse `sendPostRequest`?
   Also should we add `headers` and `authToken` support?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] kkrugler commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
kkrugler commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829485698



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,364 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String AUTH_HTTP_HEADER = "Authorization";
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader(AUTH_HTTP_HEADER, authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        String controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        String controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,

Review comment:
       Just for my own edification, I thought that since we were using slf4j, the approved approach for formatted log msgs was to use slf4j's support for `{}` parameters. E.g.
   ``` java
   LOGGER.info("Sending request: {} to controller: {}, version: {}", request.getURI(), controllerHost, controllerVersion);
   ```

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -60,6 +61,8 @@
   private static final String TRUSTSTORE_PASSWORD = "truststore.password";
   private static final String SSL_PROVIDER = "ssl.provider";
 
+  private static SSLContext _sslContext = null;

Review comment:
       Just curious about the naming convention for static variables - I'd thought they were always upper-cased.

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -232,8 +235,7 @@ public static void installDefaultSSLSocketFactory(String keyStoreType, String ke
       Protocol.registerProtocol("https",
           new Protocol(CommonConstants.HTTPS_PROTOCOL, new PinotProtocolSocketFactory(sc.getSocketFactory()), 443));
 
-      // FileUploadDownloadClient
-      FileUploadDownloadClient.installDefaultSSLContext(sc);
+      _sslContext = sc;

Review comment:
       I think this should call `setSslContext()`, and that method should be synchronized, or use an `AtomicReference< SSLContext>`, to avoid potential concurrency issues.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8329: using HttpClient

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#issuecomment-1063683490


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8329](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a29ca6f) into [master](https://codecov.io/gh/apache/pinot/commit/8ef4dbacfb42596590d08c7c6f14f88093dfe7d5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8ef4dba) will **decrease** coverage by `56.63%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8329/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #8329       +/-   ##
   =============================================
   - Coverage     70.80%   14.17%   -56.64%     
   + Complexity     4258       81     -4177     
   =============================================
     Files          1636     1592       -44     
     Lines         85804    83932     -1872     
     Branches      12920    12714      -206     
   =============================================
   - Hits          60755    11895    -48860     
   - Misses        20858    71144    +50286     
   + Partials       4191      893     -3298     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `?` | |
   | unittests2 | `14.17% <0.00%> (-0.01%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `0.00% <0.00%> (-63.01%)` | :arrow_down: |
   | [...n/java/org/apache/pinot/common/utils/TlsUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvVGxzVXRpbHMuamF2YQ==) | `0.00% <0.00%> (-77.65%)` | :arrow_down: |
   | [...org/apache/pinot/common/utils/http/HttpClient.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaHR0cC9IdHRwQ2xpZW50LmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...ger/realtime/Server2ControllerSegmentUploader.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvU2VydmVyMkNvbnRyb2xsZXJTZWdtZW50VXBsb2FkZXIuamF2YQ==) | `0.00% <0.00%> (-72.42%)` | :arrow_down: |
   | [...altime/ServerSegmentCompletionProtocolHandler.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VydmVyL3JlYWx0aW1lL1NlcnZlclNlZ21lbnRDb21wbGV0aW9uUHJvdG9jb2xIYW5kbGVyLmphdmE=) | `0.00% <0.00%> (-51.43%)` | :arrow_down: |
   | [...not/ingestion/common/DefaultControllerRestApi.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1iYXRjaC1pbmdlc3Rpb24vdjBfZGVwcmVjYXRlZC9waW5vdC1pbmdlc3Rpb24tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9pbmdlc3Rpb24vY29tbW9uL0RlZmF1bHRDb250cm9sbGVyUmVzdEFwaS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [.../tasks/BaseMultipleSegmentsConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZU11bHRpcGxlU2VnbWVudHNDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.33% <0.00%> (-86.67%)` | :arrow_down: |
   | [...ion/tasks/BaseSingleSegmentConversionExecutor.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvQmFzZVNpbmdsZVNlZ21lbnRDb252ZXJzaW9uRXhlY3V0b3IuamF2YQ==) | `3.17% <0.00%> (-71.43%)` | :arrow_down: |
   | [...ot/plugin/minion/tasks/SegmentConversionUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvU2VnbWVudENvbnZlcnNpb25VdGlscy5qYXZh) | `0.00% <ø> (-76.67%)` | :arrow_down: |
   | [...he/pinot/segment/local/utils/SegmentPushUtils.java](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9TZWdtZW50UHVzaFV0aWxzLmphdmE=) | `0.00% <0.00%> (-13.49%)` | :arrow_down: |
   | ... and [1313 more](https://codecov.io/gh/apache/pinot/pull/8329/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8ef4dba...a29ca6f](https://codecov.io/gh/apache/pinot/pull/8329?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] walterddr commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829305394



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       yeah I was thinking about using lazy static initialization similar to https://stackoverflow.com/a/7420714 since it is static thread safe. 
   
   However the current way how TLSUtils install the TLSSpec is non-static. so it might not be possible to acquire the TLSSpecs at jvm start up time. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] kkrugler commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
kkrugler commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829506216



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -232,8 +235,7 @@ public static void installDefaultSSLSocketFactory(String keyStoreType, String ke
       Protocol.registerProtocol("https",
           new Protocol(CommonConstants.HTTPS_PROTOCOL, new PinotProtocolSocketFactory(sc.getSocketFactory()), 443));
 
-      // FileUploadDownloadClient
-      FileUploadDownloadClient.installDefaultSSLContext(sc);
+      _sslContext = sc;

Review comment:
       I think this should call `setSslContext()`, and that method should be synchronized, or use an `AtomicReference< SSLContext>`, to avoid potential concurrency issues.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] yupeng9 commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r828194687



##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/ControllerRequestClient.java
##########
@@ -0,0 +1,117 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.controller.helix;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.net.URL;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.http.HttpClient;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.tenant.Tenant;
+import org.apache.pinot.spi.config.tenant.TenantRole;
+import org.apache.pinot.spi.data.Schema;
+
+
+public class ControllerRequestClient {

Review comment:
       need docs
   

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       synchronized?

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,371 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);

Review comment:
       make `Authorization ` a constant

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
##########
@@ -0,0 +1,371 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.utils.http;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import javax.net.ssl.SSLContext;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HttpEntity;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpVersion;
+import org.apache.http.NameValuePair;
+import org.apache.http.StatusLine;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpPut;
+import org.apache.http.client.methods.HttpUriRequest;
+import org.apache.http.client.methods.RequestBuilder;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.entity.mime.MultipartEntityBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.message.BasicHeader;
+import org.apache.http.util.EntityUtils;
+import org.apache.pinot.common.exception.HttpErrorStatusException;
+import org.apache.pinot.common.utils.SimpleHttpErrorInfo;
+import org.apache.pinot.common.utils.SimpleHttpResponse;
+import org.apache.pinot.common.utils.TlsUtils;
+import org.apache.pinot.spi.utils.CommonConstants;
+import org.apache.pinot.spi.utils.JsonUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The {@code HTTPClient} wraps around a {@link CloseableHttpClient} to provide a reusable client for making
+ * HTTP requests.
+ */
+public class HttpClient implements AutoCloseable {
+  private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);
+
+  public static final int DEFAULT_SOCKET_TIMEOUT_MS = 600 * 1000; // 10 minutes
+  public static final int GET_REQUEST_SOCKET_TIMEOUT_MS = 5 * 1000; // 5 seconds
+  public static final int DELETE_REQUEST_SOCKET_TIMEOUT_MS = 10 * 1000; // 10 seconds
+  public static final String JSON_CONTENT_TYPE = "application/json";
+
+  private final CloseableHttpClient _httpClient;
+
+  public HttpClient() {
+    this(null);
+  }
+
+  public HttpClient(@Nullable SSLContext sslContext) {
+    SSLContext context = sslContext != null ? sslContext : TlsUtils.getSslContext();
+    // Set NoopHostnameVerifier to skip validating hostname when uploading/downloading segments.
+    SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(context, NoopHostnameVerifier.INSTANCE);
+    _httpClient = HttpClients.custom().setSSLSocketFactory(csf).build();
+  }
+
+  // --------------------------------------------------------------------------
+  // Generic HTTP Request APIs
+  // --------------------------------------------------------------------------
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendGetRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendGetRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendGetRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.get(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, GET_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendDeleteRequest(URI, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendDeleteRequest(URI uri)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  public SimpleHttpResponse sendDeleteRequest(URI uri, @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.delete(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPostRequest(URI, HttpEntity, Map, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPostRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPostRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.post(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    if (MapUtils.isNotEmpty(headers)) {
+      for (Map.Entry<String, String> header : headers.entrySet()) {
+        requestBuilder.addHeader(header.getKey(), header.getValue());
+      }
+    }
+    setTimeout(requestBuilder, DEFAULT_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  /**
+   * Deprecated due to lack of auth header support. May break for deployments with auth enabled
+   *
+   * @see #sendPutRequest(URI, HttpEntity, Map, String)
+   */
+  @Deprecated
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers)
+      throws IOException {
+    return sendPutRequest(uri, payload, headers, null);
+  }
+
+  public SimpleHttpResponse sendPutRequest(URI uri, HttpEntity payload, Map<String, String> headers,
+      @Nullable String authToken)
+      throws IOException {
+    RequestBuilder requestBuilder = RequestBuilder.put(uri).setVersion(HttpVersion.HTTP_1_1);
+    if (payload != null) {
+      requestBuilder.setEntity(payload);
+    }
+    if (StringUtils.isNotBlank(authToken)) {
+      requestBuilder.addHeader("Authorization", authToken);
+    }
+    setTimeout(requestBuilder, DELETE_REQUEST_SOCKET_TIMEOUT_MS);
+    return sendRequest(requestBuilder.build());
+  }
+
+  // --------------------------------------------------------------------------
+  // JSON post/put utility APIs
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return postJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse postJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPostRequest(uri, entity, headers, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String jsonRequestBody)
+      throws IOException {
+    return putJsonRequest(uri, jsonRequestBody, null);
+  }
+
+  public SimpleHttpResponse putJsonRequest(URI uri, @Nullable String requestBody,
+      @Nullable Map<String, String> headers)
+      throws IOException {
+    if (MapUtils.isEmpty(headers)) {
+      headers = new HashMap<>();
+    }
+    headers.put(HttpHeaders.CONTENT_TYPE, JSON_CONTENT_TYPE);
+    HttpEntity entity = requestBody == null ? null : new StringEntity(requestBody, ContentType.APPLICATION_JSON);
+    return sendPutRequest(uri, entity, headers, null);
+  }
+
+  // --------------------------------------------------------------------------
+  // Lower-level request/execute APIs.
+  // --------------------------------------------------------------------------
+
+  public SimpleHttpResponse sendRequest(HttpUriRequest request)
+      throws IOException {
+    try (CloseableHttpResponse response = _httpClient.execute(request)) {
+      String controllerHost = null;
+      String controllerVersion = null;
+      if (response.containsHeader(CommonConstants.Controller.HOST_HTTP_HEADER)) {
+        controllerHost = response.getFirstHeader(CommonConstants.Controller.HOST_HTTP_HEADER).getValue();
+        controllerVersion = response.getFirstHeader(CommonConstants.Controller.VERSION_HTTP_HEADER).getValue();
+      }
+      if (controllerHost != null) {
+        LOGGER.info(String
+            .format("Sending request: %s to controller: %s, version: %s", request.getURI(), controllerHost,

Review comment:
       shall this be extended to talk to server/broker?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8329: Refactor Pinot http utils into HttpClient

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8329:
URL: https://github.com/apache/pinot/pull/8329#discussion_r829288558



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/utils/TlsUtils.java
##########
@@ -255,6 +257,14 @@ public static URL makeKeyStoreUrl(String storePath)
     return inputUri.toURL();
   }
 
+  public static SSLContext getSslContext() {
+    return _sslContext != null ? _sslContext : SSLContexts.createDefault();

Review comment:
       Also CC @richardstartin to give suggestions on how to set static variables




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org