You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/11/22 23:16:33 UTC
[GitHub] [pinot] xiangfu0 opened a new pull request #7816: Adding pinot client connection config to allow skip fail on broker response exception
xiangfu0 opened a new pull request #7816:
URL: https://github.com/apache/pinot/pull/7816
## Description
Follow up https://github.com/apache/pinot/issues/7770
1. Adding a constructor parameter for pinot connection with config
2. Allow skip fail on broker response exception
## Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)
* [ ] Yes (Please label as **<code>backward-incompat</code>**, and complete the section below on Release Notes)
Does this PR fix a zero-downtime upgrade introduced earlier?
* [ ] Yes (Please label this as **<code>backward-incompat</code>**, and complete the section below on Release Notes)
Does this PR otherwise need attention when creating release notes? Things to consider:
- New configuration options
- Deprecation of configurations
- Signature changes to public methods/interfaces
- New plugins added or old plugins removed
* [ ] Yes (Please label this PR as **<code>release-notes</code>** and complete the section on Release Notes)
## Release Notes
<!-- If you have tagged this as either backward-incompat or release-notes,
you MUST add text here that you would like to see appear in release notes of the
next release. -->
<!-- If you have a series of commits adding or enabling a feature, then
add this section only in final commit that marks the feature completed.
Refer to earlier release notes to see examples of text.
-->
## Documentation
<!-- If you have introduced a new feature or configuration, please add it to the documentation as well.
See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] xiangfu0 merged pull request #7816: Adding pinot client connection config to allow skip fail on broker response exception
Posted by GitBox <gi...@apache.org>.
xiangfu0 merged pull request #7816:
URL: https://github.com/apache/pinot/pull/7816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #7816: Adding pinot client connection config to allow skip fail on broker response exception
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #7816:
URL: https://github.com/apache/pinot/pull/7816#issuecomment-976405029
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7816?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7816](https://codecov.io/gh/apache/pinot/pull/7816?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6e20bb4) into [master](https://codecov.io/gh/apache/pinot/commit/b7b82584b3abc3e4bd679058a69604db828a6d25?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b7b8258) will **decrease** coverage by `7.28%`.
> The diff coverage is `39.28%`.
> :exclamation: Current head 6e20bb4 differs from pull request most recent head 1139b2e. Consider uploading reports for the commit 1139b2e to get more accurate results
[](https://codecov.io/gh/apache/pinot/pull/7816?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7816 +/- ##
============================================
- Coverage 71.66% 64.38% -7.29%
+ Complexity 4086 3950 -136
============================================
Files 1578 1570 -8
Lines 80773 80460 -313
Branches 12002 11967 -35
============================================
- Hits 57888 51805 -6083
- Misses 18993 24989 +5996
+ Partials 3892 3666 -226
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `29.44% <39.28%> (+0.15%)` | :arrow_up: |
| integration2 | `27.91% <32.14%> (+0.01%)` | :arrow_up: |
| unittests1 | `68.73% <0.00%> (+0.05%)` | :arrow_up: |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7816?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `22.44% <22.22%> (-18.86%)` | :arrow_down: |
| [...ava/org/apache/pinot/client/ConnectionFactory.java](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb25GYWN0b3J5LmphdmE=) | `35.29% <28.57%> (-48.04%)` | :arrow_down: |
| [...n/java/org/apache/pinot/client/ResultSetGroup.java](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L1Jlc3VsdFNldEdyb3VwLmphdmE=) | `50.00% <30.76%> (-15.39%)` | :arrow_down: |
| [...ces/PinotSegmentUploadDownloadRestletResource.java](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1Bpbm90U2VnbWVudFVwbG9hZERvd25sb2FkUmVzdGxldFJlc291cmNlLmphdmE=) | `53.47% <33.33%> (-6.26%)` | :arrow_down: |
| [...apache/pinot/common/utils/SimpleHttpErrorInfo.java](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvU2ltcGxlSHR0cEVycm9ySW5mby5qYXZh) | `60.00% <60.00%> (ø)` | |
| [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `62.45% <66.66%> (+0.14%)` | :arrow_up: |
| [...r/api/resources/WebApplicationExceptionMapper.java](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1dlYkFwcGxpY2F0aW9uRXhjZXB0aW9uTWFwcGVyLmphdmE=) | `75.00% <100.00%> (-9.62%)` | :arrow_down: |
| [...pinot/controller/recommender/io/ConfigManager.java](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9yZWNvbW1lbmRlci9pby9Db25maWdNYW5hZ2VyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...troller/recommender/io/metadata/FieldMetadata.java](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9yZWNvbW1lbmRlci9pby9tZXRhZGF0YS9GaWVsZE1ldGFkYXRhLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...roller/recommender/rules/impl/BloomFilterRule.java](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9yZWNvbW1lbmRlci9ydWxlcy9pbXBsL0Jsb29tRmlsdGVyUnVsZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [221 more](https://codecov.io/gh/apache/pinot/pull/7816/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7816?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7816?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [b7b8258...1139b2e](https://codecov.io/gh/apache/pinot/pull/7816?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7816: Adding pinot client connection config to allow skip fail on broker response exception
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7816:
URL: https://github.com/apache/pinot/pull/7816#discussion_r754764826
##########
File path: pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/ResultSetGroup.java
##########
@@ -53,8 +54,18 @@
}
}
}
-
_executionStats = brokerResponse.getExecutionStats();
+ _exceptions = getPinotClientExceptions(brokerResponse.getExceptions());
+ }
+
+ private static List<PinotClientException> getPinotClientExceptions(JsonNode exceptionsJson) {
Review comment:
Annotate `exceptionsJson` as nullable
##########
File path: pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/Connection.java
##########
@@ -34,14 +35,23 @@
private static final Logger LOGGER = LoggerFactory.getLogger(Connection.class);
private final PinotClientTransport _transport;
private final BrokerSelector _brokerSelector;
+ private final Config _config;
Connection(List<String> brokerList, PinotClientTransport transport) {
- LOGGER.info("Creating connection to broker list {}", brokerList);
- _brokerSelector = new SimpleBrokerSelector(brokerList);
- _transport = transport;
+ this(new Config(), new SimpleBrokerSelector(brokerList), transport);
+ }
+
+ Connection(Config config, List<String> brokerList, PinotClientTransport transport) {
Review comment:
Suggest directly passing in the `Properties`. Don't see much value wrapping it into a custom object. `Properties` is more general and easy to be extended in the future
##########
File path: pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/Connection.java
##########
@@ -222,4 +232,28 @@ public ResultSetGroup get(long timeout, TimeUnit unit)
return new ResultSetGroup(response);
}
}
+
+
+ public static class Config {
+ public static final String FAIL_ON_EXCEPTIONS = "fail_on_exceptions";
Review comment:
We don't usually use `_` in the config key, so suggest using camel case for consistency
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org