You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2023/12/19 04:06:40 UTC
[PR] Support initializing broker tags from config [pinot]
Jackie-Jiang opened a new pull request, #12175:
URL: https://github.com/apache/pinot/pull/12175
Address #11755
Support initializing broker tags from config and automatically updating broker resource when broker joins the cluster for the first time.
## Configuration
`pinot.broker.instance.tags` (comma-separated): Broker tags to set when broker joins the cluster for the first time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Support initializing broker tags from config [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #12175:
URL: https://github.com/apache/pinot/pull/12175#issuecomment-1862111517
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12175?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: `13 lines` in your changes are missing coverage. Please review.
> Comparison is base [(`c085f04`)](https://app.codecov.io/gh/apache/pinot/commit/c085f0497d200867eaaaf82ddff84b74db97ac98?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 61.64% compared to head [(`bfefbcf`)](https://app.codecov.io/gh/apache/pinot/pull/12175?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 34.80%.
| [Files](https://app.codecov.io/gh/apache/pinot/pull/12175?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...e/pinot/broker/broker/helix/BaseBrokerStarter.java](https://app.codecov.io/gh/apache/pinot/pull/12175?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jhc2VCcm9rZXJTdGFydGVyLmphdmE=) | 0.00% | [13 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/12175?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #12175 +/- ##
=============================================
- Coverage 61.64% 34.80% -26.84%
+ Complexity 1152 6 -1146
=============================================
Files 2407 2331 -76
Lines 130914 127172 -3742
Branches 20225 19669 -556
=============================================
- Hits 80697 44263 -36434
- Misses 44326 79761 +35435
+ Partials 5891 3148 -2743
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (ø)` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (ø)` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (ø)` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.80% <0.00%> (-26.71%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `<0.01% <0.00%> (-61.61%)` | :arrow_down: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `46.50% <ø> (-14.98%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.80% <0.00%> (-26.84%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `46.50% <ø> (-15.13%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `46.50% <ø> (-0.17%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/12175/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/12175?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Support initializing broker tags from config [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on code in PR #12175:
URL: https://github.com/apache/pinot/pull/12175#discussion_r1431997929
##########
pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/BaseBrokerStarter.java:
##########
@@ -439,17 +440,26 @@ private void updateInstanceConfigAndBrokerResourceIfNeeded() {
}
updated |= HelixHelper.removeDisabledPartitions(instanceConfig);
boolean shouldUpdateBrokerResource = false;
- String brokerTag = null;
List<String> instanceTags = instanceConfig.getTags();
if (instanceTags.isEmpty()) {
// This is a new broker (first time joining the cluster)
if (ZKMetadataProvider.getClusterTenantIsolationEnabled(_propertyStore)) {
- brokerTag = TagNameUtils.getBrokerTagForTenant(null);
+ instanceConfig.addTag(TagNameUtils.getBrokerTagForTenant(null));
shouldUpdateBrokerResource = true;
} else {
- brokerTag = Helix.UNTAGGED_BROKER_INSTANCE;
+ String instanceTagsConfig = _brokerConf.getProperty(Broker.CONFIG_OF_BROKER_INSTANCE_TAGS);
+ if (StringUtils.isNotEmpty(instanceTagsConfig)) {
+ for (String instanceTag : StringUtils.split(instanceTagsConfig, ',')) {
+ Preconditions.checkArgument(TagNameUtils.isBrokerTag(instanceTag), "Illegal broker instance tag: %s",
+ instanceTag);
+ instanceConfig.addTag(instanceTag);
Review Comment:
Considering this comes from a config file, shall we do `instanceTag.trim()`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Support initializing broker tags from config [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #12175:
URL: https://github.com/apache/pinot/pull/12175#discussion_r1432006745
##########
pinot-broker/src/main/java/org/apache/pinot/broker/broker/helix/BaseBrokerStarter.java:
##########
@@ -439,17 +440,26 @@ private void updateInstanceConfigAndBrokerResourceIfNeeded() {
}
updated |= HelixHelper.removeDisabledPartitions(instanceConfig);
boolean shouldUpdateBrokerResource = false;
- String brokerTag = null;
List<String> instanceTags = instanceConfig.getTags();
if (instanceTags.isEmpty()) {
// This is a new broker (first time joining the cluster)
if (ZKMetadataProvider.getClusterTenantIsolationEnabled(_propertyStore)) {
- brokerTag = TagNameUtils.getBrokerTagForTenant(null);
+ instanceConfig.addTag(TagNameUtils.getBrokerTagForTenant(null));
shouldUpdateBrokerResource = true;
} else {
- brokerTag = Helix.UNTAGGED_BROKER_INSTANCE;
+ String instanceTagsConfig = _brokerConf.getProperty(Broker.CONFIG_OF_BROKER_INSTANCE_TAGS);
+ if (StringUtils.isNotEmpty(instanceTagsConfig)) {
+ for (String instanceTag : StringUtils.split(instanceTagsConfig, ',')) {
+ Preconditions.checkArgument(TagNameUtils.isBrokerTag(instanceTag), "Illegal broker instance tag: %s",
+ instanceTag);
+ instanceConfig.addTag(instanceTag);
Review Comment:
Initially I did add `trim()`, but then decided to remove it to keep it consistent with `PinotHelixResourceManager.updateInstanceTags()`.
If we find people making mistakes frequently, we can consider changing both to use `trim()`. Currently we don't have a check that tag cannot start/end with space, so this might cause backward-incompatible
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Support initializing broker tags from config [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang merged PR #12175:
URL: https://github.com/apache/pinot/pull/12175
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org