You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "klsince (via GitHub)" <gi...@apache.org> on 2023/05/02 00:22:05 UTC
[GitHub] [pinot] klsince opened a new pull request, #10705: [WIP] check if serverTags as configured in tierConfigs are valid
klsince opened a new pull request, #10705:
URL: https://github.com/apache/pinot/pull/10705
Sanity check on serverTags as set in tierConfigs, to make sure they are valid
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] snleee commented on a diff in pull request #10705: Check if serverTags as configured in tierConfigs are valid
Posted by "snleee (via GitHub)" <gi...@apache.org>.
snleee commented on code in PR #10705:
URL: https://github.com/apache/pinot/pull/10705#discussion_r1182190015
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java:
##########
@@ -1639,6 +1639,17 @@ void validateTableTenantConfig(TableConfig tableConfig) {
"Failed to find instances with tag: " + tag + " for table: " + tableNameWithType);
}
}
+ // Check if serverTags as configured in tierConfigs are valid.
+ List<TierConfig> tierConfigList = tableConfig.getTierConfigsList();
+ if (CollectionUtils.isNotEmpty(tierConfigList)) {
+ for (TierConfig tierConfig : tierConfigList) {
+ if (getInstancesWithTag(tierConfig.getServerTag()).isEmpty()) {
Review Comment:
Does this simply fetch the list of servers based on the tag or it will filter out live instances?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] klsince commented on a diff in pull request #10705: Check if serverTags as configured in tierConfigs are valid
Posted by "klsince (via GitHub)" <gi...@apache.org>.
klsince commented on code in PR #10705:
URL: https://github.com/apache/pinot/pull/10705#discussion_r1182719683
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java:
##########
@@ -1639,6 +1639,17 @@ void validateTableTenantConfig(TableConfig tableConfig) {
"Failed to find instances with tag: " + tag + " for table: " + tableNameWithType);
}
}
+ // Check if serverTags as configured in tierConfigs are valid.
+ List<TierConfig> tierConfigList = tableConfig.getTierConfigsList();
+ if (CollectionUtils.isNotEmpty(tierConfigList)) {
+ for (TierConfig tierConfig : tierConfigList) {
+ if (getInstancesWithTag(tierConfig.getServerTag()).isEmpty()) {
Review Comment:
I think it gets all instances for the check. btw, this is the same check done for table tenant L1640.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr merged pull request #10705: Check if serverTags as configured in tierConfigs are valid
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr merged PR #10705:
URL: https://github.com/apache/pinot/pull/10705
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #10705: Check if serverTags as configured in tierConfigs are valid
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on code in PR #10705:
URL: https://github.com/apache/pinot/pull/10705#discussion_r1182900917
##########
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManagerStatelessTest.java:
##########
@@ -540,6 +540,19 @@ public void testValidateTenantConfig() {
// Valid tenant config without tagOverrideConfig
_helixResourceManager.validateTableTenantConfig(offlineTableConfig);
+ // Invalid serverTag in tierConfigs
+ TierConfig tierConfig = new TierConfig("myTier", TierFactory.TIME_SEGMENT_SELECTOR_TYPE, "10d", null,
+ TierFactory.PINOT_SERVER_STORAGE_TYPE, "Unknown_OFFLINE", null, null);
Review Comment:
add a test for null? (looks like TierConfig json property server tag is nullable)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] klsince commented on a diff in pull request #10705: Check if serverTags as configured in tierConfigs are valid
Posted by "klsince (via GitHub)" <gi...@apache.org>.
klsince commented on code in PR #10705:
URL: https://github.com/apache/pinot/pull/10705#discussion_r1182999493
##########
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManagerStatelessTest.java:
##########
@@ -540,6 +540,19 @@ public void testValidateTenantConfig() {
// Valid tenant config without tagOverrideConfig
_helixResourceManager.validateTableTenantConfig(offlineTableConfig);
+ // Invalid serverTag in tierConfigs
+ TierConfig tierConfig = new TierConfig("myTier", TierFactory.TIME_SEGMENT_SELECTOR_TYPE, "10d", null,
+ TierFactory.PINOT_SERVER_STORAGE_TYPE, "Unknown_OFFLINE", null, null);
Review Comment:
good point. added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #10705: [WIP] check if serverTags as configured in tierConfigs are valid
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #10705:
URL: https://github.com/apache/pinot/pull/10705#issuecomment-1530755297
## [Codecov](https://codecov.io/gh/apache/pinot/pull/10705?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#10705](https://codecov.io/gh/apache/pinot/pull/10705?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a078fca) into [master](https://codecov.io/gh/apache/pinot/commit/ab79548e981425033de70103de36c4edcdca2fbf?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ab79548) will **increase** coverage by `0.03%`.
> The diff coverage is `50.00%`.
```diff
@@ Coverage Diff @@
## master #10705 +/- ##
============================================
+ Coverage 70.25% 70.28% +0.03%
+ Complexity 6429 5643 -786
============================================
Files 2112 2112
Lines 113996 114004 +8
Branches 17219 17222 +3
============================================
+ Hits 80084 80130 +46
+ Misses 28312 28266 -46
- Partials 5600 5608 +8
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `24.32% <12.50%> (+<0.01%)` | :arrow_up: |
| integration2 | `24.20% <12.50%> (+0.19%)` | :arrow_up: |
| unittests1 | `67.80% <ø> (-0.01%)` | :arrow_down: |
| unittests2 | `13.82% <50.00%> (-0.03%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/10705?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ntroller/helix/core/PinotHelixResourceManager.java](https://codecov.io/gh/apache/pinot/pull/10705?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL1Bpbm90SGVsaXhSZXNvdXJjZU1hbmFnZXIuamF2YQ==) | `70.99% <50.00%> (-0.04%)` | :arrow_down: |
... and [44 files with indirect coverage changes](https://codecov.io/gh/apache/pinot/pull/10705/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org