You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "anandheritage (via GitHub)" <gi...@apache.org> on 2024/09/08 15:11:19 UTC
[PR] Fix for untag in server [pinot]
anandheritage opened a new pull request, #13954:
URL: https://github.com/apache/pinot/pull/13954
Instructions:
1. The PR has to be tagged with at least one of the following labels (*):
1. `feature`
2. `bugfix`
3. `performance`
4. `ui`
5. `backward-incompat`
6. `release-notes` (**)
2. Remove these instructions before publishing the PR.
(*) Other labels to consider:
- `testing`
- `dependencies`
- `docker`
- `kubernetes`
- `observability`
- `security`
- `code-style`
- `extension-point`
- `refactor`
- `cleanup`
(**) Use `release-notes` label for scenarios like:
- New configuration options
- Deprecation of configurations
- Signature changes to public methods/interfaces
- New plugins added or old plugins removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix handling of empty server tags in controller's availability check [pinot]
Posted by "anandheritage (via GitHub)" <gi...@apache.org>.
anandheritage commented on code in PR #13954:
URL: https://github.com/apache/pinot/pull/13954#discussion_r1750631118
##########
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManagerStatelessTest.java:
##########
@@ -1146,6 +1147,20 @@ public void testSegmentReplacementRegular()
segmentLineage = SegmentLineageAccessHelper.getSegmentLineage(_propertyStore, OFFLINE_TABLE_NAME);
assertNull(segmentLineage);
}
+ @Test
+ public void testHandleEmptyServerTags()
+ throws Exception {
+ // Create an instance with no tags
+ Instance instance = new Instance("localhost", NUM_SERVER_INSTANCES , InstanceType.SERVER,
+ Collections.emptyList(), null, 0, 12345, 0, 0, false);
+
+ _helixResourceManager.addInstance(instance,false);
+ addFakeServerInstanceToAutoJoinHelixClusterWithEmptyTag(SERVER_NAME_UNTAGGED, false);
+
+ // Verify that the server is considered untagged
+ List<String> untaggedServers = _helixResourceManager.getOnlineUnTaggedServerInstanceList();
+ assertTrue(untaggedServers.contains(SERVER_NAME_UNTAGGED), "Server with empty tags should be considered untagged");
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix handling of empty server tags in controller's availability check [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #13954:
URL: https://github.com/apache/pinot/pull/13954#discussion_r1750754674
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java:
##########
@@ -3439,7 +3439,11 @@ public List<String> getOnlineUnTaggedBrokerInstanceList() {
*/
public List<String> getOnlineUnTaggedServerInstanceList() {
List<String> instanceList = HelixHelper.getInstancesWithTag(_helixZkManager, Helix.UNTAGGED_SERVER_INSTANCE);
+ List<String> instanceListWithoutTags = HelixHelper.getInstancesWithoutTag(_helixZkManager);
Review Comment:
Suggest adding a method to `HelixHelper` to directly get untagged server instances. Currently we read all instance configs twice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix handling of empty server tags in controller's availability check [pinot]
Posted by "mayankshriv (via GitHub)" <gi...@apache.org>.
mayankshriv commented on code in PR #13954:
URL: https://github.com/apache/pinot/pull/13954#discussion_r1750606115
##########
pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java:
##########
@@ -542,6 +549,12 @@ public static List<String> getInstancesWithTag(List<InstanceConfig> instanceConf
return instancesWithTag.stream().map(InstanceConfig::getInstanceName).collect(Collectors.toList());
}
+ public static List<String> getInstancesWithoutTag(List<InstanceConfig> instanceConfigs) {
+ List<InstanceConfig> instancesWithTag = getInstancesConfigsWithoutTag(instanceConfigs);
Review Comment:
Rename variable to `instanceWIthoutTags`?
##########
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManagerStatelessTest.java:
##########
@@ -1146,6 +1147,20 @@ public void testSegmentReplacementRegular()
segmentLineage = SegmentLineageAccessHelper.getSegmentLineage(_propertyStore, OFFLINE_TABLE_NAME);
assertNull(segmentLineage);
}
+ @Test
+ public void testHandleEmptyServerTags()
+ throws Exception {
+ // Create an instance with no tags
+ Instance instance = new Instance("localhost", NUM_SERVER_INSTANCES , InstanceType.SERVER,
+ Collections.emptyList(), null, 0, 12345, 0, 0, false);
+
+ _helixResourceManager.addInstance(instance,false);
+ addFakeServerInstanceToAutoJoinHelixClusterWithEmptyTag(SERVER_NAME_UNTAGGED, false);
+
+ // Verify that the server is considered untagged
+ List<String> untaggedServers = _helixResourceManager.getOnlineUnTaggedServerInstanceList();
+ assertTrue(untaggedServers.contains(SERVER_NAME_UNTAGGED), "Server with empty tags should be considered untagged");
Review Comment:
Would be good to add the negative test as well - Servers with tag should not be considered untagged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix handling of empty server tags in controller's availability check [pinot]
Posted by "anandheritage (via GitHub)" <gi...@apache.org>.
anandheritage commented on code in PR #13954:
URL: https://github.com/apache/pinot/pull/13954#discussion_r1750623794
##########
pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java:
##########
@@ -542,6 +549,12 @@ public static List<String> getInstancesWithTag(List<InstanceConfig> instanceConf
return instancesWithTag.stream().map(InstanceConfig::getInstanceName).collect(Collectors.toList());
}
+ public static List<String> getInstancesWithoutTag(List<InstanceConfig> instanceConfigs) {
+ List<InstanceConfig> instancesWithTag = getInstancesConfigsWithoutTag(instanceConfigs);
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Handling of empty server tags in controller's availability check [pinot]
Posted by "anandheritage (via GitHub)" <gi...@apache.org>.
anandheritage commented on PR #13954:
URL: https://github.com/apache/pinot/pull/13954#issuecomment-2341278621




Looked at the failed test cases - seems like flanky ones , doesn't look related to the changes too - In local same tests are working fine.
Let me know if @mayankshriv @Jackie-Jiang if you folks think otherwise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix handling of empty server tags in controller's availability check [pinot]
Posted by "anandheritage (via GitHub)" <gi...@apache.org>.
anandheritage commented on code in PR #13954:
URL: https://github.com/apache/pinot/pull/13954#discussion_r1751243342
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java:
##########
@@ -3439,7 +3439,11 @@ public List<String> getOnlineUnTaggedBrokerInstanceList() {
*/
public List<String> getOnlineUnTaggedServerInstanceList() {
List<String> instanceList = HelixHelper.getInstancesWithTag(_helixZkManager, Helix.UNTAGGED_SERVER_INSTANCE);
+ List<String> instanceListWithoutTags = HelixHelper.getInstancesWithoutTag(_helixZkManager);
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix handling of empty server tags in controller's availability check [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #13954:
URL: https://github.com/apache/pinot/pull/13954#issuecomment-2336768625
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13954?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `0%` with `11 lines` in your changes missing coverage. Please review.
> Project coverage is 0.00%. Comparing base [(`59551e4`)](https://app.codecov.io/gh/apache/pinot/commit/59551e45224f1535c4863fd577622b37366ccc97?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`839c84d`)](https://app.codecov.io/gh/apache/pinot/commit/839c84d7b7f7ce11a8aaba6c132a55d584387617?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 1001 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/pinot/pull/13954?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...g/apache/pinot/common/utils/helix/HelixHelper.java](https://app.codecov.io/gh/apache/pinot/pull/13954?src=pr&el=tree&filepath=pinot-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcommon%2Futils%2Fhelix%2FHelixHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaGVsaXgvSGVsaXhIZWxwZXIuamF2YQ==) | 0.00% | [9 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/13954?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ntroller/helix/core/PinotHelixResourceManager.java](https://app.codecov.io/gh/apache/pinot/pull/13954?src=pr&el=tree&filepath=pinot-controller%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcontroller%2Fhelix%2Fcore%2FPinotHelixResourceManager.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL1Bpbm90SGVsaXhSZXNvdXJjZU1hbmFnZXIuamF2YQ==) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/13954?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (59551e4) and HEAD (839c84d). Click for more details.
>
> <details><summary>HEAD has 53 uploads less than BASE</summary>
>
>| Flag | BASE (59551e4) | HEAD (839c84d) |
>|------|------|------|
>|integration|7|1|
>|integration2|3|1|
>|temurin|12|1|
>|java-21|7|1|
>|skip-bytebuffers-true|3|0|
>|skip-bytebuffers-false|7|1|
>|unittests|5|0|
>|unittests1|2|0|
>|java-11|5|0|
>|unittests2|3|0|
>|integration1|2|0|
>|custom-integration1|2|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #13954 +/- ##
=============================================
- Coverage 61.75% 0.00% -61.76%
=============================================
Files 2436 2512 +76
Lines 133233 138976 +5743
Branches 20636 21367 +731
=============================================
- Hits 82274 0 -82274
- Misses 44911 138976 +94065
+ Partials 6048 0 -6048
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-0.01%)` | :arrow_down: |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (ø)` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-61.63%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-61.75%)` | :arrow_down: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-61.76%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/13954/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/13954?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Handling of empty server tags in controller's availability check [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #13954:
URL: https://github.com/apache/pinot/pull/13954#discussion_r1752532896
##########
pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java:
##########
@@ -542,6 +549,12 @@ public static List<String> getInstancesWithTag(List<InstanceConfig> instanceConf
return instancesWithTag.stream().map(InstanceConfig::getInstanceName).collect(Collectors.toList());
}
+ public static List<String> getInstancesWithoutTag(List<InstanceConfig> instanceConfigs, String tag) {
+ List<InstanceConfig> instancesWithoutTag = getInstancesConfigsWithoutTag(instanceConfigs, tag);
+ return instancesWithoutTag.stream().map(InstanceConfig::getInstanceName).collect(Collectors.toList());
+ }
+
Review Comment:
(minor) Remove extra empty line
##########
pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java:
##########
@@ -552,6 +565,17 @@ public static List<InstanceConfig> getInstancesConfigsWithTag(List<InstanceConfi
return instancesWithTag;
}
+ public static List<InstanceConfig> getInstancesConfigsWithoutTag(List<InstanceConfig> instanceConfigs, String tag) {
Review Comment:
Suggest changing `tag` to `defaultTag` and add some javadoc explaining this is the tag representing that there is no tag associated with the instance. Same for other methods
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Handling of empty server tags in controller's availability check [pinot]
Posted by "mayankshriv (via GitHub)" <gi...@apache.org>.
mayankshriv merged PR #13954:
URL: https://github.com/apache/pinot/pull/13954
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org