You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "somandal (via GitHub)" <gi...@apache.org> on 2023/03/02 00:07:50 UTC

[GitHub] [pinot] somandal opened a new pull request, #10366: Fix for flaky test ForwardIndexHandlerTest.testEnableDictionaryForMultipleColumns

somandal opened a new pull request, #10366:
URL: https://github.com/apache/pinot/pull/10366

   This PR fixes the flaky test `ForwardIndexHandlerTest.testEnableDictionaryForMultipleColumns` as identified in issue: https://github.com/apache/pinot/issues/10365
   
   Issue: Only forward index enabled columns should be used in this test, while due to choosing a random column a forward index disabled column can get picked sometimes.
   
   Fix: Only pick forward index enabled columns
   
   cc @Jackie-Jiang @vvivekiyer @siddharthteotia 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10366: Fix for flaky test ForwardIndexHandlerTest.testEnableDictionaryForMultipleColumns

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #10366:
URL: https://github.com/apache/pinot/pull/10366#discussion_r1122505726


##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandlerTest.java:
##########
@@ -1204,9 +1204,15 @@ public void testEnableDictionaryForMultipleColumns()
     IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig(null, _tableConfig);
 
     Random rand = new Random();
-    String col1 = _noDictionaryColumns.get(rand.nextInt(_noDictionaryColumns.size()));
+    String col1;
+    do {
+      col1 = _noDictionaryColumns.get(rand.nextInt(_noDictionaryColumns.size()));
+    } while (FORWARD_INDEX_DISABLED_RAW_COLUMNS.contains(col1));
     indexLoadingConfig.getNoDictionaryColumns().remove(col1);
-    String col2 = _noDictionaryColumns.get(rand.nextInt(_noDictionaryColumns.size()));
+    String col2;

Review Comment:
   Should we also ensure `col2` is different from `col1`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] somandal commented on a diff in pull request #10366: Fix for flaky test ForwardIndexHandlerTest.testEnableDictionaryForMultipleColumns

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #10366:
URL: https://github.com/apache/pinot/pull/10366#discussion_r1122537846


##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandlerTest.java:
##########
@@ -1204,9 +1204,15 @@ public void testEnableDictionaryForMultipleColumns()
     IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig(null, _tableConfig);
 
     Random rand = new Random();
-    String col1 = _noDictionaryColumns.get(rand.nextInt(_noDictionaryColumns.size()));
+    String col1;
+    do {
+      col1 = _noDictionaryColumns.get(rand.nextInt(_noDictionaryColumns.size()));
+    } while (FORWARD_INDEX_DISABLED_RAW_COLUMNS.contains(col1));
     indexLoadingConfig.getNoDictionaryColumns().remove(col1);
-    String col2 = _noDictionaryColumns.get(rand.nextInt(_noDictionaryColumns.size()));
+    String col2;

Review Comment:
   good catch, done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter commented on pull request #10366: Fix for flaky test ForwardIndexHandlerTest.testEnableDictionaryForMultipleColumns

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #10366:
URL: https://github.com/apache/pinot/pull/10366#issuecomment-1451104818

   # [Codecov](https://codecov.io/gh/apache/pinot/pull/10366?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#10366](https://codecov.io/gh/apache/pinot/pull/10366?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (259444c) into [master](https://codecov.io/gh/apache/pinot/commit/9c7e771d22a0cbd3391593e9976669fff4d4b442?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9c7e771) will **not change** coverage.
   > The diff coverage is `n/a`.
   
   ```diff
   @@            Coverage Diff            @@
   ##             master   #10366   +/-   ##
   =========================================
     Coverage     13.76%   13.76%           
     Complexity      231      231           
   =========================================
     Files          1979     1979           
     Lines        107729   107729           
     Branches      16457    16457           
   =========================================
     Hits          14831    14831           
     Misses        91717    91717           
     Partials       1181     1181           
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests2 | `13.76% <ø> (ø)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/10366?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...ntroller/helix/core/minion/TaskMetricsEmitter.java](https://codecov.io/gh/apache/pinot/pull/10366?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL21pbmlvbi9UYXNrTWV0cmljc0VtaXR0ZXIuamF2YQ==) | `86.53% <0.00%> (-1.93%)` | :arrow_down: |
   | [...lix/core/minion/PinotHelixTaskResourceManager.java](https://codecov.io/gh/apache/pinot/pull/10366?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL21pbmlvbi9QaW5vdEhlbGl4VGFza1Jlc291cmNlTWFuYWdlci5qYXZh) | `28.63% <0.00%> (-1.12%)` | :arrow_down: |
   | [...ntroller/helix/core/PinotHelixResourceManager.java](https://codecov.io/gh/apache/pinot/pull/10366?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL1Bpbm90SGVsaXhSZXNvdXJjZU1hbmFnZXIuamF2YQ==) | `62.34% <0.00%> (-0.18%)` | :arrow_down: |
   | [...controller/helix/core/minion/PinotTaskManager.java](https://codecov.io/gh/apache/pinot/pull/10366?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL21pbmlvbi9QaW5vdFRhc2tNYW5hZ2VyLmphdmE=) | `38.72% <0.00%> (+1.44%)` | :arrow_up: |
   | [...lix/core/realtime/PinotRealtimeSegmentManager.java](https://codecov.io/gh/apache/pinot/pull/10366?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL3JlYWx0aW1lL1Bpbm90UmVhbHRpbWVTZWdtZW50TWFuYWdlci5qYXZh) | `77.00% <0.00%> (+2.50%)` | :arrow_up: |
   
   :mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang merged pull request #10366: Fix for flaky test ForwardIndexHandlerTest.testEnableDictionaryForMultipleColumns

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang merged PR #10366:
URL: https://github.com/apache/pinot/pull/10366


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org