You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2023/01/10 19:08:30 UTC
[GitHub] [pinot] vvivekiyer opened a new pull request, #10093: Fix flaky ForwardIndexHandlerTest
vvivekiyer opened a new pull request, #10093:
URL: https://github.com/apache/pinot/pull/10093
This PR fixes the flaky ForwardIndexHandlerTest encountered in #9831 and https://github.com/apache/pinot/issues/10012
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang merged pull request #10093: Fix flaky ForwardIndexHandlerTest
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang merged PR #10093:
URL: https://github.com/apache/pinot/pull/10093
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #10093: Fix flaky ForwardIndexHandlerTest
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #10093:
URL: https://github.com/apache/pinot/pull/10093#issuecomment-1377769542
# [Codecov](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#10093](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4799e01) into [master](https://codecov.io/gh/apache/pinot/commit/f9c8983a1c42ed97a61a2778ba91d014ee4171ab?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f9c8983) will **decrease** coverage by `36.37%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #10093 +/- ##
=============================================
- Coverage 68.58% 32.20% -36.38%
+ Complexity 5685 191 -5494
=============================================
Files 1991 1995 +4
Lines 107641 107939 +298
Branches 16362 16400 +38
=============================================
- Hits 73821 34758 -39063
- Misses 28573 70100 +41527
+ Partials 5247 3081 -2166
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `24.72% <ø> (+0.09%)` | :arrow_up: |
| unittests1 | `?` | |
| unittests2 | `13.69% <ø> (+0.06%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ain/java/org/apache/pinot/spi/utils/LoopUtils.java](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdXRpbHMvTG9vcFV0aWxzLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...in/java/org/apache/pinot/spi/utils/BytesUtils.java](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdXRpbHMvQnl0ZXNVdGlscy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...java/org/apache/pinot/spi/trace/BaseRecording.java](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvQmFzZVJlY29yZGluZy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...java/org/apache/pinot/spi/trace/NoOpRecording.java](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdHJhY2UvTm9PcFJlY29yZGluZy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ava/org/apache/pinot/spi/config/table/FSTType.java](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvY29uZmlnL3RhYmxlL0ZTVFR5cGUuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ava/org/apache/pinot/spi/config/user/RoleType.java](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvY29uZmlnL3VzZXIvUm9sZVR5cGUuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ava/org/apache/pinot/spi/data/MetricFieldSpec.java](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvZGF0YS9NZXRyaWNGaWVsZFNwZWMuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ava/org/apache/pinot/spi/stream/StreamMessage.java](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvc3RyZWFtL1N0cmVhbU1lc3NhZ2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...java/org/apache/pinot/common/tier/TierFactory.java](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/pinot/spi/config/table/TableType.java](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvY29uZmlnL3RhYmxlL1RhYmxlVHlwZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [1166 more](https://codecov.io/gh/apache/pinot/pull/10093?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] vvivekiyer commented on a diff in pull request #10093: Fix flaky ForwardIndexHandlerTest
Posted by GitBox <gi...@apache.org>.
vvivekiyer commented on code in PR #10093:
URL: https://github.com/apache/pinot/pull/10093#discussion_r1066362108
##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandlerTest.java:
##########
@@ -1374,7 +1373,7 @@ public void testDisableForwardIndexForSingleRawColumn()
// This value is based on the rows in createTestData().
dictionaryElementSize = 7;
} else if (dataType == FieldSpec.DataType.BIG_DECIMAL) {
- dictionaryElementSize = 11;
+ dictionaryElementSize = 4;
Review Comment:
The existing code uses a random BASE_BIG_DECIMAL. Depending on the random value, the dictionaryElementSize varies.
In the `validateMetadata()`, we check if the dictionaryElementSize remains the same
`assertEquals(columnMetadata.getColumnMaxLength(), dictionaryElementSize, column);`
This was causing the flakiness. Now we make it deterministic by not using BASE_BIG_DECIMAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] vvivekiyer commented on a diff in pull request #10093: Fix flaky ForwardIndexHandlerTest
Posted by GitBox <gi...@apache.org>.
vvivekiyer commented on code in PR #10093:
URL: https://github.com/apache/pinot/pull/10093#discussion_r1066362108
##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandlerTest.java:
##########
@@ -1374,7 +1373,7 @@ public void testDisableForwardIndexForSingleRawColumn()
// This value is based on the rows in createTestData().
dictionaryElementSize = 7;
} else if (dataType == FieldSpec.DataType.BIG_DECIMAL) {
- dictionaryElementSize = 11;
+ dictionaryElementSize = 4;
Review Comment:
The existing code uses a random BASE_BIG_DECIMAL. Depending on the random value, the dictionaryElementSize varies.
In the `validateMetadata()` function, we check if the dictionaryElementSize remains the same
`assertEquals(columnMetadata.getColumnMaxLength(), dictionaryElementSize, column);`
This was causing the flakiness. Now we make it deterministic by not using BASE_BIG_DECIMAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10093: Fix flaky ForwardIndexHandlerTest
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on code in PR #10093:
URL: https://github.com/apache/pinot/pull/10093#discussion_r1066353753
##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandlerTest.java:
##########
@@ -1374,7 +1373,7 @@ public void testDisableForwardIndexForSingleRawColumn()
// This value is based on the rows in createTestData().
dictionaryElementSize = 7;
} else if (dataType == FieldSpec.DataType.BIG_DECIMAL) {
- dictionaryElementSize = 11;
+ dictionaryElementSize = 4;
Review Comment:
Not sure I follow the root cause ? Can you elaborate why changing this fixes flakeyness ?
##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandlerTest.java:
##########
@@ -1374,7 +1373,7 @@ public void testDisableForwardIndexForSingleRawColumn()
// This value is based on the rows in createTestData().
dictionaryElementSize = 7;
} else if (dataType == FieldSpec.DataType.BIG_DECIMAL) {
- dictionaryElementSize = 11;
+ dictionaryElementSize = 4;
Review Comment:
Not sure I follow the root cause. Can you elaborate why changing this fixes flakeyness ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] vvivekiyer commented on a diff in pull request #10093: Fix flaky ForwardIndexHandlerTest
Posted by GitBox <gi...@apache.org>.
vvivekiyer commented on code in PR #10093:
URL: https://github.com/apache/pinot/pull/10093#discussion_r1066362108
##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/ForwardIndexHandlerTest.java:
##########
@@ -1374,7 +1373,7 @@ public void testDisableForwardIndexForSingleRawColumn()
// This value is based on the rows in createTestData().
dictionaryElementSize = 7;
} else if (dataType == FieldSpec.DataType.BIG_DECIMAL) {
- dictionaryElementSize = 11;
+ dictionaryElementSize = 4;
Review Comment:
The existing code uses a random BASE_BIG_DECIMAL. Depending on the random value used in BASE_BIG_DECIMAL, the `dictionaryElementSize` varies.
In the `validateMetadata()` function, we check if the dictionaryElementSize remains the same
`assertEquals(columnMetadata.getColumnMaxLength(), dictionaryElementSize, column);`
This was causing the flakiness. Now we make it deterministic by not using BASE_BIG_DECIMAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org