You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2023/05/02 08:54:36 UTC
[GitHub] [pinot] Jackie-Jiang opened a new pull request, #10707: Fix a bug in star-tree filter operator which can incorrecly filter documents
Jackie-Jiang opened a new pull request, #10707:
URL: https://github.com/apache/pinot/pull/10707
This bug was introduced in #9667
When the predicate matches all the non-star nodes under a branch, we can use the star-node if it exists and the dimension is not included in the remaining predicate and group-by. In the existing code we update the global remaining predicate too late, and the branches before the update can mistakenly use the star-node even though the dimension is included in the remaining predicate.
This PR fixes that behavior by tracking whether we have added leaf node into the queue, so that we can update the global remaining predicate before traversing any child of that dimension.
Added a hard-coded query in `StarTreeClusterIntegrationTest` to test this behavior. Also print the random seed when test fails to make it much easier to reproduce.
This also fixes #9945
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #10707: Fix a bug in star-tree filter operator which can incorrecly filter documents
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on code in PR #10707:
URL: https://github.com/apache/pinot/pull/10707#discussion_r1182617154
##########
pinot-core/src/main/java/org/apache/pinot/core/startree/operator/StarTreeFilterOperator.java:
##########
@@ -203,19 +203,26 @@ private BaseFilterOperator getFilterOperator() {
@Nullable
private StarTreeResult traverseStarTree() {
MutableRoaringBitmap matchingDocIds = new MutableRoaringBitmap();
- Set<String> globalRemainingPredicateColumns = Collections.emptySet();
- boolean globalRemainingPredicateColumnsSet = false;
+ Set<String> globalRemainingPredicateColumns = null;
StarTree starTree = _starTreeV2.getStarTree();
List<String> dimensionNames = starTree.getDimensionNames();
StarTreeNode starTreeRootNode = starTree.getRoot();
+ // Track whether we have found a leaf node added to the queue. If we have found a leaf node, and traversed to the
+ // level of the leave node, we can set globalRemainingPredicateColumns if not already set because we know the leaf
+ // node won't split further on other predicate columns.
+ boolean findLeafNode = starTreeRootNode.isLeaf();
Review Comment:
`foundLeafNode`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr merged pull request #10707: Fix a bug in star-tree filter operator which can incorrecly filter documents
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr merged PR #10707:
URL: https://github.com/apache/pinot/pull/10707
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #10707: Fix a bug in star-tree filter operator which can incorrecly filter documents
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #10707:
URL: https://github.com/apache/pinot/pull/10707#issuecomment-1531169536
## [Codecov](https://codecov.io/gh/apache/pinot/pull/10707?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#10707](https://codecov.io/gh/apache/pinot/pull/10707?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (92d8ab6) into [master](https://codecov.io/gh/apache/pinot/commit/ab79548e981425033de70103de36c4edcdca2fbf?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ab79548) will **decrease** coverage by `56.44%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #10707 +/- ##
=============================================
- Coverage 70.25% 13.81% -56.44%
+ Complexity 6429 439 -5990
=============================================
Files 2112 2058 -54
Lines 113996 111518 -2478
Branches 17219 16924 -295
=============================================
- Hits 80084 15411 -64673
- Misses 28312 94847 +66535
+ Partials 5600 1260 -4340
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `13.81% <0.00%> (-0.04%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/10707?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...core/startree/operator/StarTreeFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/10707?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zdGFydHJlZS9vcGVyYXRvci9TdGFyVHJlZUZpbHRlck9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-88.16%)` | :arrow_down: |
| [...cal/startree/v2/builder/BaseSingleTreeBuilder.java](https://codecov.io/gh/apache/pinot/pull/10707?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9zdGFydHJlZS92Mi9idWlsZGVyL0Jhc2VTaW5nbGVUcmVlQnVpbGRlci5qYXZh) | `0.00% <0.00%> (-90.00%)` | :arrow_down: |
... and [1666 files with indirect coverage changes](https://codecov.io/gh/apache/pinot/pull/10707/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org