You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2024/10/15 00:41:14 UTC
[PR] [WIP] Support preceding and following in WINDOW [pinot]
Jackie-Jiang opened a new pull request, #14233:
URL: https://github.com/apache/pinot/pull/14233
Support the following functionalities in WINDOW function:
- ROWS with bounded preceding/following
- IGNORE NULLS in aggregates
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [WIP] Support preceding and following in WINDOW [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #14233:
URL: https://github.com/apache/pinot/pull/14233#discussion_r1802473173
##########
pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java:
##########
@@ -234,8 +283,13 @@ private void validateWindowFrames(Window.Group windowGroup) {
}
}
- private boolean isRowsOnlyAggregationCallType(ImmutableList<Window.RexWinAggCall> aggCalls) {
- return aggCalls.stream().anyMatch(aggCall -> aggCall.getKind().equals(SqlKind.ROW_NUMBER));
+ private boolean hasRankBasedAgg(ImmutableList<Window.RexWinAggCall> aggCalls) {
+ for (Window.RexWinAggCall aggCall : aggCalls) {
+ if (RANK_BASED_FUNCTION_KIND.contains(aggCall.getKind())) {
+ return true;
+ }
+ }
+ return false;
Review Comment:
Good call. Let's add a test to ensure it doesn't get planned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [WIP] Support preceding and following in WINDOW [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #14233:
URL: https://github.com/apache/pinot/pull/14233#discussion_r1804044667
##########
pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java:
##########
@@ -209,23 +244,37 @@ private void validateWindows(Window window) {
}
private void validateWindowAggCallsSupported(Window.Group windowGroup) {
- for (int i = 0; i < windowGroup.aggCalls.size(); i++) {
- Window.RexWinAggCall aggCall = windowGroup.aggCalls.get(i);
+ for (Window.RexWinAggCall aggCall : windowGroup.aggCalls) {
SqlKind aggKind = aggCall.getKind();
Preconditions.checkState(SUPPORTED_WINDOW_FUNCTION_KIND.contains(aggKind),
String.format("Unsupported Window function kind: %s. Only aggregation functions are supported!", aggKind));
}
}
private void validateWindowFrames(Window.Group windowGroup) {
- // Has ROWS only aggregation call kind (e.g. ROW_NUMBER)?
- boolean isRowsOnlyTypeAggregateCall = isRowsOnlyAggregationCallType(windowGroup.aggCalls);
+ // Has rank based aggregation call kind (e.g. ROW_NUMBER, RANK, DENSE_RANK)
+ boolean hasRankBasedAgg = hasRankBasedAgg(windowGroup.aggCalls);
+
+ // FOLLOWING lower bound or PRECEDING upper bound is not allowed, and won't reach here
Review Comment:
`UNBOUNDED PRECEDING` is not allowed for upper frame boundary and `UNBOUNDED FOLLOWING` is not allowed for lower frame boundary. However, `n PRECEDING` is allowed for upper frame boundary and `n FOLLOWING` is allowed for lower frame boundary (not at the same time though). This behavior is similar in both Calcite and Postgres.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [WIP] Support preceding and following in WINDOW [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #14233:
URL: https://github.com/apache/pinot/pull/14233#discussion_r1802367928
##########
pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java:
##########
@@ -209,23 +244,37 @@ private void validateWindows(Window window) {
}
private void validateWindowAggCallsSupported(Window.Group windowGroup) {
- for (int i = 0; i < windowGroup.aggCalls.size(); i++) {
- Window.RexWinAggCall aggCall = windowGroup.aggCalls.get(i);
+ for (Window.RexWinAggCall aggCall : windowGroup.aggCalls) {
SqlKind aggKind = aggCall.getKind();
Preconditions.checkState(SUPPORTED_WINDOW_FUNCTION_KIND.contains(aggKind),
String.format("Unsupported Window function kind: %s. Only aggregation functions are supported!", aggKind));
}
}
private void validateWindowFrames(Window.Group windowGroup) {
- // Has ROWS only aggregation call kind (e.g. ROW_NUMBER)?
- boolean isRowsOnlyTypeAggregateCall = isRowsOnlyAggregationCallType(windowGroup.aggCalls);
+ // Has rank based aggregation call kind (e.g. ROW_NUMBER, RANK, DENSE_RANK)
+ boolean hasRankBasedAgg = hasRankBasedAgg(windowGroup.aggCalls);
+
+ // FOLLOWING lower bound or PRECEDING upper bound is not allowed, and won't reach here
+ RexWindowBound lowerBound = windowGroup.lowerBound;
+ assert !lowerBound.isFollowing();
+ RexWindowBound upperBound = windowGroup.upperBound;
+ assert !upperBound.isPreceding();
+ boolean hasBound = (lowerBound.isPreceding() && !lowerBound.isUnbounded()) || (upperBound.isFollowing()
+ && !upperBound.isUnbounded());
+
+ if (hasRankBasedAgg) {
+ Preconditions.checkState(!hasBound, "Window frame is not supported for ROW_NUMBER, RANK, DENSE_RANK functions");
Review Comment:
Postgres seems to allow any window frame for these functions, but the result is the same regardless of the defined window frame (i.e., following the behavior of `ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING`). Although I think the behavior here of imposing the unbounded window frame condition is probably better to avoid user confusion.
##########
pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java:
##########
@@ -209,23 +244,37 @@ private void validateWindows(Window window) {
}
private void validateWindowAggCallsSupported(Window.Group windowGroup) {
- for (int i = 0; i < windowGroup.aggCalls.size(); i++) {
- Window.RexWinAggCall aggCall = windowGroup.aggCalls.get(i);
+ for (Window.RexWinAggCall aggCall : windowGroup.aggCalls) {
SqlKind aggKind = aggCall.getKind();
Preconditions.checkState(SUPPORTED_WINDOW_FUNCTION_KIND.contains(aggKind),
String.format("Unsupported Window function kind: %s. Only aggregation functions are supported!", aggKind));
}
}
private void validateWindowFrames(Window.Group windowGroup) {
- // Has ROWS only aggregation call kind (e.g. ROW_NUMBER)?
- boolean isRowsOnlyTypeAggregateCall = isRowsOnlyAggregationCallType(windowGroup.aggCalls);
+ // Has rank based aggregation call kind (e.g. ROW_NUMBER, RANK, DENSE_RANK)
+ boolean hasRankBasedAgg = hasRankBasedAgg(windowGroup.aggCalls);
+
+ // FOLLOWING lower bound or PRECEDING upper bound is not allowed, and won't reach here
+ RexWindowBound lowerBound = windowGroup.lowerBound;
+ assert !lowerBound.isFollowing();
+ RexWindowBound upperBound = windowGroup.upperBound;
+ assert !upperBound.isPreceding();
+ boolean hasBound = (lowerBound.isPreceding() && !lowerBound.isUnbounded()) || (upperBound.isFollowing()
+ && !upperBound.isUnbounded());
+
+ if (hasRankBasedAgg) {
+ Preconditions.checkState(!hasBound, "Window frame is not supported for ROW_NUMBER, RANK, DENSE_RANK functions");
+ }
+ if (!windowGroup.isRows) {
+ Preconditions.checkState(!hasBound, "Bounded RANGE window frame is currently not supported");
+ }
// For Phase 1 only the default frame is supported
- Preconditions.checkState(!windowGroup.isRows || isRowsOnlyTypeAggregateCall,
+ Preconditions.checkState(!windowGroup.isRows || hasRankBasedAgg,
Review Comment:
If it is a rank based aggregation, we already check above that the window frame is unbounded (whether range or rows). So this condition of `!windowGroup.isRows || hasRankBasedAgg` doesn't really seem to make sense here unless I'm missing something?
##########
pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java:
##########
@@ -234,8 +283,13 @@ private void validateWindowFrames(Window.Group windowGroup) {
}
}
- private boolean isRowsOnlyAggregationCallType(ImmutableList<Window.RexWinAggCall> aggCalls) {
- return aggCalls.stream().anyMatch(aggCall -> aggCall.getKind().equals(SqlKind.ROW_NUMBER));
+ private boolean hasRankBasedAgg(ImmutableList<Window.RexWinAggCall> aggCalls) {
+ for (Window.RexWinAggCall aggCall : aggCalls) {
+ if (RANK_BASED_FUNCTION_KIND.contains(aggCall.getKind())) {
+ return true;
+ }
+ }
+ return false;
Review Comment:
I just discovered that we probably don't need any of the `hasRankBasedAgg` checks at all because Calcite itself ensures that functions like `RANK` / `DENSE_RANK` / `ROW_NUMBER` aren't used with a custom `ROWS` / `RANGE` window frame - https://github.com/apache/calcite/blob/df2328b7c2834cf7ffbf7305fe2c7aec5d07fe64/core/src/main/java/org/apache/calcite/sql/SqlWindow.java#L669-L674.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [WIP] Support preceding and following in WINDOW [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #14233:
URL: https://github.com/apache/pinot/pull/14233#issuecomment-2412625957
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14233?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 0.00%. Comparing base [(`59551e4`)](https://app.codecov.io/gh/apache/pinot/commit/59551e45224f1535c4863fd577622b37366ccc97?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`bac36dc`)](https://app.codecov.io/gh/apache/pinot/commit/bac36dcee832bf3b96c2bf2953378cebd2867dd5?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 1186 commits behind head on master.
> :exclamation: There is a different number of reports uploaded between BASE (59551e4) and HEAD (bac36dc). Click for more details.
>
> <details><summary>HEAD has 53 uploads less than BASE</summary>
>
>| Flag | BASE (59551e4) | HEAD (bac36dc) |
>|------|------|------|
>|integration|7|1|
>|integration2|3|1|
>|temurin|12|1|
>|java-21|7|1|
>|skip-bytebuffers-true|3|1|
>|skip-bytebuffers-false|7|0|
>|unittests|5|0|
>|unittests1|2|0|
>|java-11|5|0|
>|unittests2|3|0|
>|integration1|2|0|
>|custom-integration1|2|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #14233 +/- ##
=============================================
- Coverage 61.75% 0.00% -61.76%
=============================================
Files 2436 3 -2433
Lines 133233 6 -133227
Branches 20636 0 -20636
=============================================
- Hits 82274 0 -82274
+ Misses 44911 6 -44905
+ Partials 6048 0 -6048
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (-0.01%)` | :arrow_down: |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (ø)` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (-61.63%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (-27.73%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (-61.76%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/14233/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/14233?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [WIP] Support preceding and following in WINDOW [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #14233:
URL: https://github.com/apache/pinot/pull/14233#discussion_r1805616633
##########
pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java:
##########
@@ -209,23 +244,37 @@ private void validateWindows(Window window) {
}
private void validateWindowAggCallsSupported(Window.Group windowGroup) {
- for (int i = 0; i < windowGroup.aggCalls.size(); i++) {
- Window.RexWinAggCall aggCall = windowGroup.aggCalls.get(i);
+ for (Window.RexWinAggCall aggCall : windowGroup.aggCalls) {
SqlKind aggKind = aggCall.getKind();
Preconditions.checkState(SUPPORTED_WINDOW_FUNCTION_KIND.contains(aggKind),
String.format("Unsupported Window function kind: %s. Only aggregation functions are supported!", aggKind));
}
}
private void validateWindowFrames(Window.Group windowGroup) {
- // Has ROWS only aggregation call kind (e.g. ROW_NUMBER)?
- boolean isRowsOnlyTypeAggregateCall = isRowsOnlyAggregationCallType(windowGroup.aggCalls);
+ // Has rank based aggregation call kind (e.g. ROW_NUMBER, RANK, DENSE_RANK)
+ boolean hasRankBasedAgg = hasRankBasedAgg(windowGroup.aggCalls);
+
+ // FOLLOWING lower bound or PRECEDING upper bound is not allowed, and won't reach here
Review Comment:
Interesting! Does it mean it can do something like `BETWEEN 5 PRECEDING AND 3 PRECEDING`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [WIP] Support preceding and following in WINDOW [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #14233:
URL: https://github.com/apache/pinot/pull/14233#discussion_r1802890826
##########
pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java:
##########
@@ -209,23 +244,37 @@ private void validateWindows(Window window) {
}
private void validateWindowAggCallsSupported(Window.Group windowGroup) {
- for (int i = 0; i < windowGroup.aggCalls.size(); i++) {
- Window.RexWinAggCall aggCall = windowGroup.aggCalls.get(i);
+ for (Window.RexWinAggCall aggCall : windowGroup.aggCalls) {
SqlKind aggKind = aggCall.getKind();
Preconditions.checkState(SUPPORTED_WINDOW_FUNCTION_KIND.contains(aggKind),
String.format("Unsupported Window function kind: %s. Only aggregation functions are supported!", aggKind));
}
}
private void validateWindowFrames(Window.Group windowGroup) {
- // Has ROWS only aggregation call kind (e.g. ROW_NUMBER)?
- boolean isRowsOnlyTypeAggregateCall = isRowsOnlyAggregationCallType(windowGroup.aggCalls);
+ // Has rank based aggregation call kind (e.g. ROW_NUMBER, RANK, DENSE_RANK)
+ boolean hasRankBasedAgg = hasRankBasedAgg(windowGroup.aggCalls);
+
+ // FOLLOWING lower bound or PRECEDING upper bound is not allowed, and won't reach here
Review Comment:
Where is this enforced?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [WIP] Support preceding and following in WINDOW [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #14233:
URL: https://github.com/apache/pinot/pull/14233#discussion_r1803610953
##########
pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java:
##########
@@ -209,23 +244,37 @@ private void validateWindows(Window window) {
}
private void validateWindowAggCallsSupported(Window.Group windowGroup) {
- for (int i = 0; i < windowGroup.aggCalls.size(); i++) {
- Window.RexWinAggCall aggCall = windowGroup.aggCalls.get(i);
+ for (Window.RexWinAggCall aggCall : windowGroup.aggCalls) {
SqlKind aggKind = aggCall.getKind();
Preconditions.checkState(SUPPORTED_WINDOW_FUNCTION_KIND.contains(aggKind),
String.format("Unsupported Window function kind: %s. Only aggregation functions are supported!", aggKind));
}
}
private void validateWindowFrames(Window.Group windowGroup) {
- // Has ROWS only aggregation call kind (e.g. ROW_NUMBER)?
- boolean isRowsOnlyTypeAggregateCall = isRowsOnlyAggregationCallType(windowGroup.aggCalls);
+ // Has rank based aggregation call kind (e.g. ROW_NUMBER, RANK, DENSE_RANK)
+ boolean hasRankBasedAgg = hasRankBasedAgg(windowGroup.aggCalls);
+
+ // FOLLOWING lower bound or PRECEDING upper bound is not allowed, and won't reach here
Review Comment:
I tried it and iirc it is banned from parsing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [WIP] Support preceding and following in WINDOW [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #14233:
URL: https://github.com/apache/pinot/pull/14233#discussion_r1802470255
##########
pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java:
##########
@@ -209,23 +244,37 @@ private void validateWindows(Window window) {
}
private void validateWindowAggCallsSupported(Window.Group windowGroup) {
- for (int i = 0; i < windowGroup.aggCalls.size(); i++) {
- Window.RexWinAggCall aggCall = windowGroup.aggCalls.get(i);
+ for (Window.RexWinAggCall aggCall : windowGroup.aggCalls) {
SqlKind aggKind = aggCall.getKind();
Preconditions.checkState(SUPPORTED_WINDOW_FUNCTION_KIND.contains(aggKind),
String.format("Unsupported Window function kind: %s. Only aggregation functions are supported!", aggKind));
}
}
private void validateWindowFrames(Window.Group windowGroup) {
- // Has ROWS only aggregation call kind (e.g. ROW_NUMBER)?
- boolean isRowsOnlyTypeAggregateCall = isRowsOnlyAggregationCallType(windowGroup.aggCalls);
+ // Has rank based aggregation call kind (e.g. ROW_NUMBER, RANK, DENSE_RANK)
+ boolean hasRankBasedAgg = hasRankBasedAgg(windowGroup.aggCalls);
+
+ // FOLLOWING lower bound or PRECEDING upper bound is not allowed, and won't reach here
+ RexWindowBound lowerBound = windowGroup.lowerBound;
+ assert !lowerBound.isFollowing();
+ RexWindowBound upperBound = windowGroup.upperBound;
+ assert !upperBound.isPreceding();
+ boolean hasBound = (lowerBound.isPreceding() && !lowerBound.isUnbounded()) || (upperBound.isFollowing()
+ && !upperBound.isUnbounded());
+
+ if (hasRankBasedAgg) {
+ Preconditions.checkState(!hasBound, "Window frame is not supported for ROW_NUMBER, RANK, DENSE_RANK functions");
+ }
+ if (!windowGroup.isRows) {
+ Preconditions.checkState(!hasBound, "Bounded RANGE window frame is currently not supported");
+ }
// For Phase 1 only the default frame is supported
- Preconditions.checkState(!windowGroup.isRows || isRowsOnlyTypeAggregateCall,
+ Preconditions.checkState(!windowGroup.isRows || hasRankBasedAgg,
Review Comment:
Line 272-283 was supposed to be removed. I accidentally reverted the removal to compare with the new logic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [WIP] Support preceding and following in WINDOW [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #14233:
URL: https://github.com/apache/pinot/pull/14233#discussion_r1805881594
##########
pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java:
##########
@@ -209,23 +244,37 @@ private void validateWindows(Window window) {
}
private void validateWindowAggCallsSupported(Window.Group windowGroup) {
- for (int i = 0; i < windowGroup.aggCalls.size(); i++) {
- Window.RexWinAggCall aggCall = windowGroup.aggCalls.get(i);
+ for (Window.RexWinAggCall aggCall : windowGroup.aggCalls) {
SqlKind aggKind = aggCall.getKind();
Preconditions.checkState(SUPPORTED_WINDOW_FUNCTION_KIND.contains(aggKind),
String.format("Unsupported Window function kind: %s. Only aggregation functions are supported!", aggKind));
}
}
private void validateWindowFrames(Window.Group windowGroup) {
- // Has ROWS only aggregation call kind (e.g. ROW_NUMBER)?
- boolean isRowsOnlyTypeAggregateCall = isRowsOnlyAggregationCallType(windowGroup.aggCalls);
+ // Has rank based aggregation call kind (e.g. ROW_NUMBER, RANK, DENSE_RANK)
+ boolean hasRankBasedAgg = hasRankBasedAgg(windowGroup.aggCalls);
+
+ // FOLLOWING lower bound or PRECEDING upper bound is not allowed, and won't reach here
Review Comment:
Yep. Postgres allows that, Calcite allows that, and with https://github.com/apache/pinot/pull/14249, Pinot will support that as well!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [WIP] Support preceding and following in WINDOW [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed pull request #14233: [WIP] Support preceding and following in WINDOW
URL: https://github.com/apache/pinot/pull/14233
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org