You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "tohuya6 (via GitHub)" <gi...@apache.org> on 2026/07/22 03:50:29 UTC
[PR] feat: add GroupColumn support for Interval in multi-column GROUP BY [datafusion]
tohuya6 opened a new pull request, #23786:
URL: https://github.com/apache/datafusion/pull/23786
## Which issue does this PR close?
- Part of #22715.
## Rationale for this change
`multi_group_by::group_column_supported_type` gates which GROUP BY columns may
use the column-wise `GroupValuesColumn` fast path, and the gate is
all-or-nothing: a single unsupported column forces the **entire** grouping onto
the byte-encoded `GroupValuesRows` fallback, even when every other key column
would have qualified. An `Interval` key triggers exactly that today.
All three `Interval` units reuse the existing `PrimitiveGroupValueBuilder` with
no new builder type and no new `HashValue` impl — the `i32` / `IntervalDayTime`
/ `IntervalMonthDayNano` natives already implement it.
## What changes are included in this PR?
- Dispatch the three `Interval*Type` units in `make_group_column`. `IntervalUnit`
has exactly three variants, so the match is exhaustive with no fallback arm
(unlike `Time32`/`Time64`, which must reject invalid unit combinations).
- Accept `Interval(_)` in `group_column_supported_type`.
- Extend the `group_column_supported_type` ↔ `make_group_column` consistency
fuzz with the three Interval units.
- Add an `(Interval, Int32)` group-count benchmark to `benches/multi_group_by.rs`.
## Are these changes tested?
Yes.
- New unit test: an `Interval` key stays on the `GroupValuesColumn` path, dedups
including nulls, keeps "1 month" and "30 days" as **distinct** groups (no
cross-unit folding), and round-trips with the `Interval` output type preserved.
- The consistency fuzz now asserts all three Interval units route through the
dispatcher.
- New single- and multi-column `Interval` `GROUP BY` coverage in `aggregate.slt`.
## Are there any user-facing changes?
No API changes. `GROUP BY` queries with an `Interval` key now use the
column-wise fast path instead of the row-encoded fallback; results are unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: add GroupColumn support for Interval in multi-column GROUP BY [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #23786:
URL: https://github.com/apache/datafusion/pull/23786#issuecomment-5141397465
<!-- semver-check-comment -->
Thank you for opening this pull request!
Reviewer note: [cargo-semver-checks](https://github.com/obi1kenobi/cargo-semver-checks) reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).
<details>
<summary>Details</summary>
```
Cloning apache/main
Building datafusion-physical-plan v54.1.0 (current)
Built [ 38.838s] (current)
Parsing datafusion-physical-plan v54.1.0 (current)
Parsed [ 0.142s] (current)
Building datafusion-physical-plan v54.1.0 (baseline)
Built [ 38.138s] (baseline)
Parsing datafusion-physical-plan v54.1.0 (baseline)
Parsed [ 0.153s] (baseline)
Checking datafusion-physical-plan v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 0.602s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip
--- failure inherent_method_missing: pub method removed or renamed ---
Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/inherent_method_missing.ron
Failed in:
ExecutionPlanDecodeCtx::expr_ctx, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/0b0f5244e930748b51242fd8a282cef534e7c632/datafusion/physical-plan/src/proto.rs:352
ExecutionPlanEncodeCtx::expr_ctx, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/0b0f5244e930748b51242fd8a282cef534e7c632/datafusion/physical-plan/src/proto.rs:227
Summary semver requires new major version: 1 major and 0 minor checks failed
Finished [ 79.410s] datafusion-physical-plan
Building datafusion-sqllogictest v54.1.0 (current)
Built [ 181.909s] (current)
Parsing datafusion-sqllogictest v54.1.0 (current)
Parsed [ 0.022s] (current)
Building datafusion-sqllogictest v54.1.0 (baseline)
Built [ 181.807s] (baseline)
Parsing datafusion-sqllogictest v54.1.0 (baseline)
Parsed [ 0.023s] (baseline)
Checking datafusion-sqllogictest v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 0.084s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 366.610s] datafusion-sqllogictest
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: add GroupColumn support for Interval in multi-column GROUP BY [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on PR #23786:
URL: https://github.com/apache/datafusion/pull/23786#issuecomment-5141919342
@tohuya6
Can you resolve the merge conflicts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: add GroupColumn support for Interval in multi-column GROUP BY [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #23786:
URL: https://github.com/apache/datafusion/pull/23786#issuecomment-5045845905
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/23786?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `94.73684%` with `2 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 80.71%. Comparing base ([`5de7f1d`](https://app.codecov.io/gh/apache/datafusion/commit/5de7f1db95191f81ce6472361785db8f63ac2db1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`b9753c4`](https://app.codecov.io/gh/apache/datafusion/commit/b9753c417da8c4dfa4bf1311b8b5df189848db93?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 2 commits behind head on main.
| [Files with missing lines](https://app.codecov.io/gh/apache/datafusion/pull/23786?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../src/aggregates/group\_values/multi\_group\_by/mod.rs](https://app.codecov.io/gh/apache/datafusion/pull/23786?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Faggregates%2Fgroup_values%2Fmulti_group_by%2Fmod.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy9hZ2dyZWdhdGVzL2dyb3VwX3ZhbHVlcy9tdWx0aV9ncm91cF9ieS9tb2QucnM=) | 94.73% | [0 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23786?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #23786 +/- ##
==========================================
- Coverage 80.71% 80.71% -0.01%
==========================================
Files 1089 1089
Lines 368760 368798 +38
Branches 368760 368798 +38
==========================================
+ Hits 297647 297666 +19
- Misses 53372 53382 +10
- Partials 17741 17750 +9
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/23786?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: add GroupColumn support for Interval in multi-column GROUP BY [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew merged PR #23786:
URL: https://github.com/apache/datafusion/pull/23786
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: add GroupColumn support for Interval in multi-column GROUP BY [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on code in PR #23786:
URL: https://github.com/apache/datafusion/pull/23786#discussion_r3689777753
##########
datafusion/physical-plan/src/aggregates/group_values/multi_group_by/mod.rs:
##########
@@ -1620,6 +1641,62 @@ mod tests {
}
}
+ // `(Interval, Int32)` keys: null keys dedup, the Int32 key splits equal
+ // intervals, and emit gives back Interval rather than the raw native.
+ #[test]
+ fn test_group_values_column_interval() {
Review Comment:
Could we extend this runtime test to also intern and emit `Interval(YearMonth)` and `Interval(DayTime)` arrays? The gate and factory test now constructs builders for all three interval units, but the end-to-end append, equality, and emit coverage currently only exercises `MonthDayNano`. A small table-driven unit test would help confirm that all three interval units behave as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: add GroupColumn support for Interval in multi-column GROUP BY [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on PR #23786:
URL: https://github.com/apache/datafusion/pull/23786#issuecomment-5102216834
@tohuya6
Can you resolve the conflicts on this pr?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: add GroupColumn support for Interval in multi-column GROUP BY [datafusion]
Posted by "tohuya6 (via GitHub)" <gi...@apache.org>.
tohuya6 commented on PR #23786:
URL: https://github.com/apache/datafusion/pull/23786#issuecomment-5151221789
@kosiew resolved, appreciate your help
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org