You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "hareshkh (via GitHub)" <gi...@apache.org> on 2026/03/03 17:46:52 UTC
[PR] Fix cast wrapped AggregateUDF during Substrait Conversion [datafusion]
hareshkh opened a new pull request, #20679:
URL: https://github.com/apache/datafusion/pull/20679
## Which issue does this PR close?
- Closes #.
## Rationale for this change
When the Substrait consumer applies root-level schema renames that involve struct field renames inside an aggregate measure's return type (e.g. `List<Struct{c0,c1}>` → `List<Struct{one,two}>`), `rename_expressions` injects `Expr::Cast` around the aggregate function. The physical planner rejects Cast-wrapped aggregates, causing a planning error.
## What changes are included in this PR?
- Extracted the root-level rename logic from `from_substrait_plan_with_consumer` into a dedicated `apply_renames` function in `plan.rs`.
- Added a new `alias_expressions` utility in `utils.rs` that applies only top-level column name aliases without injecting `Expr::Cast`, making it safe for aggregate expressions.
- Changed the `Aggregate` rename strategy: first apply alias-only renames via `alias_expressions`; if the schema still doesn't match (e.g. nested struct field renames require casts), wrap the Aggregate in a Projection that carries those casts.
- Added a test (`aggregate_with_struct_field_rename`) that constructs a Substrait plan with `array_agg` returning `List<Struct{c0,c1}>` and root names that rename the struct fields to `one` and `two`, verifying the consumer produces a valid plan that the physical planner accepts.
## Are these changes tested?
Yes. A new test case `aggregate_with_struct_field_rename` in `aggregation_tests.rs` builds a Substrait plan with a struct field rename on an aggregate measure and verifies that:
1. The resulting logical plan wraps the Aggregate in a Projection.
2. The physical planner accepts the plan and can execute it (`DataFrame::show()`).
## Are there any user-facing changes?
No. This is a bug fix in the Substrait consumer that corrects plan generation for aggregate expressions with nested struct field renames. There are no API changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fix cast wrapped AggregateUDF during Substrait Conversion [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #20679: Fix cast wrapped AggregateUDF during Substrait Conversion
URL: https://github.com/apache/datafusion/pull/20679
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fix cast wrapped AggregateUDF during Substrait Conversion [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20679:
URL: https://github.com/apache/datafusion/pull/20679#issuecomment-4367816496
Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org