You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/06/29 15:37:19 UTC
[PR] feat: name incompatible aggregate functions in mixed-execution fallback reason [datafusion-comet]
andygrove opened a new pull request, #4750:
URL: https://github.com/apache/datafusion-comet/pull/4750
## Which issue does this PR close?
<!-- No dedicated issue; minor diagnostic improvement. -->
N/A
## Rationale for this change
When a Spark Final `HashAggregate` cannot be converted to Comet because the partial aggregate ran natively (or would have) and the intermediate buffer formats are incompatible between Spark and Comet, the fallback reason shown in `EXPLAIN` output and the tpcds plan-stability golden files was:
```
HashAggregate [COMET: Spark Final aggregate without Comet Partial requires compatible intermediate buffer formats]
```
This did not tell the reader which aggregate function caused the incompatibility, making it harder to understand why an aggregate fell back to Spark.
## What changes are included in this PR?
- Added `QueryPlanSerde.aggsNotSupportingMixedExecution(aggExprs)`, which returns the aggregate functions whose `supportsMixedPartialFinal` is `false` (or that have no Comet serde). `allAggsSupportMixedExecution` is now implemented on top of it so there is a single source of truth.
- `CometBaseAggregate.doConvert` now appends the offending function names (deduped `prettyName`s) to the fallback reason, for example:
```
HashAggregate [COMET: Spark Final aggregate without Comet Partial requires compatible intermediate buffer formats, but the following aggregate function(s) have incompatible buffers: count, avg, sum]
```
- Regenerated the affected tpcds plan-stability golden files across all Spark versions.
## How are these changes tested?
Covered by the existing `CometTPCDSV1_4_PlanStabilitySuite` / `CometTPCDSV2_7_PlanStabilitySuite` plan-stability tests, whose golden files embed this fallback reason and were regenerated and verified to pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: name incompatible aggregate functions in mixed-execution fallback reason [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove merged PR #4750:
URL: https://github.com/apache/datafusion-comet/pull/4750
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org