You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2025/12/11 18:57:53 UTC
[I] Avoid `CometColumnarExchange` when next query stage requires `CometColumnarToRow` [datafusion-comet]
andygrove opened a new issue, #2880:
URL: https://github.com/apache/datafusion-comet/issues/2880
### What is the problem the feature request solves?
The golden file at `spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1.native_datafusion/extended.txt` has this section:
```
: : : +- HashAggregate
: : : +- CometColumnarToRow
: : : +- CometColumnarExchange
: : : +- HashAggregate
```
Presumably the first `HashAggregate` shown here is not supported by Comet (fallback reasons are missing for some reason) so this query stage needs to call `CometColumnarToRow`.
There is _probably_ no benefit in using columnar shuffle in the child stage if the parent stage immediately converts back to row-based, although we should confirm that assumption.
### Describe the potential solution
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Avoid `CometColumnarExchange` when next query stage requires `CometColumnarToRow` [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #2880:
URL: https://github.com/apache/datafusion-comet/issues/2880#issuecomment-5861034362
This is the same shape as #4004, which #4010 fixed by reverting a Comet columnar shuffle to a Spark shuffle when both its parent and child are Spark hash aggregates. The pattern no longer appears in any of the TPC-DS golden plans, and the broader question of when Comet should decline a stage is tracked in #4005.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Avoid `CometColumnarExchange` when next query stage requires `CometColumnarToRow` [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #2880: Avoid `CometColumnarExchange` when next query stage requires `CometColumnarToRow`
URL: https://github.com/apache/datafusion-comet/issues/2880
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org