You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2025/11/06 22:29:47 UTC
[I] Enable `spark.comet.sparkToColumnar.enabled` when running Spark SQL tests [datafusion-comet]
andygrove opened a new issue, #2723:
URL: https://github.com/apache/datafusion-comet/issues/2723
### What is the problem the feature request solves?
Many (most?) Spark SQL tests do not start with Parquet scan, but use `RDDScanExec`, `LocalTableScanExec`, etc. so Comet falls back to Spark very early. This means that we are missing out on test coverage.
### Describe the potential solution
- Make `spark.comet.sparkToColumnar.enabled` configurable via env var
- Set that env var in CI when running Spark SQL tests
- Fix failing tests (or ignore them until we can fix the bugs)
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Enable `spark.comet.sparkToColumnar.enabled` when running Spark SQL tests [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #2723:
URL: https://github.com/apache/datafusion-comet/issues/2723#issuecomment-4802683819
closing in favor of https://github.com/apache/datafusion-comet/issues/4347
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Enable `spark.comet.sparkToColumnar.enabled` when running Spark SQL tests [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #2723: Enable `spark.comet.sparkToColumnar.enabled` when running Spark SQL tests
URL: https://github.com/apache/datafusion-comet/issues/2723
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org