You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/01/29 14:51:18 UTC
[PR] Add SQL file-based test framework [datafusion-comet]
andygrove opened a new pull request, #3325:
URL: https://github.com/apache/datafusion-comet/pull/3325
## Summary
- Adds a sqllogictest-inspired framework for writing Comet SQL tests as plain `.sql` files
- Converts 11 tests from `CometExpressionSuite` into 8 `.sql` files under `spark/src/test/resources/sql-tests/expressions/`
- Adds `CometSqlFileTestSuite` (test runner) and `SqlFileTestParser` (parser)
- Adds the new suite to Linux and macOS CI workflows
### Supported directives
| Directive | Purpose |
|-----------|---------|
| `-- Config: key=value` | Static Spark/Comet SQL config |
| `-- ConfigMatrix: key=val1,val2` | Run file once per value (cross-product) |
| `statement` | Execute DDL/DML |
| `query` | Run `checkSparkAnswerAndOperator` |
| `query spark_answer_only` | Run `checkSparkAnswer` (no operator check) |
| `query tolerance=0.001` | Floating-point tolerance comparison |
### Why
- **No recompilation needed** to add or modify SQL tests — just edit a `.sql` file and run
- **Aligns with upstream DataFusion**, which uses sqllogictest `.slt` files extensively for SQL testing
- **Lower barrier** for contributors to add test coverage — no Scala knowledge required
- **Declarative configs** via `Config`/`ConfigMatrix` headers replace nested `Seq.foreach`/`withSQLConf` boilerplate
## Test plan
- [x] All 16 SQL file tests pass (including ConfigMatrix expansion)
- [x] Remaining `CometExpressionSuite` tests still pass (118 tests)
- [x] Total: 134 tests succeeded, 0 failed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] tests: Add SQL file-based test framework [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on PR #3325:
URL: https://github.com/apache/datafusion-comet/pull/3325#issuecomment-3818750647
Actually, I may as well replace this with https://github.com/apache/datafusion-comet/pull/3328
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] tests: Add SQL file-based test framework [datafusion-comet]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #3325:
URL: https://github.com/apache/datafusion-comet/pull/3325#issuecomment-3818269157
## [Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/3325?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 16.41%. Comparing base ([`f09f8af`](https://app.codecov.io/gh/apache/datafusion-comet/commit/f09f8af64c6599255e116a376f4f008f2fd63b43?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`4ab3f19`](https://app.codecov.io/gh/apache/datafusion-comet/commit/4ab3f1987b0e4d628582c1f9e71592eead065e1c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 905 commits behind head on main.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #3325 +/- ##
=============================================
- Coverage 56.12% 16.41% -39.72%
+ Complexity 976 411 -565
=============================================
Files 119 175 +56
Lines 11743 16153 +4410
Branches 2251 2656 +405
=============================================
- Hits 6591 2651 -3940
- Misses 4012 13080 +9068
+ Partials 1140 422 -718
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/datafusion-comet/pull/3325?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] tests: Add SQL file-based test framework [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed pull request #3325: tests: Add SQL file-based test framework
URL: https://github.com/apache/datafusion-comet/pull/3325
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org