You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/04/28 11:47:45 UTC
[I] SPARK-53968 SQLViewSuite: decimal arithmetic returns ~10x smaller values through view CTE on Spark 4.1.1 [datafusion-comet]
andygrove opened a new issue, #4124:
URL: https://github.com/apache/datafusion-comet/issues/4124
## Describe the bug
The new Spark 4.1 test `SPARK-53968 reading the view after allowPrecisionLoss is changed` in `SQLViewSuite` fails with Comet enabled. The test stores `DECIMAL(38, 18)` values and computes `unit_price + COALESCE(shipping_price, 0)` through a CTE wrapped in a view. Comet returns values approximately 10x smaller than expected:
| Row | Expected | Actual (with Comet) |
|---|---|---|
| part1 | 100.00000000000000000 | 10.00000000000000000 |
| part2 | 100.00000000000000000 | 10.00000000000000000 |
| part3 | 300.23000000000000000 | 30.02300000000000000 |
The plan involves `CometBroadcastHashJoin → CometProject(unit_price + COALESCE(shipping_price, 0E-18)) → CometExchange(rangepartitioning) → CometSort`.
## Steps to reproduce
Run Spark 4.1.1's `SQLViewSuite` with Comet enabled. The reproducer is the test body itself (see `sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala` `SPARK-53968 reading the view after allowPrecisionLoss is changed`).
## Expected behavior
`unit_price + shipping_price` should match Spark's result (100, 100, 300.23).
## Workaround
The test is currently tagged `IgnoreComet(...)` in `dev/diffs/4.1.1.diff`.
## Additional context
PR #4093 enables Spark 4.1.1 in the `Spark SQL Tests` workflow. The 10x discrepancy and the `DECIMAL(38, 18)` schema strongly suggest a precision/scale handling bug in Comet's decimal addition or in the view-resolution path that drops 1 digit of scale.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] SPARK-53968 SQLViewSuite: decimal arithmetic returns ~10x smaller values through view CTE on Spark 4.1.1 [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #4124: SPARK-53968 SQLViewSuite: decimal arithmetic returns ~10x smaller values through view CTE on Spark 4.1.1
URL: https://github.com/apache/datafusion-comet/issues/4124
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org