You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "peterxcli (via GitHub)" <gi...@apache.org> on 2026/07/31 15:06:02 UTC
[PR] fix: use per-expression eval mode for decimal promotion [datafusion-comet]
peterxcli opened a new pull request, #5171:
URL: https://github.com/apache/datafusion-comet/pull/5171
## Which issue does this PR close?
Closes #5075.
## Rationale for this change
`DecimalPrecision.promote` currently derives `nullOnOverflow` from the live session ANSI setting. Spark expressions capture their own evaluation mode, and one expression tree can contain mixed modes, such as a `TRY` operation nested inside an ANSI expression. Each generated `CheckOverflow` wrapper must therefore follow the evaluation mode of the arithmetic expression it guards.
## What changes are included in this PR?
- Remove the session-derived `nullOnOverflow` argument from `DecimalPrecision.promote`.
- Derive `nullOnOverflow` from each decimal arithmetic node's evaluation mode.
- Add a Spark 4.1 regression test that verifies nested ANSI and TRY expressions serialize with independent overflow behavior.
## How are these changes tested?
- `make core`
- `./mvnw test -Pspark-4.1 -Dtest=none -Dsuites=org.apache.spark.sql.comet.CometDecimalArithmeticViewSuite -Dscalastyle.skip=true` (1 suite, 2 tests)
- `./mvnw scalastyle:check -Pspark-4.1`
- `./mvnw spotless:apply -Pspark-4.1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: use per-expression eval mode for decimal promotion [datafusion-comet]
Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on PR #5171:
URL: https://github.com/apache/datafusion-comet/pull/5171#issuecomment-5149335280
@andygrove Thanks for the review!
> 1. The new regression test does not run on Spark 4.2.
- moved `CometDecimalArithmeticViewSuite` to `spark/src/test/spark-4.1+/`
- added `src/test/${shims.minorPlusVerSrc}` into `pom.xml`
> 2. The TRY assertion in the new test is currently vacuous.
added scala test
3. Only Add is covered, but the change touches five operators.
added Subtract, Multiply, Divide and Remainder scala tests.
4. Please document why the flag comes from the expression.
added comment.
5. The Divide wrapper is now provably redundant, and I would like a tracking issue for it.
filed https://github.com/apache/datafusion-comet/issues/5190
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: use per-expression eval mode for decimal promotion [datafusion-comet]
Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on PR #5171:
URL: https://github.com/apache/datafusion-comet/pull/5171#issuecomment-5151935740
@andygrove thanks for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: use per-expression eval mode for decimal promotion [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove merged PR #5171:
URL: https://github.com/apache/datafusion-comet/pull/5171
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org