You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "rafafrdz (via GitHub)" <gi...@apache.org> on 2026/03/25 07:01:07 UTC
[PR] feat: Add support for Spark NaNvl math expression [datafusion-comet]
rafafrdz opened a new pull request, #3790:
URL: https://github.com/apache/datafusion-comet/pull/3790
## Summary
- Add support for Spark's `NaNvl` expression (returns first arg if not NaN, otherwise second arg)
- Delegates to DataFusion's built-in `nanvl` function via `CometScalarFunction`
- No native Rust code needed — DataFusion's function registry handles execution
- SQL test added covering NaN, normal values, NULL, and mixed inputs
## Test plan
- [x] SQL test added: `nanvl.sql`
- [ ] Run `CometSqlFileTestSuite` to verify results match Spark
Related to #2084
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: Add support for Spark NaNvl math expression [datafusion-comet]
Posted by "rafafrdz (via GitHub)" <gi...@apache.org>.
rafafrdz closed pull request #3790: feat: Add support for Spark NaNvl math expression
URL: https://github.com/apache/datafusion-comet/pull/3790
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: Add support for Spark NaNvl math expression [datafusion-comet]
Posted by "rafafrdz (via GitHub)" <gi...@apache.org>.
rafafrdz commented on PR #3790:
URL: https://github.com/apache/datafusion-comet/pull/3790#issuecomment-4124349005
Closing this PR — DataFusion's `nanvl` has different NULL handling semantics than Spark's `NaNvl`. For the row `nanvl(4.0, NULL)`, DataFusion returns NULL while Spark returns 4.0. This function needs a custom Comet implementation instead of simple delegation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org