You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "coderfender (via GitHub)" <gi...@apache.org> on 2025/08/13 16:55:24 UTC
[PR] docs: Update to support try arithmetic functions [datafusion-comet]
coderfender opened a new pull request, #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143
## Which issue does this PR close?
Documentation update to #2073 changes
Closes #.
## Rationale for this change
Doc update
## What changes are included in this PR?
Doc update
## How are these changes tested?
Local testing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Update to support try arithmetic functions [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on code in PR #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143#discussion_r2274815404
##########
docs/source/user-guide/expressions.md:
##########
@@ -44,6 +44,15 @@ The following Spark expressions are currently available. Any known compatibility
| IntegralDivide (`div`) | All operands are cast to DecimalType (in case the input type is not already decima type) with precision 19 and scale 0. Please set `spark.comet.cast.allowIncompatible` to `true` to enable DataFusion’s cast operation for LongType inputs. |
| Remainder (`%`) | |
+## Binary Try Arithmetic
+
+| Expression | Notes |
+|------------|----------------------------------------------------------------------------|
+| `try_add` | Adds operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_sub` | Subtracts operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_mul` | Multiplies operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_div` | Subtracts operands (IntegerTypes only) or results NULL incase of overflow |
Review Comment:
The only numeric datatypes spark supports with NULL on Overflow are IntegerTypes (int / long -> int32 /int64) . Along with that Spark supports try arithmetic for datetime literals as well but that isnt supported in comet yet and fallsback to spark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Update to support try arithmetic functions [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on code in PR #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143#discussion_r2277997609
##########
docs/source/user-guide/expressions.md:
##########
@@ -44,6 +44,15 @@ The following Spark expressions are currently available. Any known compatibility
| IntegralDivide (`div`) | All operands are cast to DecimalType (in case the input type is not already decima type) with precision 19 and scale 0. Please set `spark.comet.cast.allowIncompatible` to `true` to enable DataFusion’s cast operation for LongType inputs. |
| Remainder (`%`) | |
+## Binary Try Arithmetic
+
+| Expression | Notes |
+|------------|----------------------------------------------------------------------------|
+| `try_add` | Adds operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_sub` | Subtracts operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_mul` | Multiplies operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_div` | Subtracts operands (IntegerTypes only) or results NULL incase of overflow |
Review Comment:
Thank you for the approval
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Update to support try arithmetic functions [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove merged PR #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Update to support try arithmetic functions [datafusion-comet]
Posted by "kazuyukitanimura (via GitHub)" <gi...@apache.org>.
kazuyukitanimura commented on code in PR #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143#discussion_r2274802165
##########
docs/source/user-guide/expressions.md:
##########
@@ -44,6 +44,15 @@ The following Spark expressions are currently available. Any known compatibility
| IntegralDivide (`div`) | All operands are cast to DecimalType (in case the input type is not already decima type) with precision 19 and scale 0. Please set `spark.comet.cast.allowIncompatible` to `true` to enable DataFusion’s cast operation for LongType inputs. |
| Remainder (`%`) | |
+## Binary Try Arithmetic
+
+| Expression | Notes |
+|------------|----------------------------------------------------------------------------|
+| `try_add` | Adds operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_sub` | Subtracts operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_mul` | Multiplies operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_div` | Subtracts operands (IntegerTypes only) or results NULL incase of overflow |
Review Comment:
Question if Comet encounters try_add with non-integer types, do we safely fallback to Spark?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Update to support try arithmetic functions [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on PR #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143#issuecomment-3184706326
@andygrove , Documentation update to let users know that we now support Try arithmetic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Update to support try arithmetic functions [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on PR #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143#issuecomment-3184795871
Thanks @coderfender could you also update docs/source/user-guide/expressions.md
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Update to support try arithmetic functions [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on PR #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143#issuecomment-3185130392
Thank you for the feedback @andygrove , I updated the expressions doc to provide further info re Try arithmetic functions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Update to support try arithmetic functions [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on code in PR #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143#discussion_r2274815404
##########
docs/source/user-guide/expressions.md:
##########
@@ -44,6 +44,15 @@ The following Spark expressions are currently available. Any known compatibility
| IntegralDivide (`div`) | All operands are cast to DecimalType (in case the input type is not already decima type) with precision 19 and scale 0. Please set `spark.comet.cast.allowIncompatible` to `true` to enable DataFusion’s cast operation for LongType inputs. |
| Remainder (`%`) | |
+## Binary Try Arithmetic
+
+| Expression | Notes |
+|------------|----------------------------------------------------------------------------|
+| `try_add` | Adds operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_sub` | Subtracts operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_mul` | Multiplies operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_div` | Subtracts operands (IntegerTypes only) or results NULL incase of overflow |
Review Comment:
Thats a good question @kazuyukitanimura . The only numeric datatypes spark supports with NULL on Overflow are IntegerTypes (int / long -> int32 /int64) . Along with that Spark supports try arithmetic for datetime literals as well but that isnt supported in comet yet and fallsback to spark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Update to support try arithmetic functions [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on PR #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143#issuecomment-3184708001
Please review and would love to add / update more docs if needed .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Update to support try arithmetic functions [datafusion-comet]
Posted by "kazuyukitanimura (via GitHub)" <gi...@apache.org>.
kazuyukitanimura commented on code in PR #2143:
URL: https://github.com/apache/datafusion-comet/pull/2143#discussion_r2274887676
##########
docs/source/user-guide/expressions.md:
##########
@@ -44,6 +44,15 @@ The following Spark expressions are currently available. Any known compatibility
| IntegralDivide (`div`) | All operands are cast to DecimalType (in case the input type is not already decima type) with precision 19 and scale 0. Please set `spark.comet.cast.allowIncompatible` to `true` to enable DataFusion’s cast operation for LongType inputs. |
| Remainder (`%`) | |
+## Binary Try Arithmetic
+
+| Expression | Notes |
+|------------|----------------------------------------------------------------------------|
+| `try_add` | Adds operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_sub` | Subtracts operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_mul` | Multiplies operands (IntegerTypes only) or results NULL incase of overflow |
+| `try_div` | Subtracts operands (IntegerTypes only) or results NULL incase of overflow |
Review Comment:
If it falls back to Spark, then it's fine, but I was unable to find the code that checks the data type. I wonder what will happen with float/double/decimal...etc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org