You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/10/21 14:55:05 UTC
(datafusion-comet) branch asf-site updated: Publish built docs triggered by ab884e24aca7baeaa8df723e70a1bdfc3efc7943
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 06e678cb06 Publish built docs triggered by ab884e24aca7baeaa8df723e70a1bdfc3efc7943
06e678cb06 is described below
commit 06e678cb06666ef40b92608e2991e87750994fd2
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Wed Oct 21 14:42:26 2026 +0000
Publish built docs triggered by ab884e24aca7baeaa8df723e70a1bdfc3efc7943
---
...ermaid-09341aca03757d9d3637a2eaff73472d472d0673.svg | 2 +-
.../compatibility/expressions/spark-3.4/cast.md.txt | 18 ++++++++++++++++++
.../compatibility/expressions/spark-3.4/math.md.txt | 12 ++++++++++++
.../compatibility/expressions/spark-3.5/cast.md.txt | 18 ++++++++++++++++++
.../compatibility/expressions/spark-3.5/math.md.txt | 12 ++++++++++++
.../compatibility/expressions/spark-4.0/cast.md.txt | 18 ++++++++++++++++++
.../compatibility/expressions/spark-4.0/math.md.txt | 12 ++++++++++++
.../compatibility/expressions/spark-4.1/cast.md.txt | 18 ++++++++++++++++++
.../compatibility/expressions/spark-4.1/math.md.txt | 12 ++++++++++++
.../compatibility/expressions/spark-4.2/cast.md.txt | 18 ++++++++++++++++++
.../compatibility/expressions/spark-4.2/math.md.txt | 12 ++++++++++++
searchindex.js | 2 +-
.../compatibility/expressions/spark-3.4/cast.html | 17 +++++++++++++++++
.../compatibility/expressions/spark-3.4/math.html | 16 ++++++++++++++++
.../compatibility/expressions/spark-3.5/cast.html | 17 +++++++++++++++++
.../compatibility/expressions/spark-3.5/math.html | 16 ++++++++++++++++
.../compatibility/expressions/spark-4.0/cast.html | 17 +++++++++++++++++
.../compatibility/expressions/spark-4.0/math.html | 16 ++++++++++++++++
.../compatibility/expressions/spark-4.1/cast.html | 17 +++++++++++++++++
.../compatibility/expressions/spark-4.1/math.html | 16 ++++++++++++++++
.../compatibility/expressions/spark-4.2/cast.html | 17 +++++++++++++++++
.../compatibility/expressions/spark-4.2/math.html | 16 ++++++++++++++++
22 files changed, 317 insertions(+), 2 deletions(-)
diff --git a/_images/mermaid-09341aca03757d9d3637a2eaff73472d472d0673.svg b/_images/mermaid-09341aca03757d9d3637a2eaff73472d472d0673.svg
index 4eeaed1f9e..9ea9f2c693 100644
--- a/_images/mermaid-09341aca03757d9d3637a2eaff73472d472d0673.svg
+++ b/_images/mermaid-09341aca03757d9d3637a2eaff73472d472d0673.svg
@@ -1 +1 @@
-<svg id="my-svg" width="100%" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" class="flowchart" style="max-width: 511.73px; background-color: transparent;" viewBox="0 0 511.7299499511719 1014" role="graphics-document document" aria-roledescription="flowchart-v2"><style>#my-svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#my-sv [...]
\ No newline at end of file
+<svg id="my-svg" width="100%" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" class="flowchart" style="max-width: 511.73px; background-color: transparent;" viewBox="0 0 511.7299499511719 1014" role="graphics-document document" aria-roledescription="flowchart-v2"><style>#my-svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#my-sv [...]
\ No newline at end of file
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-3.4/cast.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-3.4/cast.md.txt
index cc8cfbbbd6..11f858012f 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-3.4/cast.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-3.4/cast.md.txt
@@ -165,6 +165,24 @@ Spark's behavior of using Java `BigDecimal.toString()` semantics, which produces
notation (e.g. a value of 12300 stored as `Decimal(7,-2)` with unscaled value 123 is rendered
as `"1.23E+4"`).
+## Decimal with Negative Scale to Integer and Timestamp
+
+Casting a `DecimalType` with a negative scale to `ByteType`, `ShortType`, `IntegerType`,
+`LongType` or `TimestampType` has no native implementation, and neither does casting any of the
+integer types to a negative-scale `DecimalType`. Comet reports these as unsupported, which
+routes them through the JVM codegen dispatcher (Spark's own generated code) inside the Comet
+pipeline, so the results match Spark exactly.
+
+The native kernels align scale by computing `10^|scale|` in fixed-point arithmetic, which
+overflows when the scale is negative. See
+[#5013](https://github.com/apache/datafusion-comet/issues/5013).
+
+All other cast directions to and from negative-scale decimals run natively: `FloatType` and
+`DoubleType`, `StringType` in both directions (subject to the config gate described above),
+casts from a negative-scale decimal to `BooleanType`, and widening to another `DecimalType`.
+Casts from `BooleanType` to a `DecimalType` route through the dispatcher at any scale, which is
+not specific to a negative one.
+
## Complex Types
Comet casts arrays, structs, and maps by casting each element, field, key, and value with the
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-3.4/math.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-3.4/math.md.txt
index 3191b91640..86d8c635dc 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-3.4/math.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-3.4/math.md.txt
@@ -27,12 +27,24 @@ The following cases have no native implementation and always run in the JVM usin
- `INTERVAL YEAR TO MONTH` and `INTERVAL DAY TO SECOND` inputs
+## Divide
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Decimal division with negative scale is not supported natively
+
## Greatest
For applicable cases that are not selected for native execution automatically, `Greatest` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.Greatest.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
- Spark evaluates non-UTF8_BINARY collated string input under its collation, while Comet's native greatest compares raw bytes
+## IntegralDivide
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Decimal division with negative scale is not supported natively
+
## Least
For applicable cases that are not selected for native execution automatically, `Least` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.Least.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-3.5/cast.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-3.5/cast.md.txt
index cc8cfbbbd6..11f858012f 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-3.5/cast.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-3.5/cast.md.txt
@@ -165,6 +165,24 @@ Spark's behavior of using Java `BigDecimal.toString()` semantics, which produces
notation (e.g. a value of 12300 stored as `Decimal(7,-2)` with unscaled value 123 is rendered
as `"1.23E+4"`).
+## Decimal with Negative Scale to Integer and Timestamp
+
+Casting a `DecimalType` with a negative scale to `ByteType`, `ShortType`, `IntegerType`,
+`LongType` or `TimestampType` has no native implementation, and neither does casting any of the
+integer types to a negative-scale `DecimalType`. Comet reports these as unsupported, which
+routes them through the JVM codegen dispatcher (Spark's own generated code) inside the Comet
+pipeline, so the results match Spark exactly.
+
+The native kernels align scale by computing `10^|scale|` in fixed-point arithmetic, which
+overflows when the scale is negative. See
+[#5013](https://github.com/apache/datafusion-comet/issues/5013).
+
+All other cast directions to and from negative-scale decimals run natively: `FloatType` and
+`DoubleType`, `StringType` in both directions (subject to the config gate described above),
+casts from a negative-scale decimal to `BooleanType`, and widening to another `DecimalType`.
+Casts from `BooleanType` to a `DecimalType` route through the dispatcher at any scale, which is
+not specific to a negative one.
+
## Complex Types
Comet casts arrays, structs, and maps by casting each element, field, key, and value with the
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-3.5/math.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-3.5/math.md.txt
index 3191b91640..86d8c635dc 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-3.5/math.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-3.5/math.md.txt
@@ -27,12 +27,24 @@ The following cases have no native implementation and always run in the JVM usin
- `INTERVAL YEAR TO MONTH` and `INTERVAL DAY TO SECOND` inputs
+## Divide
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Decimal division with negative scale is not supported natively
+
## Greatest
For applicable cases that are not selected for native execution automatically, `Greatest` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.Greatest.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
- Spark evaluates non-UTF8_BINARY collated string input under its collation, while Comet's native greatest compares raw bytes
+## IntegralDivide
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Decimal division with negative scale is not supported natively
+
## Least
For applicable cases that are not selected for native execution automatically, `Least` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.Least.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/cast.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/cast.md.txt
index cc8cfbbbd6..11f858012f 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/cast.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/cast.md.txt
@@ -165,6 +165,24 @@ Spark's behavior of using Java `BigDecimal.toString()` semantics, which produces
notation (e.g. a value of 12300 stored as `Decimal(7,-2)` with unscaled value 123 is rendered
as `"1.23E+4"`).
+## Decimal with Negative Scale to Integer and Timestamp
+
+Casting a `DecimalType` with a negative scale to `ByteType`, `ShortType`, `IntegerType`,
+`LongType` or `TimestampType` has no native implementation, and neither does casting any of the
+integer types to a negative-scale `DecimalType`. Comet reports these as unsupported, which
+routes them through the JVM codegen dispatcher (Spark's own generated code) inside the Comet
+pipeline, so the results match Spark exactly.
+
+The native kernels align scale by computing `10^|scale|` in fixed-point arithmetic, which
+overflows when the scale is negative. See
+[#5013](https://github.com/apache/datafusion-comet/issues/5013).
+
+All other cast directions to and from negative-scale decimals run natively: `FloatType` and
+`DoubleType`, `StringType` in both directions (subject to the config gate described above),
+casts from a negative-scale decimal to `BooleanType`, and widening to another `DecimalType`.
+Casts from `BooleanType` to a `DecimalType` route through the dispatcher at any scale, which is
+not specific to a negative one.
+
## Complex Types
Comet casts arrays, structs, and maps by casting each element, field, key, and value with the
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/math.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/math.md.txt
index 3191b91640..86d8c635dc 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/math.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/math.md.txt
@@ -27,12 +27,24 @@ The following cases have no native implementation and always run in the JVM usin
- `INTERVAL YEAR TO MONTH` and `INTERVAL DAY TO SECOND` inputs
+## Divide
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Decimal division with negative scale is not supported natively
+
## Greatest
For applicable cases that are not selected for native execution automatically, `Greatest` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.Greatest.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
- Spark evaluates non-UTF8_BINARY collated string input under its collation, while Comet's native greatest compares raw bytes
+## IntegralDivide
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Decimal division with negative scale is not supported natively
+
## Least
For applicable cases that are not selected for native execution automatically, `Least` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.Least.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/cast.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/cast.md.txt
index cc8cfbbbd6..11f858012f 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/cast.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/cast.md.txt
@@ -165,6 +165,24 @@ Spark's behavior of using Java `BigDecimal.toString()` semantics, which produces
notation (e.g. a value of 12300 stored as `Decimal(7,-2)` with unscaled value 123 is rendered
as `"1.23E+4"`).
+## Decimal with Negative Scale to Integer and Timestamp
+
+Casting a `DecimalType` with a negative scale to `ByteType`, `ShortType`, `IntegerType`,
+`LongType` or `TimestampType` has no native implementation, and neither does casting any of the
+integer types to a negative-scale `DecimalType`. Comet reports these as unsupported, which
+routes them through the JVM codegen dispatcher (Spark's own generated code) inside the Comet
+pipeline, so the results match Spark exactly.
+
+The native kernels align scale by computing `10^|scale|` in fixed-point arithmetic, which
+overflows when the scale is negative. See
+[#5013](https://github.com/apache/datafusion-comet/issues/5013).
+
+All other cast directions to and from negative-scale decimals run natively: `FloatType` and
+`DoubleType`, `StringType` in both directions (subject to the config gate described above),
+casts from a negative-scale decimal to `BooleanType`, and widening to another `DecimalType`.
+Casts from `BooleanType` to a `DecimalType` route through the dispatcher at any scale, which is
+not specific to a negative one.
+
## Complex Types
Comet casts arrays, structs, and maps by casting each element, field, key, and value with the
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/math.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/math.md.txt
index 3191b91640..86d8c635dc 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/math.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/math.md.txt
@@ -27,12 +27,24 @@ The following cases have no native implementation and always run in the JVM usin
- `INTERVAL YEAR TO MONTH` and `INTERVAL DAY TO SECOND` inputs
+## Divide
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Decimal division with negative scale is not supported natively
+
## Greatest
For applicable cases that are not selected for native execution automatically, `Greatest` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.Greatest.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
- Spark evaluates non-UTF8_BINARY collated string input under its collation, while Comet's native greatest compares raw bytes
+## IntegralDivide
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Decimal division with negative scale is not supported natively
+
## Least
For applicable cases that are not selected for native execution automatically, `Least` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.Least.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.2/cast.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.2/cast.md.txt
index cc8cfbbbd6..11f858012f 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.2/cast.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.2/cast.md.txt
@@ -165,6 +165,24 @@ Spark's behavior of using Java `BigDecimal.toString()` semantics, which produces
notation (e.g. a value of 12300 stored as `Decimal(7,-2)` with unscaled value 123 is rendered
as `"1.23E+4"`).
+## Decimal with Negative Scale to Integer and Timestamp
+
+Casting a `DecimalType` with a negative scale to `ByteType`, `ShortType`, `IntegerType`,
+`LongType` or `TimestampType` has no native implementation, and neither does casting any of the
+integer types to a negative-scale `DecimalType`. Comet reports these as unsupported, which
+routes them through the JVM codegen dispatcher (Spark's own generated code) inside the Comet
+pipeline, so the results match Spark exactly.
+
+The native kernels align scale by computing `10^|scale|` in fixed-point arithmetic, which
+overflows when the scale is negative. See
+[#5013](https://github.com/apache/datafusion-comet/issues/5013).
+
+All other cast directions to and from negative-scale decimals run natively: `FloatType` and
+`DoubleType`, `StringType` in both directions (subject to the config gate described above),
+casts from a negative-scale decimal to `BooleanType`, and widening to another `DecimalType`.
+Casts from `BooleanType` to a `DecimalType` route through the dispatcher at any scale, which is
+not specific to a negative one.
+
## Complex Types
Comet casts arrays, structs, and maps by casting each element, field, key, and value with the
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.2/math.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.2/math.md.txt
index 3191b91640..86d8c635dc 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.2/math.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.2/math.md.txt
@@ -27,12 +27,24 @@ The following cases have no native implementation and always run in the JVM usin
- `INTERVAL YEAR TO MONTH` and `INTERVAL DAY TO SECOND` inputs
+## Divide
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Decimal division with negative scale is not supported natively
+
## Greatest
For applicable cases that are not selected for native execution automatically, `Greatest` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.Greatest.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
- Spark evaluates non-UTF8_BINARY collated string input under its collation, while Comet's native greatest compares raw bytes
+## IntegralDivide
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Decimal division with negative scale is not supported natively
+
## Least
For applicable cases that are not selected for native execution automatically, `Least` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.Least.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
diff --git a/searchindex.js b/searchindex.js
index 8c108ab0c1..204901f629 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[63, "id1"]], "%": [[61, "id1"]], "&": [[51, "id1"]], "*": [[61, "id2"]], "+": [[61, "id3"]], "-": [[61, "id4"]], "/": [[61, "id5"]], "1. Format Your Code": [[48, "format-your-code"]], "1. Install Comet": [[70, "install-comet"], [82, "install-comet"]], "1. Native Operators (nativeExecs map)": [[31, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[48, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[70, "clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[63, "id1"]], "%": [[61, "id1"]], "&": [[51, "id1"]], "*": [[61, "id2"]], "+": [[61, "id3"]], "-": [[61, "id4"]], "/": [[61, "id5"]], "1. Format Your Code": [[48, "format-your-code"]], "1. Install Comet": [[70, "install-comet"], [82, "install-comet"]], "1. Native Operators (nativeExecs map)": [[31, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[48, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[70, "clone-iceberg-and-apply- [...]
\ No newline at end of file
diff --git a/user-guide/latest/compatibility/expressions/spark-3.4/cast.html b/user-guide/latest/compatibility/expressions/spark-3.4/cast.html
index 541ff0ff71..0d550bf378 100644
--- a/user-guide/latest/compatibility/expressions/spark-3.4/cast.html
+++ b/user-guide/latest/compatibility/expressions/spark-3.4/cast.html
@@ -1032,6 +1032,22 @@ Spark’s behavior of using Java <code class="docutils literal notranslate"><spa
notation (e.g. a value of 12300 stored as <code class="docutils literal notranslate"><span class="pre">Decimal(7,-2)</span></code> with unscaled value 123 is rendered
as <code class="docutils literal notranslate"><span class="pre">"1.23E+4"</span></code>).</p>
</section>
+<section id="decimal-with-negative-scale-to-integer-and-timestamp">
+<h2>Decimal with Negative Scale to Integer and Timestamp<a class="headerlink" href="#decimal-with-negative-scale-to-integer-and-timestamp" title="Link to this heading">#</a></h2>
+<p>Casting a <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code> with a negative scale to <code class="docutils literal notranslate"><span class="pre">ByteType</span></code>, <code class="docutils literal notranslate"><span class="pre">ShortType</span></code>, <code class="docutils literal notranslate"><span class="pre">IntegerType</span></code>,
+<code class="docutils literal notranslate"><span class="pre">LongType</span></code> or <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> has no native implementation, and neither does casting any of the
+integer types to a negative-scale <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>. Comet reports these as unsupported, which
+routes them through the JVM codegen dispatcher (Spark’s own generated code) inside the Comet
+pipeline, so the results match Spark exactly.</p>
+<p>The native kernels align scale by computing <code class="docutils literal notranslate"><span class="pre">10^|scale|</span></code> in fixed-point arithmetic, which
+overflows when the scale is negative. See
+<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/5013">#5013</a>.</p>
+<p>All other cast directions to and from negative-scale decimals run natively: <code class="docutils literal notranslate"><span class="pre">FloatType</span></code> and
+<code class="docutils literal notranslate"><span class="pre">DoubleType</span></code>, <code class="docutils literal notranslate"><span class="pre">StringType</span></code> in both directions (subject to the config gate described above),
+casts from a negative-scale decimal to <code class="docutils literal notranslate"><span class="pre">BooleanType</span></code>, and widening to another <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>.
+Casts from <code class="docutils literal notranslate"><span class="pre">BooleanType</span></code> to a <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code> route through the dispatcher at any scale, which is
+not specific to a negative one.</p>
+</section>
<section id="complex-types">
<h2>Complex Types<a class="headerlink" href="#complex-types" title="Link to this heading">#</a></h2>
<p>Comet casts arrays, structs, and maps by casting each element, field, key, and value with the
@@ -1855,6 +1871,7 @@ or <code class="docutils literal notranslate"><span class="pre">LongType</span><
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#timestampntz-casts">TimestampNTZ Casts</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#date-to-string">Date to String</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-string">Decimal with Negative Scale to String</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-integer-and-timestamp">Decimal with Negative Scale to Integer and Timestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#complex-types">Complex Types</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-integral-types">Decimal with Negative Scale to Integral Types</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#legacy-mode">Legacy Mode</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-3.4/math.html b/user-guide/latest/compatibility/expressions/spark-3.4/math.html
index 4148be8d51..a6015e8ce8 100644
--- a/user-guide/latest/compatibility/expressions/spark-3.4/math.html
+++ b/user-guide/latest/compatibility/expressions/spark-3.4/math.html
@@ -861,6 +861,13 @@ under the License.
<li><p><code class="docutils literal notranslate"><span class="pre">INTERVAL</span> <span class="pre">YEAR</span> <span class="pre">TO</span> <span class="pre">MONTH</span></code> and <code class="docutils literal notranslate"><span class="pre">INTERVAL</span> <span class="pre">DAY</span> <span class="pre">TO</span> <span class="pre">SECOND</span></code> inputs</p></li>
</ul>
</section>
+<section id="divide">
+<h2>Divide<a class="headerlink" href="#divide" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet and always fall back to Spark, regardless of any <code class="docutils literal notranslate"><span class="pre">allowIncompatible</span></code> setting:</p>
+<ul class="simple">
+<li><p>Decimal division with negative scale is not supported natively</p></li>
+</ul>
+</section>
<section id="greatest">
<h2>Greatest<a class="headerlink" href="#greatest" title="Link to this heading">#</a></h2>
<p>For applicable cases that are not selected for native execution automatically, <code class="docutils literal notranslate"><span class="pre">Greatest</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline) by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Greatest.allowIncompatible=true</span></code> to explicitly select Comet’s native implementation, which has the following [...]
@@ -868,6 +875,13 @@ under the License.
<li><p>Spark evaluates non-UTF8_BINARY collated string input under its collation, while Comet’s native greatest compares raw bytes</p></li>
</ul>
</section>
+<section id="integraldivide">
+<h2>IntegralDivide<a class="headerlink" href="#integraldivide" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet and always fall back to Spark, regardless of any <code class="docutils literal notranslate"><span class="pre">allowIncompatible</span></code> setting:</p>
+<ul class="simple">
+<li><p>Decimal division with negative scale is not supported natively</p></li>
+</ul>
+</section>
<section id="least">
<h2>Least<a class="headerlink" href="#least" title="Link to this heading">#</a></h2>
<p>For applicable cases that are not selected for native execution automatically, <code class="docutils literal notranslate"><span class="pre">Least</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline) by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Least.allowIncompatible=true</span></code> to explicitly select Comet’s native implementation, which has the following differ [...]
@@ -955,7 +969,9 @@ under the License.
<nav class="bd-toc-nav page-toc" aria-labelledby="pst-page-navigation-heading-2">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#abs">Abs</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#divide">Divide</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#greatest">Greatest</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#integraldivide">IntegralDivide</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#least">Least</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#rand">Rand</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#randn">Randn</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-3.5/cast.html b/user-guide/latest/compatibility/expressions/spark-3.5/cast.html
index 153f7dd941..f57d7ab768 100644
--- a/user-guide/latest/compatibility/expressions/spark-3.5/cast.html
+++ b/user-guide/latest/compatibility/expressions/spark-3.5/cast.html
@@ -1032,6 +1032,22 @@ Spark’s behavior of using Java <code class="docutils literal notranslate"><spa
notation (e.g. a value of 12300 stored as <code class="docutils literal notranslate"><span class="pre">Decimal(7,-2)</span></code> with unscaled value 123 is rendered
as <code class="docutils literal notranslate"><span class="pre">"1.23E+4"</span></code>).</p>
</section>
+<section id="decimal-with-negative-scale-to-integer-and-timestamp">
+<h2>Decimal with Negative Scale to Integer and Timestamp<a class="headerlink" href="#decimal-with-negative-scale-to-integer-and-timestamp" title="Link to this heading">#</a></h2>
+<p>Casting a <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code> with a negative scale to <code class="docutils literal notranslate"><span class="pre">ByteType</span></code>, <code class="docutils literal notranslate"><span class="pre">ShortType</span></code>, <code class="docutils literal notranslate"><span class="pre">IntegerType</span></code>,
+<code class="docutils literal notranslate"><span class="pre">LongType</span></code> or <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> has no native implementation, and neither does casting any of the
+integer types to a negative-scale <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>. Comet reports these as unsupported, which
+routes them through the JVM codegen dispatcher (Spark’s own generated code) inside the Comet
+pipeline, so the results match Spark exactly.</p>
+<p>The native kernels align scale by computing <code class="docutils literal notranslate"><span class="pre">10^|scale|</span></code> in fixed-point arithmetic, which
+overflows when the scale is negative. See
+<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/5013">#5013</a>.</p>
+<p>All other cast directions to and from negative-scale decimals run natively: <code class="docutils literal notranslate"><span class="pre">FloatType</span></code> and
+<code class="docutils literal notranslate"><span class="pre">DoubleType</span></code>, <code class="docutils literal notranslate"><span class="pre">StringType</span></code> in both directions (subject to the config gate described above),
+casts from a negative-scale decimal to <code class="docutils literal notranslate"><span class="pre">BooleanType</span></code>, and widening to another <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>.
+Casts from <code class="docutils literal notranslate"><span class="pre">BooleanType</span></code> to a <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code> route through the dispatcher at any scale, which is
+not specific to a negative one.</p>
+</section>
<section id="complex-types">
<h2>Complex Types<a class="headerlink" href="#complex-types" title="Link to this heading">#</a></h2>
<p>Comet casts arrays, structs, and maps by casting each element, field, key, and value with the
@@ -1855,6 +1871,7 @@ or <code class="docutils literal notranslate"><span class="pre">LongType</span><
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#timestampntz-casts">TimestampNTZ Casts</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#date-to-string">Date to String</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-string">Decimal with Negative Scale to String</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-integer-and-timestamp">Decimal with Negative Scale to Integer and Timestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#complex-types">Complex Types</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-integral-types">Decimal with Negative Scale to Integral Types</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#legacy-mode">Legacy Mode</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-3.5/math.html b/user-guide/latest/compatibility/expressions/spark-3.5/math.html
index 994156a532..60dbb060fe 100644
--- a/user-guide/latest/compatibility/expressions/spark-3.5/math.html
+++ b/user-guide/latest/compatibility/expressions/spark-3.5/math.html
@@ -861,6 +861,13 @@ under the License.
<li><p><code class="docutils literal notranslate"><span class="pre">INTERVAL</span> <span class="pre">YEAR</span> <span class="pre">TO</span> <span class="pre">MONTH</span></code> and <code class="docutils literal notranslate"><span class="pre">INTERVAL</span> <span class="pre">DAY</span> <span class="pre">TO</span> <span class="pre">SECOND</span></code> inputs</p></li>
</ul>
</section>
+<section id="divide">
+<h2>Divide<a class="headerlink" href="#divide" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet and always fall back to Spark, regardless of any <code class="docutils literal notranslate"><span class="pre">allowIncompatible</span></code> setting:</p>
+<ul class="simple">
+<li><p>Decimal division with negative scale is not supported natively</p></li>
+</ul>
+</section>
<section id="greatest">
<h2>Greatest<a class="headerlink" href="#greatest" title="Link to this heading">#</a></h2>
<p>For applicable cases that are not selected for native execution automatically, <code class="docutils literal notranslate"><span class="pre">Greatest</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline) by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Greatest.allowIncompatible=true</span></code> to explicitly select Comet’s native implementation, which has the following [...]
@@ -868,6 +875,13 @@ under the License.
<li><p>Spark evaluates non-UTF8_BINARY collated string input under its collation, while Comet’s native greatest compares raw bytes</p></li>
</ul>
</section>
+<section id="integraldivide">
+<h2>IntegralDivide<a class="headerlink" href="#integraldivide" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet and always fall back to Spark, regardless of any <code class="docutils literal notranslate"><span class="pre">allowIncompatible</span></code> setting:</p>
+<ul class="simple">
+<li><p>Decimal division with negative scale is not supported natively</p></li>
+</ul>
+</section>
<section id="least">
<h2>Least<a class="headerlink" href="#least" title="Link to this heading">#</a></h2>
<p>For applicable cases that are not selected for native execution automatically, <code class="docutils literal notranslate"><span class="pre">Least</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline) by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Least.allowIncompatible=true</span></code> to explicitly select Comet’s native implementation, which has the following differ [...]
@@ -955,7 +969,9 @@ under the License.
<nav class="bd-toc-nav page-toc" aria-labelledby="pst-page-navigation-heading-2">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#abs">Abs</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#divide">Divide</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#greatest">Greatest</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#integraldivide">IntegralDivide</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#least">Least</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#rand">Rand</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#randn">Randn</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.0/cast.html b/user-guide/latest/compatibility/expressions/spark-4.0/cast.html
index bec2589b87..b2eb292afc 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.0/cast.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.0/cast.html
@@ -1032,6 +1032,22 @@ Spark’s behavior of using Java <code class="docutils literal notranslate"><spa
notation (e.g. a value of 12300 stored as <code class="docutils literal notranslate"><span class="pre">Decimal(7,-2)</span></code> with unscaled value 123 is rendered
as <code class="docutils literal notranslate"><span class="pre">"1.23E+4"</span></code>).</p>
</section>
+<section id="decimal-with-negative-scale-to-integer-and-timestamp">
+<h2>Decimal with Negative Scale to Integer and Timestamp<a class="headerlink" href="#decimal-with-negative-scale-to-integer-and-timestamp" title="Link to this heading">#</a></h2>
+<p>Casting a <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code> with a negative scale to <code class="docutils literal notranslate"><span class="pre">ByteType</span></code>, <code class="docutils literal notranslate"><span class="pre">ShortType</span></code>, <code class="docutils literal notranslate"><span class="pre">IntegerType</span></code>,
+<code class="docutils literal notranslate"><span class="pre">LongType</span></code> or <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> has no native implementation, and neither does casting any of the
+integer types to a negative-scale <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>. Comet reports these as unsupported, which
+routes them through the JVM codegen dispatcher (Spark’s own generated code) inside the Comet
+pipeline, so the results match Spark exactly.</p>
+<p>The native kernels align scale by computing <code class="docutils literal notranslate"><span class="pre">10^|scale|</span></code> in fixed-point arithmetic, which
+overflows when the scale is negative. See
+<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/5013">#5013</a>.</p>
+<p>All other cast directions to and from negative-scale decimals run natively: <code class="docutils literal notranslate"><span class="pre">FloatType</span></code> and
+<code class="docutils literal notranslate"><span class="pre">DoubleType</span></code>, <code class="docutils literal notranslate"><span class="pre">StringType</span></code> in both directions (subject to the config gate described above),
+casts from a negative-scale decimal to <code class="docutils literal notranslate"><span class="pre">BooleanType</span></code>, and widening to another <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>.
+Casts from <code class="docutils literal notranslate"><span class="pre">BooleanType</span></code> to a <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code> route through the dispatcher at any scale, which is
+not specific to a negative one.</p>
+</section>
<section id="complex-types">
<h2>Complex Types<a class="headerlink" href="#complex-types" title="Link to this heading">#</a></h2>
<p>Comet casts arrays, structs, and maps by casting each element, field, key, and value with the
@@ -1855,6 +1871,7 @@ or <code class="docutils literal notranslate"><span class="pre">LongType</span><
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#timestampntz-casts">TimestampNTZ Casts</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#date-to-string">Date to String</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-string">Decimal with Negative Scale to String</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-integer-and-timestamp">Decimal with Negative Scale to Integer and Timestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#complex-types">Complex Types</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-integral-types">Decimal with Negative Scale to Integral Types</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#legacy-mode">Legacy Mode</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.0/math.html b/user-guide/latest/compatibility/expressions/spark-4.0/math.html
index ce2dbe2e27..d4b679f3c3 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.0/math.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.0/math.html
@@ -861,6 +861,13 @@ under the License.
<li><p><code class="docutils literal notranslate"><span class="pre">INTERVAL</span> <span class="pre">YEAR</span> <span class="pre">TO</span> <span class="pre">MONTH</span></code> and <code class="docutils literal notranslate"><span class="pre">INTERVAL</span> <span class="pre">DAY</span> <span class="pre">TO</span> <span class="pre">SECOND</span></code> inputs</p></li>
</ul>
</section>
+<section id="divide">
+<h2>Divide<a class="headerlink" href="#divide" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet and always fall back to Spark, regardless of any <code class="docutils literal notranslate"><span class="pre">allowIncompatible</span></code> setting:</p>
+<ul class="simple">
+<li><p>Decimal division with negative scale is not supported natively</p></li>
+</ul>
+</section>
<section id="greatest">
<h2>Greatest<a class="headerlink" href="#greatest" title="Link to this heading">#</a></h2>
<p>For applicable cases that are not selected for native execution automatically, <code class="docutils literal notranslate"><span class="pre">Greatest</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline) by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Greatest.allowIncompatible=true</span></code> to explicitly select Comet’s native implementation, which has the following [...]
@@ -868,6 +875,13 @@ under the License.
<li><p>Spark evaluates non-UTF8_BINARY collated string input under its collation, while Comet’s native greatest compares raw bytes</p></li>
</ul>
</section>
+<section id="integraldivide">
+<h2>IntegralDivide<a class="headerlink" href="#integraldivide" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet and always fall back to Spark, regardless of any <code class="docutils literal notranslate"><span class="pre">allowIncompatible</span></code> setting:</p>
+<ul class="simple">
+<li><p>Decimal division with negative scale is not supported natively</p></li>
+</ul>
+</section>
<section id="least">
<h2>Least<a class="headerlink" href="#least" title="Link to this heading">#</a></h2>
<p>For applicable cases that are not selected for native execution automatically, <code class="docutils literal notranslate"><span class="pre">Least</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline) by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Least.allowIncompatible=true</span></code> to explicitly select Comet’s native implementation, which has the following differ [...]
@@ -955,7 +969,9 @@ under the License.
<nav class="bd-toc-nav page-toc" aria-labelledby="pst-page-navigation-heading-2">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#abs">Abs</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#divide">Divide</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#greatest">Greatest</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#integraldivide">IntegralDivide</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#least">Least</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#rand">Rand</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#randn">Randn</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.1/cast.html b/user-guide/latest/compatibility/expressions/spark-4.1/cast.html
index fc51ae6ecb..e2132e6640 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.1/cast.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.1/cast.html
@@ -1032,6 +1032,22 @@ Spark’s behavior of using Java <code class="docutils literal notranslate"><spa
notation (e.g. a value of 12300 stored as <code class="docutils literal notranslate"><span class="pre">Decimal(7,-2)</span></code> with unscaled value 123 is rendered
as <code class="docutils literal notranslate"><span class="pre">"1.23E+4"</span></code>).</p>
</section>
+<section id="decimal-with-negative-scale-to-integer-and-timestamp">
+<h2>Decimal with Negative Scale to Integer and Timestamp<a class="headerlink" href="#decimal-with-negative-scale-to-integer-and-timestamp" title="Link to this heading">#</a></h2>
+<p>Casting a <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code> with a negative scale to <code class="docutils literal notranslate"><span class="pre">ByteType</span></code>, <code class="docutils literal notranslate"><span class="pre">ShortType</span></code>, <code class="docutils literal notranslate"><span class="pre">IntegerType</span></code>,
+<code class="docutils literal notranslate"><span class="pre">LongType</span></code> or <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> has no native implementation, and neither does casting any of the
+integer types to a negative-scale <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>. Comet reports these as unsupported, which
+routes them through the JVM codegen dispatcher (Spark’s own generated code) inside the Comet
+pipeline, so the results match Spark exactly.</p>
+<p>The native kernels align scale by computing <code class="docutils literal notranslate"><span class="pre">10^|scale|</span></code> in fixed-point arithmetic, which
+overflows when the scale is negative. See
+<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/5013">#5013</a>.</p>
+<p>All other cast directions to and from negative-scale decimals run natively: <code class="docutils literal notranslate"><span class="pre">FloatType</span></code> and
+<code class="docutils literal notranslate"><span class="pre">DoubleType</span></code>, <code class="docutils literal notranslate"><span class="pre">StringType</span></code> in both directions (subject to the config gate described above),
+casts from a negative-scale decimal to <code class="docutils literal notranslate"><span class="pre">BooleanType</span></code>, and widening to another <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>.
+Casts from <code class="docutils literal notranslate"><span class="pre">BooleanType</span></code> to a <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code> route through the dispatcher at any scale, which is
+not specific to a negative one.</p>
+</section>
<section id="complex-types">
<h2>Complex Types<a class="headerlink" href="#complex-types" title="Link to this heading">#</a></h2>
<p>Comet casts arrays, structs, and maps by casting each element, field, key, and value with the
@@ -1855,6 +1871,7 @@ or <code class="docutils literal notranslate"><span class="pre">LongType</span><
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#timestampntz-casts">TimestampNTZ Casts</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#date-to-string">Date to String</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-string">Decimal with Negative Scale to String</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-integer-and-timestamp">Decimal with Negative Scale to Integer and Timestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#complex-types">Complex Types</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-integral-types">Decimal with Negative Scale to Integral Types</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#legacy-mode">Legacy Mode</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.1/math.html b/user-guide/latest/compatibility/expressions/spark-4.1/math.html
index d83d39535c..6b001f75d5 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.1/math.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.1/math.html
@@ -861,6 +861,13 @@ under the License.
<li><p><code class="docutils literal notranslate"><span class="pre">INTERVAL</span> <span class="pre">YEAR</span> <span class="pre">TO</span> <span class="pre">MONTH</span></code> and <code class="docutils literal notranslate"><span class="pre">INTERVAL</span> <span class="pre">DAY</span> <span class="pre">TO</span> <span class="pre">SECOND</span></code> inputs</p></li>
</ul>
</section>
+<section id="divide">
+<h2>Divide<a class="headerlink" href="#divide" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet and always fall back to Spark, regardless of any <code class="docutils literal notranslate"><span class="pre">allowIncompatible</span></code> setting:</p>
+<ul class="simple">
+<li><p>Decimal division with negative scale is not supported natively</p></li>
+</ul>
+</section>
<section id="greatest">
<h2>Greatest<a class="headerlink" href="#greatest" title="Link to this heading">#</a></h2>
<p>For applicable cases that are not selected for native execution automatically, <code class="docutils literal notranslate"><span class="pre">Greatest</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline) by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Greatest.allowIncompatible=true</span></code> to explicitly select Comet’s native implementation, which has the following [...]
@@ -868,6 +875,13 @@ under the License.
<li><p>Spark evaluates non-UTF8_BINARY collated string input under its collation, while Comet’s native greatest compares raw bytes</p></li>
</ul>
</section>
+<section id="integraldivide">
+<h2>IntegralDivide<a class="headerlink" href="#integraldivide" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet and always fall back to Spark, regardless of any <code class="docutils literal notranslate"><span class="pre">allowIncompatible</span></code> setting:</p>
+<ul class="simple">
+<li><p>Decimal division with negative scale is not supported natively</p></li>
+</ul>
+</section>
<section id="least">
<h2>Least<a class="headerlink" href="#least" title="Link to this heading">#</a></h2>
<p>For applicable cases that are not selected for native execution automatically, <code class="docutils literal notranslate"><span class="pre">Least</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline) by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Least.allowIncompatible=true</span></code> to explicitly select Comet’s native implementation, which has the following differ [...]
@@ -955,7 +969,9 @@ under the License.
<nav class="bd-toc-nav page-toc" aria-labelledby="pst-page-navigation-heading-2">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#abs">Abs</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#divide">Divide</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#greatest">Greatest</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#integraldivide">IntegralDivide</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#least">Least</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#rand">Rand</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#randn">Randn</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.2/cast.html b/user-guide/latest/compatibility/expressions/spark-4.2/cast.html
index 68e75456a6..719bb24f10 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.2/cast.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.2/cast.html
@@ -1032,6 +1032,22 @@ Spark’s behavior of using Java <code class="docutils literal notranslate"><spa
notation (e.g. a value of 12300 stored as <code class="docutils literal notranslate"><span class="pre">Decimal(7,-2)</span></code> with unscaled value 123 is rendered
as <code class="docutils literal notranslate"><span class="pre">"1.23E+4"</span></code>).</p>
</section>
+<section id="decimal-with-negative-scale-to-integer-and-timestamp">
+<h2>Decimal with Negative Scale to Integer and Timestamp<a class="headerlink" href="#decimal-with-negative-scale-to-integer-and-timestamp" title="Link to this heading">#</a></h2>
+<p>Casting a <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code> with a negative scale to <code class="docutils literal notranslate"><span class="pre">ByteType</span></code>, <code class="docutils literal notranslate"><span class="pre">ShortType</span></code>, <code class="docutils literal notranslate"><span class="pre">IntegerType</span></code>,
+<code class="docutils literal notranslate"><span class="pre">LongType</span></code> or <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> has no native implementation, and neither does casting any of the
+integer types to a negative-scale <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>. Comet reports these as unsupported, which
+routes them through the JVM codegen dispatcher (Spark’s own generated code) inside the Comet
+pipeline, so the results match Spark exactly.</p>
+<p>The native kernels align scale by computing <code class="docutils literal notranslate"><span class="pre">10^|scale|</span></code> in fixed-point arithmetic, which
+overflows when the scale is negative. See
+<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/5013">#5013</a>.</p>
+<p>All other cast directions to and from negative-scale decimals run natively: <code class="docutils literal notranslate"><span class="pre">FloatType</span></code> and
+<code class="docutils literal notranslate"><span class="pre">DoubleType</span></code>, <code class="docutils literal notranslate"><span class="pre">StringType</span></code> in both directions (subject to the config gate described above),
+casts from a negative-scale decimal to <code class="docutils literal notranslate"><span class="pre">BooleanType</span></code>, and widening to another <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>.
+Casts from <code class="docutils literal notranslate"><span class="pre">BooleanType</span></code> to a <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code> route through the dispatcher at any scale, which is
+not specific to a negative one.</p>
+</section>
<section id="complex-types">
<h2>Complex Types<a class="headerlink" href="#complex-types" title="Link to this heading">#</a></h2>
<p>Comet casts arrays, structs, and maps by casting each element, field, key, and value with the
@@ -1855,6 +1871,7 @@ or <code class="docutils literal notranslate"><span class="pre">LongType</span><
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#timestampntz-casts">TimestampNTZ Casts</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#date-to-string">Date to String</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-string">Decimal with Negative Scale to String</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-integer-and-timestamp">Decimal with Negative Scale to Integer and Timestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#complex-types">Complex Types</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#decimal-with-negative-scale-to-integral-types">Decimal with Negative Scale to Integral Types</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#legacy-mode">Legacy Mode</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.2/math.html b/user-guide/latest/compatibility/expressions/spark-4.2/math.html
index 2f435defec..a29100fc6d 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.2/math.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.2/math.html
@@ -861,6 +861,13 @@ under the License.
<li><p><code class="docutils literal notranslate"><span class="pre">INTERVAL</span> <span class="pre">YEAR</span> <span class="pre">TO</span> <span class="pre">MONTH</span></code> and <code class="docutils literal notranslate"><span class="pre">INTERVAL</span> <span class="pre">DAY</span> <span class="pre">TO</span> <span class="pre">SECOND</span></code> inputs</p></li>
</ul>
</section>
+<section id="divide">
+<h2>Divide<a class="headerlink" href="#divide" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet and always fall back to Spark, regardless of any <code class="docutils literal notranslate"><span class="pre">allowIncompatible</span></code> setting:</p>
+<ul class="simple">
+<li><p>Decimal division with negative scale is not supported natively</p></li>
+</ul>
+</section>
<section id="greatest">
<h2>Greatest<a class="headerlink" href="#greatest" title="Link to this heading">#</a></h2>
<p>For applicable cases that are not selected for native execution automatically, <code class="docutils literal notranslate"><span class="pre">Greatest</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline) by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Greatest.allowIncompatible=true</span></code> to explicitly select Comet’s native implementation, which has the following [...]
@@ -868,6 +875,13 @@ under the License.
<li><p>Spark evaluates non-UTF8_BINARY collated string input under its collation, while Comet’s native greatest compares raw bytes</p></li>
</ul>
</section>
+<section id="integraldivide">
+<h2>IntegralDivide<a class="headerlink" href="#integraldivide" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet and always fall back to Spark, regardless of any <code class="docutils literal notranslate"><span class="pre">allowIncompatible</span></code> setting:</p>
+<ul class="simple">
+<li><p>Decimal division with negative scale is not supported natively</p></li>
+</ul>
+</section>
<section id="least">
<h2>Least<a class="headerlink" href="#least" title="Link to this heading">#</a></h2>
<p>For applicable cases that are not selected for native execution automatically, <code class="docutils literal notranslate"><span class="pre">Least</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline) by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Least.allowIncompatible=true</span></code> to explicitly select Comet’s native implementation, which has the following differ [...]
@@ -955,7 +969,9 @@ under the License.
<nav class="bd-toc-nav page-toc" aria-labelledby="pst-page-navigation-heading-2">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#abs">Abs</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#divide">Divide</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#greatest">Greatest</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#integraldivide">IntegralDivide</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#least">Least</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#rand">Rand</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#randn">Randn</a></li>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org