You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/06/12 00:38:39 UTC
(datafusion-comet) branch asf-site updated: Publish built docs triggered by 03c1478bd205ab31e5c9425d15cf062a7e877453
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 60f36e4dee Publish built docs triggered by 03c1478bd205ab31e5c9425d15cf062a7e877453
60f36e4dee is described below
commit 60f36e4dee15facbe3c481d755be748b69726402
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Fri Jun 12 00:38:35 2026 +0000
Publish built docs triggered by 03c1478bd205ab31e5c9425d15cf062a7e877453
---
.../expressions/spark-3.4/datetime.md.txt | 23 +++++++++++++++++++
.../expressions/spark-3.4/string.md.txt | 18 +++++++++++++++
.../expressions/spark-3.5/datetime.md.txt | 23 +++++++++++++++++++
.../expressions/spark-3.5/string.md.txt | 18 +++++++++++++++
.../expressions/spark-4.0/datetime.md.txt | 23 +++++++++++++++++++
.../expressions/spark-4.0/string.md.txt | 18 +++++++++++++++
.../expressions/spark-4.1/datetime.md.txt | 23 +++++++++++++++++++
.../expressions/spark-4.1/string.md.txt | 18 +++++++++++++++
_sources/user-guide/latest/expressions.md.txt | 2 +-
searchindex.js | 2 +-
.../expressions/spark-3.4/datetime.html | 26 ++++++++++++++++++++++
.../expressions/spark-3.4/string.html | 21 +++++++++++++++++
.../expressions/spark-3.5/datetime.html | 26 ++++++++++++++++++++++
.../expressions/spark-3.5/string.html | 21 +++++++++++++++++
.../expressions/spark-4.0/datetime.html | 26 ++++++++++++++++++++++
.../expressions/spark-4.0/string.html | 21 +++++++++++++++++
.../expressions/spark-4.1/datetime.html | 26 ++++++++++++++++++++++
.../expressions/spark-4.1/string.html | 21 +++++++++++++++++
user-guide/latest/expressions.html | 2 +-
19 files changed, 355 insertions(+), 3 deletions(-)
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-3.4/datetime.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-3.4/datetime.md.txt
index 5471aba19f..2d70dbd7fb 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-3.4/datetime.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-3.4/datetime.md.txt
@@ -56,6 +56,12 @@ The following differences from Spark are always present and do not require any a
- Format strings in a curated allow-list run natively via DataFusion's `to_char` for UTC sessions. Other format strings (including non-literal formats), as well as non-UTC sessions, route through Spark's own `DateFormatClass.doGenCode` via the Arrow-direct codegen dispatcher when `spark.comet.exec.scalaUDF.codegen.enabled=true`. When the codegen dispatcher is disabled (default) the operator falls back to Spark in those cases.
+## Days
+
+The following cases are not supported by Comet:
+
+- Only `DateType` and `TimestampType` inputs are supported. `TimestampNTZType` is not supported.
+
## FromUTCTimestamp
The following incompatibilities cause `FromUTCTimestamp` to fall back to Spark by default. Set `spark.comet.expression.FromUTCTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -74,6 +80,12 @@ The following incompatibilities cause `Hour` to fall back to Spark by default. S
- Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)
+## Hours
+
+The following cases are not supported by Comet:
+
+- Only `TimestampType` and `TimestampNTZType` inputs are supported.
+
## Minute
The following incompatibilities cause `Minute` to fall back to Spark by default. Set `spark.comet.expression.Minute.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -86,6 +98,12 @@ The following incompatibilities cause `Second` to fall back to Spark by default.
- Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)
+## SecondsToTimestamp
+
+The following cases are not supported by Comet:
+
+- Only `IntegerType`, `LongType`, `FloatType`, and `DoubleType` inputs are supported. `DecimalType`, `ByteType`, and `ShortType` fall back to Spark.
+
## ToUTCTimestamp
The following incompatibilities cause `ToUTCTimestamp` to fall back to Spark by default. Set `spark.comet.expression.ToUTCTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -107,6 +125,11 @@ The following cases are not supported by Comet:
The following incompatibilities cause `TruncTimestamp` to fall back to Spark by default. Set `spark.comet.expression.TruncTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
- Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)
+- Non-literal format strings will throw an exception instead of returning NULL
+
+The following cases are not supported by Comet:
+
+- Only the following formats are supported: year, yyyy, yy, quarter, mon, month, mm, week, day, dd, hour, minute, second, millisecond, microsecond
## UnixTimestamp
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-3.4/string.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-3.4/string.md.txt
index c442066282..0c0ad254f5 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-3.4/string.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-3.4/string.md.txt
@@ -21,6 +21,12 @@ under the License.
<!--BEGIN:EXPR_COMPAT[string]-->
+## BitLength
+
+The following cases are not supported by Comet:
+
+- `BinaryType` input is not supported
+
## Concat
The following incompatibilities cause `Concat` to fall back to Spark by default. Set `spark.comet.expression.Concat.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -44,6 +50,12 @@ The following cases are not supported by Comet:
- `BinaryType` input is not supported
+## OctetLength
+
+The following cases are not supported by Comet:
+
+- `BinaryType` input is not supported
+
## Reverse
The following incompatibilities cause `Reverse` to fall back to Spark by default. Set `spark.comet.expression.Reverse.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -82,4 +94,10 @@ The following cases are not supported by Comet:
The following differences from Spark are always present and do not require any additional configuration:
- A negative argument for the number of times to repeat throws an exception instead of returning an empty string as Spark does
+
+## StringTranslate
+
+The following incompatibilities cause `StringTranslate` to fall back to Spark by default. Set `spark.comet.expression.StringTranslate.allowIncompatible=true` to enable Comet acceleration despite these differences.
+
+- DataFusion's translate iterates over Unicode graphemes (Spark uses code points) and substitutes U+0000 instead of treating it as a deletion sentinel
<!--END:EXPR_COMPAT-->
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-3.5/datetime.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-3.5/datetime.md.txt
index 5471aba19f..2d70dbd7fb 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-3.5/datetime.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-3.5/datetime.md.txt
@@ -56,6 +56,12 @@ The following differences from Spark are always present and do not require any a
- Format strings in a curated allow-list run natively via DataFusion's `to_char` for UTC sessions. Other format strings (including non-literal formats), as well as non-UTC sessions, route through Spark's own `DateFormatClass.doGenCode` via the Arrow-direct codegen dispatcher when `spark.comet.exec.scalaUDF.codegen.enabled=true`. When the codegen dispatcher is disabled (default) the operator falls back to Spark in those cases.
+## Days
+
+The following cases are not supported by Comet:
+
+- Only `DateType` and `TimestampType` inputs are supported. `TimestampNTZType` is not supported.
+
## FromUTCTimestamp
The following incompatibilities cause `FromUTCTimestamp` to fall back to Spark by default. Set `spark.comet.expression.FromUTCTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -74,6 +80,12 @@ The following incompatibilities cause `Hour` to fall back to Spark by default. S
- Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)
+## Hours
+
+The following cases are not supported by Comet:
+
+- Only `TimestampType` and `TimestampNTZType` inputs are supported.
+
## Minute
The following incompatibilities cause `Minute` to fall back to Spark by default. Set `spark.comet.expression.Minute.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -86,6 +98,12 @@ The following incompatibilities cause `Second` to fall back to Spark by default.
- Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)
+## SecondsToTimestamp
+
+The following cases are not supported by Comet:
+
+- Only `IntegerType`, `LongType`, `FloatType`, and `DoubleType` inputs are supported. `DecimalType`, `ByteType`, and `ShortType` fall back to Spark.
+
## ToUTCTimestamp
The following incompatibilities cause `ToUTCTimestamp` to fall back to Spark by default. Set `spark.comet.expression.ToUTCTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -107,6 +125,11 @@ The following cases are not supported by Comet:
The following incompatibilities cause `TruncTimestamp` to fall back to Spark by default. Set `spark.comet.expression.TruncTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
- Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)
+- Non-literal format strings will throw an exception instead of returning NULL
+
+The following cases are not supported by Comet:
+
+- Only the following formats are supported: year, yyyy, yy, quarter, mon, month, mm, week, day, dd, hour, minute, second, millisecond, microsecond
## UnixTimestamp
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-3.5/string.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-3.5/string.md.txt
index c442066282..0c0ad254f5 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-3.5/string.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-3.5/string.md.txt
@@ -21,6 +21,12 @@ under the License.
<!--BEGIN:EXPR_COMPAT[string]-->
+## BitLength
+
+The following cases are not supported by Comet:
+
+- `BinaryType` input is not supported
+
## Concat
The following incompatibilities cause `Concat` to fall back to Spark by default. Set `spark.comet.expression.Concat.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -44,6 +50,12 @@ The following cases are not supported by Comet:
- `BinaryType` input is not supported
+## OctetLength
+
+The following cases are not supported by Comet:
+
+- `BinaryType` input is not supported
+
## Reverse
The following incompatibilities cause `Reverse` to fall back to Spark by default. Set `spark.comet.expression.Reverse.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -82,4 +94,10 @@ The following cases are not supported by Comet:
The following differences from Spark are always present and do not require any additional configuration:
- A negative argument for the number of times to repeat throws an exception instead of returning an empty string as Spark does
+
+## StringTranslate
+
+The following incompatibilities cause `StringTranslate` to fall back to Spark by default. Set `spark.comet.expression.StringTranslate.allowIncompatible=true` to enable Comet acceleration despite these differences.
+
+- DataFusion's translate iterates over Unicode graphemes (Spark uses code points) and substitutes U+0000 instead of treating it as a deletion sentinel
<!--END:EXPR_COMPAT-->
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/datetime.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/datetime.md.txt
index 5471aba19f..2d70dbd7fb 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/datetime.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/datetime.md.txt
@@ -56,6 +56,12 @@ The following differences from Spark are always present and do not require any a
- Format strings in a curated allow-list run natively via DataFusion's `to_char` for UTC sessions. Other format strings (including non-literal formats), as well as non-UTC sessions, route through Spark's own `DateFormatClass.doGenCode` via the Arrow-direct codegen dispatcher when `spark.comet.exec.scalaUDF.codegen.enabled=true`. When the codegen dispatcher is disabled (default) the operator falls back to Spark in those cases.
+## Days
+
+The following cases are not supported by Comet:
+
+- Only `DateType` and `TimestampType` inputs are supported. `TimestampNTZType` is not supported.
+
## FromUTCTimestamp
The following incompatibilities cause `FromUTCTimestamp` to fall back to Spark by default. Set `spark.comet.expression.FromUTCTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -74,6 +80,12 @@ The following incompatibilities cause `Hour` to fall back to Spark by default. S
- Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)
+## Hours
+
+The following cases are not supported by Comet:
+
+- Only `TimestampType` and `TimestampNTZType` inputs are supported.
+
## Minute
The following incompatibilities cause `Minute` to fall back to Spark by default. Set `spark.comet.expression.Minute.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -86,6 +98,12 @@ The following incompatibilities cause `Second` to fall back to Spark by default.
- Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)
+## SecondsToTimestamp
+
+The following cases are not supported by Comet:
+
+- Only `IntegerType`, `LongType`, `FloatType`, and `DoubleType` inputs are supported. `DecimalType`, `ByteType`, and `ShortType` fall back to Spark.
+
## ToUTCTimestamp
The following incompatibilities cause `ToUTCTimestamp` to fall back to Spark by default. Set `spark.comet.expression.ToUTCTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -107,6 +125,11 @@ The following cases are not supported by Comet:
The following incompatibilities cause `TruncTimestamp` to fall back to Spark by default. Set `spark.comet.expression.TruncTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
- Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)
+- Non-literal format strings will throw an exception instead of returning NULL
+
+The following cases are not supported by Comet:
+
+- Only the following formats are supported: year, yyyy, yy, quarter, mon, month, mm, week, day, dd, hour, minute, second, millisecond, microsecond
## UnixTimestamp
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/string.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/string.md.txt
index 329f99312f..b407c1c19c 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/string.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/string.md.txt
@@ -21,6 +21,12 @@ under the License.
<!--BEGIN:EXPR_COMPAT[string]-->
+## BitLength
+
+The following cases are not supported by Comet:
+
+- `BinaryType` input is not supported
+
## Concat
The following incompatibilities cause `Concat` to fall back to Spark by default. Set `spark.comet.expression.Concat.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -44,6 +50,12 @@ The following cases are not supported by Comet:
- `BinaryType` input is not supported
+## OctetLength
+
+The following cases are not supported by Comet:
+
+- `BinaryType` input is not supported
+
## Reverse
The following incompatibilities cause `Reverse` to fall back to Spark by default. Set `spark.comet.expression.Reverse.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -76,4 +88,10 @@ The following cases are not supported by Comet:
The following differences from Spark are always present and do not require any additional configuration:
- A negative argument for the number of times to repeat throws an exception instead of returning an empty string as Spark does
+
+## StringTranslate
+
+The following incompatibilities cause `StringTranslate` to fall back to Spark by default. Set `spark.comet.expression.StringTranslate.allowIncompatible=true` to enable Comet acceleration despite these differences.
+
+- DataFusion's translate iterates over Unicode graphemes (Spark uses code points) and substitutes U+0000 instead of treating it as a deletion sentinel
<!--END:EXPR_COMPAT-->
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/datetime.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/datetime.md.txt
index 5471aba19f..2d70dbd7fb 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/datetime.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/datetime.md.txt
@@ -56,6 +56,12 @@ The following differences from Spark are always present and do not require any a
- Format strings in a curated allow-list run natively via DataFusion's `to_char` for UTC sessions. Other format strings (including non-literal formats), as well as non-UTC sessions, route through Spark's own `DateFormatClass.doGenCode` via the Arrow-direct codegen dispatcher when `spark.comet.exec.scalaUDF.codegen.enabled=true`. When the codegen dispatcher is disabled (default) the operator falls back to Spark in those cases.
+## Days
+
+The following cases are not supported by Comet:
+
+- Only `DateType` and `TimestampType` inputs are supported. `TimestampNTZType` is not supported.
+
## FromUTCTimestamp
The following incompatibilities cause `FromUTCTimestamp` to fall back to Spark by default. Set `spark.comet.expression.FromUTCTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -74,6 +80,12 @@ The following incompatibilities cause `Hour` to fall back to Spark by default. S
- Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)
+## Hours
+
+The following cases are not supported by Comet:
+
+- Only `TimestampType` and `TimestampNTZType` inputs are supported.
+
## Minute
The following incompatibilities cause `Minute` to fall back to Spark by default. Set `spark.comet.expression.Minute.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -86,6 +98,12 @@ The following incompatibilities cause `Second` to fall back to Spark by default.
- Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)
+## SecondsToTimestamp
+
+The following cases are not supported by Comet:
+
+- Only `IntegerType`, `LongType`, `FloatType`, and `DoubleType` inputs are supported. `DecimalType`, `ByteType`, and `ShortType` fall back to Spark.
+
## ToUTCTimestamp
The following incompatibilities cause `ToUTCTimestamp` to fall back to Spark by default. Set `spark.comet.expression.ToUTCTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -107,6 +125,11 @@ The following cases are not supported by Comet:
The following incompatibilities cause `TruncTimestamp` to fall back to Spark by default. Set `spark.comet.expression.TruncTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
- Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)
+- Non-literal format strings will throw an exception instead of returning NULL
+
+The following cases are not supported by Comet:
+
+- Only the following formats are supported: year, yyyy, yy, quarter, mon, month, mm, week, day, dd, hour, minute, second, millisecond, microsecond
## UnixTimestamp
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/string.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/string.md.txt
index 329f99312f..b407c1c19c 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/string.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/string.md.txt
@@ -21,6 +21,12 @@ under the License.
<!--BEGIN:EXPR_COMPAT[string]-->
+## BitLength
+
+The following cases are not supported by Comet:
+
+- `BinaryType` input is not supported
+
## Concat
The following incompatibilities cause `Concat` to fall back to Spark by default. Set `spark.comet.expression.Concat.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -44,6 +50,12 @@ The following cases are not supported by Comet:
- `BinaryType` input is not supported
+## OctetLength
+
+The following cases are not supported by Comet:
+
+- `BinaryType` input is not supported
+
## Reverse
The following incompatibilities cause `Reverse` to fall back to Spark by default. Set `spark.comet.expression.Reverse.allowIncompatible=true` to enable Comet acceleration despite these differences.
@@ -76,4 +88,10 @@ The following cases are not supported by Comet:
The following differences from Spark are always present and do not require any additional configuration:
- A negative argument for the number of times to repeat throws an exception instead of returning an empty string as Spark does
+
+## StringTranslate
+
+The following incompatibilities cause `StringTranslate` to fall back to Spark by default. Set `spark.comet.expression.StringTranslate.allowIncompatible=true` to enable Comet acceleration despite these differences.
+
+- DataFusion's translate iterates over Unicode graphemes (Spark uses code points) and substitutes U+0000 instead of treating it as a deletion sentinel
<!--END:EXPR_COMPAT-->
diff --git a/_sources/user-guide/latest/expressions.md.txt b/_sources/user-guide/latest/expressions.md.txt
index 1281f3bad2..4bd8916448 100644
--- a/_sources/user-guide/latest/expressions.md.txt
+++ b/_sources/user-guide/latest/expressions.md.txt
@@ -589,7 +589,7 @@ expression-level). The `outer` variants are wired but marked `Incompatible`; the
| `to_char` | ✅ | |
| `to_number` | ✅ | |
| `to_varchar` | ✅ | |
-| `translate` | ✅ | |
+| `translate` | ✅ | Falls back by default; opt-in via allowIncompatible ([#4463](https://github.com/apache/datafusion-comet/issues/4463)) |
| `trim` | ✅ | |
| `try_to_binary` | 🔜 | Lowers to `TryEval(...)`, which falls back |
| `try_to_number` | 🔜 | TRY variant of `to_number` |
diff --git a/searchindex.js b/searchindex.js
index 7707225095..2d1cddd8dc 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[49, "id1"]], "%": [[47, "id1"]], "&": [[37, "id1"]], "*": [[47, "id2"]], "+": [[47, "id3"]], "-": [[47, "id4"]], "/": [[47, "id5"]], "1. Format Your Code": [[34, "format-your-code"]], "1. Install Comet": [[55, "install-comet"], [64, "install-comet"]], "1. Native Operators (nativeExecs map)": [[23, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[34, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[55, "clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[49, "id1"]], "%": [[47, "id1"]], "&": [[37, "id1"]], "*": [[47, "id2"]], "+": [[47, "id3"]], "-": [[47, "id4"]], "/": [[47, "id5"]], "1. Format Your Code": [[34, "format-your-code"]], "1. Install Comet": [[55, "install-comet"], [64, "install-comet"]], "1. Native Operators (nativeExecs map)": [[23, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[34, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[55, "clone-iceberg-and-apply- [...]
\ No newline at end of file
diff --git a/user-guide/latest/compatibility/expressions/spark-3.4/datetime.html b/user-guide/latest/compatibility/expressions/spark-3.4/datetime.html
index dabd15ba65..fab5d3be8c 100644
--- a/user-guide/latest/compatibility/expressions/spark-3.4/datetime.html
+++ b/user-guide/latest/compatibility/expressions/spark-3.4/datetime.html
@@ -612,6 +612,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Format strings in a curated allow-list run natively via DataFusion’s <code class="docutils literal notranslate"><span class="pre">to_char</span></code> for UTC sessions. Other format strings (including non-literal formats), as well as non-UTC sessions, route through Spark’s own <code class="docutils literal notranslate"><span class="pre">DateFormatClass.doGenCode</span></code> via the Arrow-direct codegen dispatcher when <code class="docutils literal notranslate"><span class="pre" [...]
</ul>
</section>
+<section id="days">
+<h2>Days<a class="headerlink" href="#days" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">DateType</span></code> and <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> inputs are supported. <code class="docutils literal notranslate"><span class="pre">TimestampNTZType</span></code> is not supported.</p></li>
+</ul>
+</section>
<section id="fromutctimestamp">
<h2>FromUTCTimestamp<a class="headerlink" href="#fromutctimestamp" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">FromUTCTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.FromUTCTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -633,6 +640,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)</p></li>
</ul>
</section>
+<section id="hours">
+<h2>Hours<a class="headerlink" href="#hours" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> and <code class="docutils literal notranslate"><span class="pre">TimestampNTZType</span></code> inputs are supported.</p></li>
+</ul>
+</section>
<section id="minute">
<h2>Minute<a class="headerlink" href="#minute" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Minute</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Minute.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -647,6 +661,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)</p></li>
</ul>
</section>
+<section id="secondstotimestamp">
+<h2>SecondsToTimestamp<a class="headerlink" href="#secondstotimestamp" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">IntegerType</span></code>, <code class="docutils literal notranslate"><span class="pre">LongType</span></code>, <code class="docutils literal notranslate"><span class="pre">FloatType</span></code>, and <code class="docutils literal notranslate"><span class="pre">DoubleType</span></code> inputs are supported. <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>, <code class="docutils [...]
+</ul>
+</section>
<section id="toutctimestamp">
<h2>ToUTCTimestamp<a class="headerlink" href="#toutctimestamp" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">ToUTCTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.ToUTCTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -670,6 +691,11 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">TruncTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.TruncTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
<ul class="simple">
<li><p>Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)</p></li>
+<li><p>Non-literal format strings will throw an exception instead of returning NULL</p></li>
+</ul>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only the following formats are supported: year, yyyy, yy, quarter, mon, month, mm, week, day, dd, hour, minute, second, millisecond, microsecond</p></li>
</ul>
</section>
<section id="unixtimestamp">
diff --git a/user-guide/latest/compatibility/expressions/spark-3.4/string.html b/user-guide/latest/compatibility/expressions/spark-3.4/string.html
index 5de041712c..e5c249d4dc 100644
--- a/user-guide/latest/compatibility/expressions/spark-3.4/string.html
+++ b/user-guide/latest/compatibility/expressions/spark-3.4/string.html
@@ -575,6 +575,13 @@ under the License.
<section id="string-expressions">
<h1>String Expressions<a class="headerlink" href="#string-expressions" title="Link to this heading">#</a></h1>
<!--BEGIN:EXPR_COMPAT[string]-->
+<section id="bitlength">
+<h2>BitLength<a class="headerlink" href="#bitlength" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
+</ul>
+</section>
<section id="concat">
<h2>Concat<a class="headerlink" href="#concat" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Concat</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Concat.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -601,6 +608,13 @@ under the License.
<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
</ul>
</section>
+<section id="octetlength">
+<h2>OctetLength<a class="headerlink" href="#octetlength" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
+</ul>
+</section>
<section id="reverse">
<h2>Reverse<a class="headerlink" href="#reverse" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Reverse</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Reverse.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -645,6 +659,13 @@ under the License.
<ul class="simple">
<li><p>A negative argument for the number of times to repeat throws an exception instead of returning an empty string as Spark does</p></li>
</ul>
+</section>
+<section id="stringtranslate">
+<h2>StringTranslate<a class="headerlink" href="#stringtranslate" title="Link to this heading">#</a></h2>
+<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">StringTranslate</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.StringTranslate.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
+<ul class="simple">
+<li><p>DataFusion’s translate iterates over Unicode graphemes (Spark uses code points) and substitutes U+0000 instead of treating it as a deletion sentinel</p></li>
+</ul>
<!--END:EXPR_COMPAT-->
</section>
</section>
diff --git a/user-guide/latest/compatibility/expressions/spark-3.5/datetime.html b/user-guide/latest/compatibility/expressions/spark-3.5/datetime.html
index 76b2e3b488..6b627606f3 100644
--- a/user-guide/latest/compatibility/expressions/spark-3.5/datetime.html
+++ b/user-guide/latest/compatibility/expressions/spark-3.5/datetime.html
@@ -612,6 +612,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Format strings in a curated allow-list run natively via DataFusion’s <code class="docutils literal notranslate"><span class="pre">to_char</span></code> for UTC sessions. Other format strings (including non-literal formats), as well as non-UTC sessions, route through Spark’s own <code class="docutils literal notranslate"><span class="pre">DateFormatClass.doGenCode</span></code> via the Arrow-direct codegen dispatcher when <code class="docutils literal notranslate"><span class="pre" [...]
</ul>
</section>
+<section id="days">
+<h2>Days<a class="headerlink" href="#days" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">DateType</span></code> and <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> inputs are supported. <code class="docutils literal notranslate"><span class="pre">TimestampNTZType</span></code> is not supported.</p></li>
+</ul>
+</section>
<section id="fromutctimestamp">
<h2>FromUTCTimestamp<a class="headerlink" href="#fromutctimestamp" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">FromUTCTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.FromUTCTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -633,6 +640,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)</p></li>
</ul>
</section>
+<section id="hours">
+<h2>Hours<a class="headerlink" href="#hours" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> and <code class="docutils literal notranslate"><span class="pre">TimestampNTZType</span></code> inputs are supported.</p></li>
+</ul>
+</section>
<section id="minute">
<h2>Minute<a class="headerlink" href="#minute" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Minute</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Minute.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -647,6 +661,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)</p></li>
</ul>
</section>
+<section id="secondstotimestamp">
+<h2>SecondsToTimestamp<a class="headerlink" href="#secondstotimestamp" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">IntegerType</span></code>, <code class="docutils literal notranslate"><span class="pre">LongType</span></code>, <code class="docutils literal notranslate"><span class="pre">FloatType</span></code>, and <code class="docutils literal notranslate"><span class="pre">DoubleType</span></code> inputs are supported. <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>, <code class="docutils [...]
+</ul>
+</section>
<section id="toutctimestamp">
<h2>ToUTCTimestamp<a class="headerlink" href="#toutctimestamp" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">ToUTCTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.ToUTCTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -670,6 +691,11 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">TruncTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.TruncTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
<ul class="simple">
<li><p>Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)</p></li>
+<li><p>Non-literal format strings will throw an exception instead of returning NULL</p></li>
+</ul>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only the following formats are supported: year, yyyy, yy, quarter, mon, month, mm, week, day, dd, hour, minute, second, millisecond, microsecond</p></li>
</ul>
</section>
<section id="unixtimestamp">
diff --git a/user-guide/latest/compatibility/expressions/spark-3.5/string.html b/user-guide/latest/compatibility/expressions/spark-3.5/string.html
index 7a474057f2..5a4ca7a32d 100644
--- a/user-guide/latest/compatibility/expressions/spark-3.5/string.html
+++ b/user-guide/latest/compatibility/expressions/spark-3.5/string.html
@@ -575,6 +575,13 @@ under the License.
<section id="string-expressions">
<h1>String Expressions<a class="headerlink" href="#string-expressions" title="Link to this heading">#</a></h1>
<!--BEGIN:EXPR_COMPAT[string]-->
+<section id="bitlength">
+<h2>BitLength<a class="headerlink" href="#bitlength" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
+</ul>
+</section>
<section id="concat">
<h2>Concat<a class="headerlink" href="#concat" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Concat</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Concat.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -601,6 +608,13 @@ under the License.
<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
</ul>
</section>
+<section id="octetlength">
+<h2>OctetLength<a class="headerlink" href="#octetlength" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
+</ul>
+</section>
<section id="reverse">
<h2>Reverse<a class="headerlink" href="#reverse" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Reverse</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Reverse.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -645,6 +659,13 @@ under the License.
<ul class="simple">
<li><p>A negative argument for the number of times to repeat throws an exception instead of returning an empty string as Spark does</p></li>
</ul>
+</section>
+<section id="stringtranslate">
+<h2>StringTranslate<a class="headerlink" href="#stringtranslate" title="Link to this heading">#</a></h2>
+<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">StringTranslate</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.StringTranslate.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
+<ul class="simple">
+<li><p>DataFusion’s translate iterates over Unicode graphemes (Spark uses code points) and substitutes U+0000 instead of treating it as a deletion sentinel</p></li>
+</ul>
<!--END:EXPR_COMPAT-->
</section>
</section>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.0/datetime.html b/user-guide/latest/compatibility/expressions/spark-4.0/datetime.html
index 87ca1f9bb8..8b187baaf2 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.0/datetime.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.0/datetime.html
@@ -612,6 +612,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Format strings in a curated allow-list run natively via DataFusion’s <code class="docutils literal notranslate"><span class="pre">to_char</span></code> for UTC sessions. Other format strings (including non-literal formats), as well as non-UTC sessions, route through Spark’s own <code class="docutils literal notranslate"><span class="pre">DateFormatClass.doGenCode</span></code> via the Arrow-direct codegen dispatcher when <code class="docutils literal notranslate"><span class="pre" [...]
</ul>
</section>
+<section id="days">
+<h2>Days<a class="headerlink" href="#days" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">DateType</span></code> and <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> inputs are supported. <code class="docutils literal notranslate"><span class="pre">TimestampNTZType</span></code> is not supported.</p></li>
+</ul>
+</section>
<section id="fromutctimestamp">
<h2>FromUTCTimestamp<a class="headerlink" href="#fromutctimestamp" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">FromUTCTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.FromUTCTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -633,6 +640,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)</p></li>
</ul>
</section>
+<section id="hours">
+<h2>Hours<a class="headerlink" href="#hours" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> and <code class="docutils literal notranslate"><span class="pre">TimestampNTZType</span></code> inputs are supported.</p></li>
+</ul>
+</section>
<section id="minute">
<h2>Minute<a class="headerlink" href="#minute" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Minute</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Minute.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -647,6 +661,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)</p></li>
</ul>
</section>
+<section id="secondstotimestamp">
+<h2>SecondsToTimestamp<a class="headerlink" href="#secondstotimestamp" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">IntegerType</span></code>, <code class="docutils literal notranslate"><span class="pre">LongType</span></code>, <code class="docutils literal notranslate"><span class="pre">FloatType</span></code>, and <code class="docutils literal notranslate"><span class="pre">DoubleType</span></code> inputs are supported. <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>, <code class="docutils [...]
+</ul>
+</section>
<section id="toutctimestamp">
<h2>ToUTCTimestamp<a class="headerlink" href="#toutctimestamp" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">ToUTCTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.ToUTCTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -670,6 +691,11 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">TruncTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.TruncTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
<ul class="simple">
<li><p>Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)</p></li>
+<li><p>Non-literal format strings will throw an exception instead of returning NULL</p></li>
+</ul>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only the following formats are supported: year, yyyy, yy, quarter, mon, month, mm, week, day, dd, hour, minute, second, millisecond, microsecond</p></li>
</ul>
</section>
<section id="unixtimestamp">
diff --git a/user-guide/latest/compatibility/expressions/spark-4.0/string.html b/user-guide/latest/compatibility/expressions/spark-4.0/string.html
index 5fb6a36deb..0cbfdf236e 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.0/string.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.0/string.html
@@ -575,6 +575,13 @@ under the License.
<section id="string-expressions">
<h1>String Expressions<a class="headerlink" href="#string-expressions" title="Link to this heading">#</a></h1>
<!--BEGIN:EXPR_COMPAT[string]-->
+<section id="bitlength">
+<h2>BitLength<a class="headerlink" href="#bitlength" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
+</ul>
+</section>
<section id="concat">
<h2>Concat<a class="headerlink" href="#concat" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Concat</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Concat.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -601,6 +608,13 @@ under the License.
<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
</ul>
</section>
+<section id="octetlength">
+<h2>OctetLength<a class="headerlink" href="#octetlength" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
+</ul>
+</section>
<section id="reverse">
<h2>Reverse<a class="headerlink" href="#reverse" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Reverse</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Reverse.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -638,6 +652,13 @@ under the License.
<ul class="simple">
<li><p>A negative argument for the number of times to repeat throws an exception instead of returning an empty string as Spark does</p></li>
</ul>
+</section>
+<section id="stringtranslate">
+<h2>StringTranslate<a class="headerlink" href="#stringtranslate" title="Link to this heading">#</a></h2>
+<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">StringTranslate</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.StringTranslate.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
+<ul class="simple">
+<li><p>DataFusion’s translate iterates over Unicode graphemes (Spark uses code points) and substitutes U+0000 instead of treating it as a deletion sentinel</p></li>
+</ul>
<!--END:EXPR_COMPAT-->
</section>
</section>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.1/datetime.html b/user-guide/latest/compatibility/expressions/spark-4.1/datetime.html
index ffa83767ee..8679b91f41 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.1/datetime.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.1/datetime.html
@@ -612,6 +612,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Format strings in a curated allow-list run natively via DataFusion’s <code class="docutils literal notranslate"><span class="pre">to_char</span></code> for UTC sessions. Other format strings (including non-literal formats), as well as non-UTC sessions, route through Spark’s own <code class="docutils literal notranslate"><span class="pre">DateFormatClass.doGenCode</span></code> via the Arrow-direct codegen dispatcher when <code class="docutils literal notranslate"><span class="pre" [...]
</ul>
</section>
+<section id="days">
+<h2>Days<a class="headerlink" href="#days" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">DateType</span></code> and <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> inputs are supported. <code class="docutils literal notranslate"><span class="pre">TimestampNTZType</span></code> is not supported.</p></li>
+</ul>
+</section>
<section id="fromutctimestamp">
<h2>FromUTCTimestamp<a class="headerlink" href="#fromutctimestamp" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">FromUTCTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.FromUTCTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -633,6 +640,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)</p></li>
</ul>
</section>
+<section id="hours">
+<h2>Hours<a class="headerlink" href="#hours" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">TimestampType</span></code> and <code class="docutils literal notranslate"><span class="pre">TimestampNTZType</span></code> inputs are supported.</p></li>
+</ul>
+</section>
<section id="minute">
<h2>Minute<a class="headerlink" href="#minute" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Minute</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Minute.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -647,6 +661,13 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)</p></li>
</ul>
</section>
+<section id="secondstotimestamp">
+<h2>SecondsToTimestamp<a class="headerlink" href="#secondstotimestamp" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only <code class="docutils literal notranslate"><span class="pre">IntegerType</span></code>, <code class="docutils literal notranslate"><span class="pre">LongType</span></code>, <code class="docutils literal notranslate"><span class="pre">FloatType</span></code>, and <code class="docutils literal notranslate"><span class="pre">DoubleType</span></code> inputs are supported. <code class="docutils literal notranslate"><span class="pre">DecimalType</span></code>, <code class="docutils [...]
+</ul>
+</section>
<section id="toutctimestamp">
<h2>ToUTCTimestamp<a class="headerlink" href="#toutctimestamp" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">ToUTCTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.ToUTCTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -670,6 +691,11 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">TruncTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.TruncTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
<ul class="simple">
<li><p>Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)</p></li>
+<li><p>Non-literal format strings will throw an exception instead of returning NULL</p></li>
+</ul>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>Only the following formats are supported: year, yyyy, yy, quarter, mon, month, mm, week, day, dd, hour, minute, second, millisecond, microsecond</p></li>
</ul>
</section>
<section id="unixtimestamp">
diff --git a/user-guide/latest/compatibility/expressions/spark-4.1/string.html b/user-guide/latest/compatibility/expressions/spark-4.1/string.html
index 5d7d052e4f..6537f3d56e 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.1/string.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.1/string.html
@@ -575,6 +575,13 @@ under the License.
<section id="string-expressions">
<h1>String Expressions<a class="headerlink" href="#string-expressions" title="Link to this heading">#</a></h1>
<!--BEGIN:EXPR_COMPAT[string]-->
+<section id="bitlength">
+<h2>BitLength<a class="headerlink" href="#bitlength" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
+</ul>
+</section>
<section id="concat">
<h2>Concat<a class="headerlink" href="#concat" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Concat</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Concat.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -601,6 +608,13 @@ under the License.
<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
</ul>
</section>
+<section id="octetlength">
+<h2>OctetLength<a class="headerlink" href="#octetlength" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> input is not supported</p></li>
+</ul>
+</section>
<section id="reverse">
<h2>Reverse<a class="headerlink" href="#reverse" title="Link to this heading">#</a></h2>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Reverse</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Reverse.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
@@ -638,6 +652,13 @@ under the License.
<ul class="simple">
<li><p>A negative argument for the number of times to repeat throws an exception instead of returning an empty string as Spark does</p></li>
</ul>
+</section>
+<section id="stringtranslate">
+<h2>StringTranslate<a class="headerlink" href="#stringtranslate" title="Link to this heading">#</a></h2>
+<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">StringTranslate</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.StringTranslate.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
+<ul class="simple">
+<li><p>DataFusion’s translate iterates over Unicode graphemes (Spark uses code points) and substitutes U+0000 instead of treating it as a deletion sentinel</p></li>
+</ul>
<!--END:EXPR_COMPAT-->
</section>
</section>
diff --git a/user-guide/latest/expressions.html b/user-guide/latest/expressions.html
index 0f0ac6f395..16da76d928 100644
--- a/user-guide/latest/expressions.html
+++ b/user-guide/latest/expressions.html
@@ -2499,7 +2499,7 @@ expression-level). The <code class="docutils literal notranslate"><span class="p
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">translate</span></code></p></td>
<td><p>✅</p></td>
-<td><p></p></td>
+<td><p>Falls back by default; opt-in via allowIncompatible (<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/4463">#4463</a>)</p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">trim</span></code></p></td>
<td><p>✅</p></td>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org