You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/09/25 00:24:05 UTC
(datafusion-comet) branch asf-site updated: Publish built docs triggered by 801b099101b685d661e9332588f9de6fc9e194da
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 4de11811d1 Publish built docs triggered by 801b099101b685d661e9332588f9de6fc9e194da
4de11811d1 is described below
commit 4de11811d1792a08f10d036d53f4ead4927858af
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Fri Sep 25 00:24:00 2026 +0000
Publish built docs triggered by 801b099101b685d661e9332588f9de6fc9e194da
---
.../mermaid-013012468fa27c9e9fdb0a14196d357c619c68a8.svg | 2 +-
_sources/user-guide/1.0/configs.md.txt | 2 +-
_sources/user-guide/1.0/tuning.md.txt | 14 ++++++++------
.../compatibility/expressions/spark-4.0/datetime.md.txt | 12 ------------
.../compatibility/expressions/spark-4.1/datetime.md.txt | 12 ------------
_sources/user-guide/latest/expressions.md.txt | 2 +-
searchindex.js | 2 +-
user-guide/1.0/configs.html | 2 +-
user-guide/1.0/tuning.html | 14 ++++++++------
.../compatibility/expressions/spark-4.0/datetime.html | 16 ----------------
.../compatibility/expressions/spark-4.1/datetime.html | 16 ----------------
user-guide/latest/expressions.html | 2 +-
12 files changed, 22 insertions(+), 74 deletions(-)
diff --git a/_images/mermaid-013012468fa27c9e9fdb0a14196d357c619c68a8.svg b/_images/mermaid-013012468fa27c9e9fdb0a14196d357c619c68a8.svg
index bcbddd6e33..d03a7a4785 100644
--- a/_images/mermaid-013012468fa27c9e9fdb0a14196d357c619c68a8.svg
+++ b/_images/mermaid-013012468fa27c9e9fdb0a14196d357c619c68a8.svg
@@ -1 +1 @@
-<svg id="my-svg" width="100%" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" class="flowchart" style="max-width: 511.73px; background-color: transparent;" viewBox="0 0 511.7299499511719 942" role="graphics-document document" aria-roledescription="flowchart-v2"><style>#my-svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#my-svg [...]
\ No newline at end of file
+<svg id="my-svg" width="100%" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" class="flowchart" style="max-width: 511.73px; background-color: transparent;" viewBox="0 0 511.7299499511719 942" role="graphics-document document" aria-roledescription="flowchart-v2"><style>#my-svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#my-svg [...]
\ No newline at end of file
diff --git a/_sources/user-guide/1.0/configs.md.txt b/_sources/user-guide/1.0/configs.md.txt
index add59e103a..8eae309725 100644
--- a/_sources/user-guide/1.0/configs.md.txt
+++ b/_sources/user-guide/1.0/configs.md.txt
@@ -125,7 +125,7 @@ These settings can be used to determine which parts of the plan are accelerated
| `spark.comet.batchSize` | The columnar batch size, i.e., the maximum number of rows that a batch can contain. | 8192 |
| `spark.comet.exec.memoryPool` | The type of memory pool to be used for Comet native execution when running Spark in off-heap mode. Available pool types are `greedy_unified` and `fair_unified`. For more information, refer to the [Comet Tuning Guide](https://datafusion.apache.org/comet/user-guide/latest/tuning.html). | fair_unified |
| `spark.comet.exec.memoryPool.fraction` | Fraction of off-heap memory pool that is available to Comet. Only applies to off-heap mode. For more information, refer to the [Comet Tuning Guide](https://datafusion.apache.org/comet/user-guide/latest/tuning.html). | 1.0 |
-| `spark.comet.maxTempDirectorySize` | The maximum amount of data (in bytes) stored inside the temporary directories used by native operators when spilling. Applied per Spark task, so an executor running N concurrent tasks may use up to N times this value on shared local disks. Once the limit is reached, further spills will fail and the query will error out. | 107374182400b |
+| `spark.comet.maxTempDirectorySize` | The maximum amount of data (in bytes) stored inside the temporary directories used by native operators when spilling. Applied to each Comet native plan separately, and a Spark task can run more than one native plan at a time, so an executor running N concurrent tasks may use more than N times this value on shared local disks. Once the limit is reached, further spills will fail and the query will error out. | 107374182400b |
| `spark.comet.tracing.enabled` | Enable fine-grained tracing of events and memory usage. For more information, refer to the [Comet Tracing Guide](https://datafusion.apache.org/comet/contributor-guide/tracing.html). | false |
<!-- prettier-ignore-end -->
<!--END:CONFIG_TABLE-->
diff --git a/_sources/user-guide/1.0/tuning.md.txt b/_sources/user-guide/1.0/tuning.md.txt
index 37b6f03669..2686eb34db 100644
--- a/_sources/user-guide/1.0/tuning.md.txt
+++ b/_sources/user-guide/1.0/tuning.md.txt
@@ -116,12 +116,14 @@ flushes sorted spill files. It must not exceed `spark.comet.batchSize`.
### Limiting Spill Disk Usage
-Native operators that spill to disk (aggregate, sort, shuffle) are collectively bounded by
-`spark.comet.maxTempDirectorySize` (default 100 GB). The limit is applied per Spark task, so an
-executor running `N` concurrent tasks may use up to `N` times this value on shared local disks.
-If the limit is reached, further spills fail and the query errors out. Raise this on workloads
-with large sort/aggregate/shuffle spills, or lower it to protect executors on shared disks
-(remembering to divide by task concurrency to reason about the aggregate).
+Native operators that spill to disk (aggregate, sort, shuffle) are bounded by
+`spark.comet.maxTempDirectorySize` (default 100 GB). The operators of one Comet native plan share
+the limit. A Spark task can run more than one native plan at a time, for example the native
+operators on either side of a union or a coalesce, so an executor running `N` concurrent tasks may
+use more than `N` times this value on shared local disks. If the limit is reached, further spills
+fail and the query errors out. Raise this on workloads with large sort/aggregate/shuffle spills, or
+lower it to protect executors on shared disks, remembering that the total across an executor is a
+multiple of this value.
## Parquet Reader Tuning
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/datetime.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/datetime.md.txt
index 8c5dff1dcd..22f09e9891 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/datetime.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/datetime.md.txt
@@ -100,12 +100,6 @@ By default, `MakeInterval` is evaluated in the JVM using Spark's own code-genera
- The native implementation converts seconds to `Float64`, which can lose microsecond precision, and stores time in nanoseconds, which overflows for large time components (hours, minutes, seconds) that Spark can represent.
-## MakeTimestamp
-
-By default, `MakeTimestamp` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.MakeTimestamp.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
-
-- make_timestamp does not support non-UTF8_BINARY collations (https://github.com/apache/datafusion-comet/issues/4646)
-
## NextDay
By default, `NextDay` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.NextDay.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
@@ -130,12 +124,6 @@ By default, `ToUTCTimestamp` is evaluated in the JVM using Spark's own code-gene
- Comet's native timezone parser only accepts IANA zone IDs (e.g. `America/Los_Angeles`) and fixed offsets in `+HH:MM` form. Spark also accepts forms such as `GMT+1`, `UTC+1`, or three-letter abbreviations like `PST`; queries using those forms will throw a native parse error at execution time. See https://github.com/apache/datafusion-comet/issues/2013.
-## ToUnixTimestamp
-
-By default, `ToUnixTimestamp` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.ToUnixTimestamp.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
-
-- to_unix_timestamp does not support non-UTF8_BINARY collations (https://github.com/apache/datafusion-comet/issues/4646)
-
## TruncDate
By default, `TruncDate` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.TruncDate.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/datetime.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/datetime.md.txt
index 8c5dff1dcd..22f09e9891 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/datetime.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/datetime.md.txt
@@ -100,12 +100,6 @@ By default, `MakeInterval` is evaluated in the JVM using Spark's own code-genera
- The native implementation converts seconds to `Float64`, which can lose microsecond precision, and stores time in nanoseconds, which overflows for large time components (hours, minutes, seconds) that Spark can represent.
-## MakeTimestamp
-
-By default, `MakeTimestamp` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.MakeTimestamp.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
-
-- make_timestamp does not support non-UTF8_BINARY collations (https://github.com/apache/datafusion-comet/issues/4646)
-
## NextDay
By default, `NextDay` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.NextDay.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
@@ -130,12 +124,6 @@ By default, `ToUTCTimestamp` is evaluated in the JVM using Spark's own code-gene
- Comet's native timezone parser only accepts IANA zone IDs (e.g. `America/Los_Angeles`) and fixed offsets in `+HH:MM` form. Spark also accepts forms such as `GMT+1`, `UTC+1`, or three-letter abbreviations like `PST`; queries using those forms will throw a native parse error at execution time. See https://github.com/apache/datafusion-comet/issues/2013.
-## ToUnixTimestamp
-
-By default, `ToUnixTimestamp` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.ToUnixTimestamp.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
-
-- to_unix_timestamp does not support non-UTF8_BINARY collations (https://github.com/apache/datafusion-comet/issues/4646)
-
## TruncDate
By default, `TruncDate` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.TruncDate.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
diff --git a/_sources/user-guide/latest/expressions.md.txt b/_sources/user-guide/latest/expressions.md.txt
index c5d23dd2e8..85ae0cc509 100644
--- a/_sources/user-guide/latest/expressions.md.txt
+++ b/_sources/user-guide/latest/expressions.md.txt
@@ -305,7 +305,7 @@ The type-name conversion functions (`bigint`, `binary`, `boolean`, `date`, `deci
| `to_timestamp` | ✅ | — | Rewrites to `Cast` (or `GetTimestamp` with a format) before Comet sees the plan |
| `to_timestamp_ltz` | ✅ | — | Rewrites to `to_timestamp` (`TimestampType`) |
| `to_timestamp_ntz` | ✅ | — | Rewrites to `to_timestamp` (`TimestampNTZType`) |
-| `to_unix_timestamp` | ✅ | Hybrid | |
+| `to_unix_timestamp` | ✅ | Codegen dispatch | |
| `to_utc_timestamp` | ✅ | Hybrid | Routes through the JVM codegen dispatcher by default (handles all timezone forms); the native path is opt-in via allowIncompatible ([details](compatibility/expressions/datetime.md)) |
| `trunc` | ✅ | Hybrid | |
| `try_make_interval` | ✅ | — | Rewrites to `MakeInterval`; same support as `make_interval` (Spark 4.0+) |
diff --git a/searchindex.js b/searchindex.js
index f1d4974745..1bcedfeb1d 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[56, "id1"]], "%": [[54, "id1"]], "&": [[44, "id1"]], "*": [[54, "id2"]], "+": [[54, "id3"]], "-": [[54, "id4"]], "/": [[54, "id5"]], "1. Format Your Code": [[41, "format-your-code"]], "1. Install Comet": [[62, "install-comet"], [73, "install-comet"]], "1. Native Operators (nativeExecs map)": [[28, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[41, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[62, "clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[56, "id1"]], "%": [[54, "id1"]], "&": [[44, "id1"]], "*": [[54, "id2"]], "+": [[54, "id3"]], "-": [[54, "id4"]], "/": [[54, "id5"]], "1. Format Your Code": [[41, "format-your-code"]], "1. Install Comet": [[62, "install-comet"], [73, "install-comet"]], "1. Native Operators (nativeExecs map)": [[28, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[41, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[62, "clone-iceberg-and-apply- [...]
\ No newline at end of file
diff --git a/user-guide/1.0/configs.html b/user-guide/1.0/configs.html
index c033fd4779..fa3e936228 100644
--- a/user-guide/1.0/configs.html
+++ b/user-guide/1.0/configs.html
@@ -1086,7 +1086,7 @@ under the License.
<td><p>1.0</p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">spark.comet.maxTempDirectorySize</span></code></p></td>
-<td><p>The maximum amount of data (in bytes) stored inside the temporary directories used by native operators when spilling. Applied per Spark task, so an executor running N concurrent tasks may use up to N times this value on shared local disks. Once the limit is reached, further spills will fail and the query will error out.</p></td>
+<td><p>The maximum amount of data (in bytes) stored inside the temporary directories used by native operators when spilling. Applied to each Comet native plan separately, and a Spark task can run more than one native plan at a time, so an executor running N concurrent tasks may use more than N times this value on shared local disks. Once the limit is reached, further spills will fail and the query will error out.</p></td>
<td><p>107374182400b</p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">spark.comet.tracing.enabled</span></code></p></td>
diff --git a/user-guide/1.0/tuning.html b/user-guide/1.0/tuning.html
index 91e5a57f2d..ff23302a7a 100644
--- a/user-guide/1.0/tuning.html
+++ b/user-guide/1.0/tuning.html
@@ -855,12 +855,14 @@ flushes sorted spill files. It must not exceed <code class="docutils literal not
</section>
<section id="limiting-spill-disk-usage">
<h3>Limiting Spill Disk Usage<a class="headerlink" href="#limiting-spill-disk-usage" title="Link to this heading">#</a></h3>
-<p>Native operators that spill to disk (aggregate, sort, shuffle) are collectively bounded by
-<code class="docutils literal notranslate"><span class="pre">spark.comet.maxTempDirectorySize</span></code> (default 100 GB). The limit is applied per Spark task, so an
-executor running <code class="docutils literal notranslate"><span class="pre">N</span></code> concurrent tasks may use up to <code class="docutils literal notranslate"><span class="pre">N</span></code> times this value on shared local disks.
-If the limit is reached, further spills fail and the query errors out. Raise this on workloads
-with large sort/aggregate/shuffle spills, or lower it to protect executors on shared disks
-(remembering to divide by task concurrency to reason about the aggregate).</p>
+<p>Native operators that spill to disk (aggregate, sort, shuffle) are bounded by
+<code class="docutils literal notranslate"><span class="pre">spark.comet.maxTempDirectorySize</span></code> (default 100 GB). The operators of one Comet native plan share
+the limit. A Spark task can run more than one native plan at a time, for example the native
+operators on either side of a union or a coalesce, so an executor running <code class="docutils literal notranslate"><span class="pre">N</span></code> concurrent tasks may
+use more than <code class="docutils literal notranslate"><span class="pre">N</span></code> times this value on shared local disks. If the limit is reached, further spills
+fail and the query errors out. Raise this on workloads with large sort/aggregate/shuffle spills, or
+lower it to protect executors on shared disks, remembering that the total across an executor is a
+multiple of this value.</p>
</section>
</section>
<section id="parquet-reader-tuning">
diff --git a/user-guide/latest/compatibility/expressions/spark-4.0/datetime.html b/user-guide/latest/compatibility/expressions/spark-4.0/datetime.html
index 809f6262a6..cf2d1be7af 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.0/datetime.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.0/datetime.html
@@ -870,13 +870,6 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>The native implementation converts seconds to <code class="docutils literal notranslate"><span class="pre">Float64</span></code>, which can lose microsecond precision, and stores time in nanoseconds, which overflows for large time components (hours, minutes, seconds) that Spark can represent.</p></li>
</ul>
</section>
-<section id="maketimestamp">
-<h2>MakeTimestamp<a class="headerlink" href="#maketimestamp" title="Link to this heading">#</a></h2>
-<p>By default, <code class="docutils literal notranslate"><span class="pre">MakeTimestamp</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.MakeTimestamp.allowIncompatible=true</span></code> to opt into Comet’s native implementation instead, which has the following differences from Spark:</p>
-<ul class="simple">
-<li><p>make_timestamp does not support non-UTF8_BINARY collations (https://github.com/apache/datafusion-comet/issues/4646)</p></li>
-</ul>
-</section>
<section id="nextday">
<h2>NextDay<a class="headerlink" href="#nextday" title="Link to this heading">#</a></h2>
<p>By default, <code class="docutils literal notranslate"><span class="pre">NextDay</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.NextDay.allowIncompatible=true</span></code> to opt into Comet’s native implementation instead, which has the following differences from Spark:</p>
@@ -905,13 +898,6 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Comet’s native timezone parser only accepts IANA zone IDs (e.g. <code class="docutils literal notranslate"><span class="pre">America/Los_Angeles</span></code>) and fixed offsets in <code class="docutils literal notranslate"><span class="pre">+HH:MM</span></code> form. Spark also accepts forms such as <code class="docutils literal notranslate"><span class="pre">GMT+1</span></code>, <code class="docutils literal notranslate"><span class="pre">UTC+1</span></code>, or three-letter abb [...]
</ul>
</section>
-<section id="tounixtimestamp">
-<h2>ToUnixTimestamp<a class="headerlink" href="#tounixtimestamp" title="Link to this heading">#</a></h2>
-<p>By default, <code class="docutils literal notranslate"><span class="pre">ToUnixTimestamp</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.ToUnixTimestamp.allowIncompatible=true</span></code> to opt into Comet’s native implementation instead, which has the following differences from Spark:</p>
-<ul class="simple">
-<li><p>to_unix_timestamp does not support non-UTF8_BINARY collations (https://github.com/apache/datafusion-comet/issues/4646)</p></li>
-</ul>
-</section>
<section id="truncdate">
<h2>TruncDate<a class="headerlink" href="#truncdate" title="Link to this heading">#</a></h2>
<p>By default, <code class="docutils literal notranslate"><span class="pre">TruncDate</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.TruncDate.allowIncompatible=true</span></code> to opt into Comet’s native implementation instead, which has the following differences from Spark:</p>
@@ -1002,12 +988,10 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#fromunixtime">FromUnixTime</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#hours">Hours</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#makeinterval">MakeInterval</a></li>
-<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#maketimestamp">MakeTimestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#nextday">NextDay</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#precisetimestampconversion">PreciseTimestampConversion</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#secondstotimestamp">SecondsToTimestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#toutctimestamp">ToUTCTimestamp</a></li>
-<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#tounixtimestamp">ToUnixTimestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#truncdate">TruncDate</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#trunctimestamp">TruncTimestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#unixtimestamp">UnixTimestamp</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.1/datetime.html b/user-guide/latest/compatibility/expressions/spark-4.1/datetime.html
index 0b046d57de..fa7aeeaf18 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.1/datetime.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.1/datetime.html
@@ -870,13 +870,6 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>The native implementation converts seconds to <code class="docutils literal notranslate"><span class="pre">Float64</span></code>, which can lose microsecond precision, and stores time in nanoseconds, which overflows for large time components (hours, minutes, seconds) that Spark can represent.</p></li>
</ul>
</section>
-<section id="maketimestamp">
-<h2>MakeTimestamp<a class="headerlink" href="#maketimestamp" title="Link to this heading">#</a></h2>
-<p>By default, <code class="docutils literal notranslate"><span class="pre">MakeTimestamp</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.MakeTimestamp.allowIncompatible=true</span></code> to opt into Comet’s native implementation instead, which has the following differences from Spark:</p>
-<ul class="simple">
-<li><p>make_timestamp does not support non-UTF8_BINARY collations (https://github.com/apache/datafusion-comet/issues/4646)</p></li>
-</ul>
-</section>
<section id="nextday">
<h2>NextDay<a class="headerlink" href="#nextday" title="Link to this heading">#</a></h2>
<p>By default, <code class="docutils literal notranslate"><span class="pre">NextDay</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.NextDay.allowIncompatible=true</span></code> to opt into Comet’s native implementation instead, which has the following differences from Spark:</p>
@@ -905,13 +898,6 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li><p>Comet’s native timezone parser only accepts IANA zone IDs (e.g. <code class="docutils literal notranslate"><span class="pre">America/Los_Angeles</span></code>) and fixed offsets in <code class="docutils literal notranslate"><span class="pre">+HH:MM</span></code> form. Spark also accepts forms such as <code class="docutils literal notranslate"><span class="pre">GMT+1</span></code>, <code class="docutils literal notranslate"><span class="pre">UTC+1</span></code>, or three-letter abb [...]
</ul>
</section>
-<section id="tounixtimestamp">
-<h2>ToUnixTimestamp<a class="headerlink" href="#tounixtimestamp" title="Link to this heading">#</a></h2>
-<p>By default, <code class="docutils literal notranslate"><span class="pre">ToUnixTimestamp</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.ToUnixTimestamp.allowIncompatible=true</span></code> to opt into Comet’s native implementation instead, which has the following differences from Spark:</p>
-<ul class="simple">
-<li><p>to_unix_timestamp does not support non-UTF8_BINARY collations (https://github.com/apache/datafusion-comet/issues/4646)</p></li>
-</ul>
-</section>
<section id="truncdate">
<h2>TruncDate<a class="headerlink" href="#truncdate" title="Link to this heading">#</a></h2>
<p>By default, <code class="docutils literal notranslate"><span class="pre">TruncDate</span></code> is evaluated in the JVM using Spark’s own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.TruncDate.allowIncompatible=true</span></code> to opt into Comet’s native implementation instead, which has the following differences from Spark:</p>
@@ -1002,12 +988,10 @@ with Spark. For dates beyond this range, functions that involve timezone-aware c
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#fromunixtime">FromUnixTime</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#hours">Hours</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#makeinterval">MakeInterval</a></li>
-<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#maketimestamp">MakeTimestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#nextday">NextDay</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#precisetimestampconversion">PreciseTimestampConversion</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#secondstotimestamp">SecondsToTimestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#toutctimestamp">ToUTCTimestamp</a></li>
-<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#tounixtimestamp">ToUnixTimestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#truncdate">TruncDate</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#trunctimestamp">TruncTimestamp</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#unixtimestamp">UnixTimestamp</a></li>
diff --git a/user-guide/latest/expressions.html b/user-guide/latest/expressions.html
index 9d00412ed0..411a937b64 100644
--- a/user-guide/latest/expressions.html
+++ b/user-guide/latest/expressions.html
@@ -1846,7 +1846,7 @@ expressions. The following function families are <strong>not currently planned</
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">to_unix_timestamp</span></code></p></td>
<td><p>✅</p></td>
-<td><p>Hybrid</p></td>
+<td><p>Codegen dispatch</p></td>
<td><p></p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">to_utc_timestamp</span></code></p></td>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org