You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/04/24 19:52:06 UTC
(datafusion-comet) branch asf-site updated: Publish built docs triggered by 724152e3be063813b56d9e846b54ced18f596788
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new c5d197a2f Publish built docs triggered by 724152e3be063813b56d9e846b54ced18f596788
c5d197a2f is described below
commit c5d197a2fae0bb831be4d31ff725e7702b6ed5a3
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Fri Apr 24 19:52:03 2026 +0000
Publish built docs triggered by 724152e3be063813b56d9e846b54ced18f596788
---
.../compatibility/expressions/datetime.md.txt | 26 +++++++++++++--
searchindex.js | 2 +-
.../latest/compatibility/expressions/datetime.html | 37 ++++++++++++++++++----
3 files changed, 56 insertions(+), 9 deletions(-)
diff --git a/_sources/user-guide/latest/compatibility/expressions/datetime.md.txt b/_sources/user-guide/latest/compatibility/expressions/datetime.md.txt
index d352de1f1..b6716f544 100644
--- a/_sources/user-guide/latest/compatibility/expressions/datetime.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/datetime.md.txt
@@ -19,7 +19,29 @@ under the License.
# Date/Time Expressions
-<!--BEGIN:EXPR_COMPAT[datetime]-->
+- **Hour, Minute, Second**: Incorrectly apply timezone conversion to TimestampNTZ inputs. TimestampNTZ stores local
+ time without timezone, so no conversion should be applied. These expressions work correctly with Timestamp inputs.
+ [#3180](https://github.com/apache/datafusion-comet/issues/3180)
+- **TruncTimestamp (date_trunc)**: Produces incorrect results when used with non-UTC timezones. Compatible when
+ timezone is UTC.
+ [#2649](https://github.com/apache/datafusion-comet/issues/2649)
+
+## Date and Time Functions
+
+Comet's native implementation of date and time functions may produce different results than Spark for dates
+far in the future (approximately beyond year 2100). This is because Comet uses the chrono-tz library for
+timezone calculations, which has limited support for Daylight Saving Time (DST) rules beyond the IANA
+time zone database's explicit transitions.
+
+For dates within a reasonable range (approximately 1970-2100), Comet's date and time functions are compatible
+with Spark. For dates beyond this range, functions that involve timezone-aware calculations (such as
+`date_trunc` with timezone-aware timestamps) may produce results with incorrect DST offsets.
+
+If you need to process dates far in the future with accurate timezone handling, consider:
+
+- Using timezone-naive types (`timestamp_ntz`) when timezone conversion is not required
+- Falling back to Spark for these specific operations
+ <!--BEGIN:EXPR_COMPAT[datetime]-->
### DateFormatClass
@@ -65,4 +87,4 @@ The following incompatibilities cause `Hour` to fall back to Spark by default. S
The following incompatibilities cause `TruncTimestamp` to fall back to Spark by default. Set `spark.comet.expression.TruncTimestamp.allowIncompatible=true` to enable Comet acceleration despite these differences.
- Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)
-<!--END:EXPR_COMPAT-->
+ <!--END:EXPR_COMPAT-->
diff --git a/searchindex.js b/searchindex.js
index 8c73ae92f..c2a5ae03a 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"1. Format Your Code": [[14, "format-your-code"]], "1. Install Comet": [[16, "install-comet"], [24, "install-comet"]], "1. Native Operators (nativeExecs map)": [[4, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[14, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[16, "clone-iceberg-and-apply-diff"]], "2. Clone Spark and Apply Diff": [[24, "clone-spark-and-apply-diff"]], "2. Sink Operators (sinks map)": [[4, "sink-operators-sinks-m [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"1. Format Your Code": [[14, "format-your-code"]], "1. Install Comet": [[16, "install-comet"], [24, "install-comet"]], "1. Native Operators (nativeExecs map)": [[4, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[14, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[16, "clone-iceberg-and-apply-diff"]], "2. Clone Spark and Apply Diff": [[24, "clone-spark-and-apply-diff"]], "2. Sink Operators (sinks map)": [[4, "sink-operators-sinks-m [...]
\ No newline at end of file
diff --git a/user-guide/latest/compatibility/expressions/datetime.html b/user-guide/latest/compatibility/expressions/datetime.html
index ca8046c9f..04fcafdba 100644
--- a/user-guide/latest/compatibility/expressions/datetime.html
+++ b/user-guide/latest/compatibility/expressions/datetime.html
@@ -473,9 +473,32 @@ under the License.
-->
<section id="date-time-expressions">
<h1>Date/Time Expressions<a class="headerlink" href="#date-time-expressions" title="Link to this heading">#</a></h1>
+<ul class="simple">
+<li><p><strong>Hour, Minute, Second</strong>: Incorrectly apply timezone conversion to TimestampNTZ inputs. TimestampNTZ stores local
+time without timezone, so no conversion should be applied. These expressions work correctly with Timestamp inputs.
+<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/3180">#3180</a></p></li>
+<li><p><strong>TruncTimestamp (date_trunc)</strong>: Produces incorrect results when used with non-UTC timezones. Compatible when
+timezone is UTC.
+<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/2649">#2649</a></p></li>
+</ul>
+<section id="date-and-time-functions">
+<h2>Date and Time Functions<a class="headerlink" href="#date-and-time-functions" title="Link to this heading">#</a></h2>
+<p>Comet’s native implementation of date and time functions may produce different results than Spark for dates
+far in the future (approximately beyond year 2100). This is because Comet uses the chrono-tz library for
+timezone calculations, which has limited support for Daylight Saving Time (DST) rules beyond the IANA
+time zone database’s explicit transitions.</p>
+<p>For dates within a reasonable range (approximately 1970-2100), Comet’s date and time functions are compatible
+with Spark. For dates beyond this range, functions that involve timezone-aware calculations (such as
+<code class="docutils literal notranslate"><span class="pre">date_trunc</span></code> with timezone-aware timestamps) may produce results with incorrect DST offsets.</p>
+<p>If you need to process dates far in the future with accurate timezone handling, consider:</p>
+<ul>
+<li><p>Using timezone-naive types (<code class="docutils literal notranslate"><span class="pre">timestamp_ntz</span></code>) when timezone conversion is not required</p></li>
+<li><p>Falling back to Spark for these specific operations</p>
<!--BEGIN:EXPR_COMPAT[datetime]-->
+</li>
+</ul>
<section id="dateformatclass">
-<h2>DateFormatClass<a class="headerlink" href="#dateformatclass" title="Link to this heading">#</a></h2>
+<h3>DateFormatClass<a class="headerlink" href="#dateformatclass" title="Link to this heading">#</a></h3>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">DateFormatClass</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.DateFormatClass.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
<ul class="simple">
<li><p>Non-UTC timezones may produce different results than Spark</p></li>
@@ -512,19 +535,21 @@ under the License.
</ul>
</section>
<section id="hour">
-<h2>Hour<a class="headerlink" href="#hour" title="Link to this heading">#</a></h2>
+<h3>Hour<a class="headerlink" href="#hour" title="Link to this heading">#</a></h3>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Hour</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Hour.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
<ul class="simple">
<li><p>Incorrectly applies timezone conversion to TimestampNTZ inputs (https://github.com/apache/datafusion-comet/issues/3180)</p></li>
</ul>
</section>
<section id="trunctimestamp">
-<h2>TruncTimestamp<a class="headerlink" href="#trunctimestamp" title="Link to this heading">#</a></h2>
+<h3>TruncTimestamp<a class="headerlink" href="#trunctimestamp" title="Link to this heading">#</a></h3>
<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">TruncTimestamp</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.TruncTimestamp.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
-<ul class="simple">
-<li><p>Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)</p></li>
-</ul>
+<ul>
+<li><p>Produces incorrect results when used with non-UTC timezones. Compatible when timezone is UTC. (https://github.com/apache/datafusion-comet/issues/2649)</p>
<!--END:EXPR_COMPAT-->
+</li>
+</ul>
+</section>
</section>
</section>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org