You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/10/18 19:23:49 UTC

(datafusion-comet) branch main updated: test: write ancient dates with CORRECTED rebase mode in the DATE RANGE window test (#7103)

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git


The following commit(s) were added to refs/heads/main by this push:
     new d0f367dc02 test: write ancient dates with CORRECTED rebase mode in the DATE RANGE window test (#7103)
d0f367dc02 is described below

commit d0f367dc02520fe68f524db22083cc0af6e58bca
Author: Andy Grove <ag...@apache.org>
AuthorDate: Sun Oct 18 18:13:37 2026 +0000

    test: write ancient dates with CORRECTED rebase mode in the DATE RANGE window test (#7103)
---
 .../org/apache/comet/exec/CometWindowExecSuite.scala   | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/spark/src/test/scala/org/apache/comet/exec/CometWindowExecSuite.scala b/spark/src/test/scala/org/apache/comet/exec/CometWindowExecSuite.scala
index cf881d42c4..b392706d12 100644
--- a/spark/src/test/scala/org/apache/comet/exec/CometWindowExecSuite.scala
+++ b/spark/src/test/scala/org/apache/comet/exec/CometWindowExecSuite.scala
@@ -1829,13 +1829,17 @@ class CometWindowExecSuite extends CometTestBase {
     // becomes the partition edge, which agrees with Spark's unwrapped bound
     // lying beyond every row.
     withTempDir { dir =>
-      Seq("0001-01-01", "1970-01-01", "2024-01-05", "9999-12-31")
-        .map(java.sql.Date.valueOf)
-        .toDF("d")
-        .repartition(1)
-        .write
-        .mode("overwrite")
-        .parquet(dir.toString)
+      // 0001-01-01 predates the Gregorian cutover, and Spark 3.x defaults the write rebase mode to
+      // EXCEPTION, so write it as-is.
+      withSQLConf("spark.sql.parquet.datetimeRebaseModeInWrite" -> "CORRECTED") {
+        Seq("0001-01-01", "1970-01-01", "2024-01-05", "9999-12-31")
+          .map(java.sql.Date.valueOf)
+          .toDF("d")
+          .repartition(1)
+          .write
+          .mode("overwrite")
+          .parquet(dir.toString)
+      }
       spark.read.parquet(dir.toString).createOrReplaceTempView("date_edge")
       for {
         order <- Seq("ASC", "DESC")


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org