[GitHub] [spark] cloud-fan commented on a change in pull request #28067: [WIP][SPARK-31297][SQL] Speed up dates rebasing

[GitBox <gi...@apache.org>]

cloud-fan commented on a change in pull request #28067: [WIP][SPARK-31297][SQL] Speed up dates rebasing
URL: https://github.com/apache/spark/pull/28067#discussion_r400130761
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
 ##########
 @@ -1033,6 +1033,40 @@ object DateTimeUtils {
     instantToMicros(localDateTime.atZone(ZoneId.systemDefault).toInstant)
   }
 
+  /**
+   * Rebases days since the epoch from an original to an target calendar, from instance
+   * from a hybrid (Julian + Gregorian) to Proleptic Gregorian calendar.
+   *
+   * It finds the latest switch day which is less than `days`, and adds the difference
+   * in days associated with the switch day to the given `days`. The function is based
+   * on linear search which starts from the most recent switch days. This allows to perform
+   * less comparisons for modern dates.
+   *
+   * @param switchDays The days when difference in days between original and target
+   *                   calendar was changed.
+   * @param diffs The differences in days between calendars.
+   * @param days The number of days since the epoch 1970-01-01 to be rebased to the
+   *             target calendar.
+   * @return The rebased day
+   */
+  private def rebaseDays(switchDays: Array[Int], diffs: Array[Int], days: Int): Int = {
+    var i = switchDays.length - 1
+    while (i >= 0 && days < switchDays(i)) {
+      i -= 1
+    }
+    val rebased = days + diffs(if (i < 0) 0 else i)
+    rebased
+  }
+
+  // The differences in days between Julian and Proleptic Gregorian dates.
+  // The diff at the index `i` is applicable for all days in the date interval:
+  // [julianGregDiffSwitchDay(i), julianGregDiffSwitchDay(i+1))
 
 Review comment:
   Then it means `julianGregDiffSwitchDay(0)` is useless? It's not a point that difference in days changes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org