You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by we...@apache.org on 2020/04/14 08:09:42 UTC
[spark] branch branch-3.0 updated: [SPARK-31402][SQL][FOLLOWUP]
Refine code comments in RebaseDateTime
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new be5dfcd [SPARK-31402][SQL][FOLLOWUP] Refine code comments in RebaseDateTime
be5dfcd is described below
commit be5dfcd2f2e42976db65cf2306371f0e7a74d4cc
Author: Wenchen Fan <we...@databricks.com>
AuthorDate: Tue Apr 14 08:06:55 2020 +0000
[SPARK-31402][SQL][FOLLOWUP] Refine code comments in RebaseDateTime
### What changes were proposed in this pull request?
Refine the code comments of days rebasing, to be consistent with the micros rebasing. i.e. one method is the actual implementation and the other variant is the optimized version.
### Why are the changes needed?
improve code comments
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
N/A
Closes #28199 from cloud-fan/comment.
Authored-by: Wenchen Fan <we...@databricks.com>
Signed-off-by: Wenchen Fan <we...@databricks.com>
(cherry picked from commit 6b88d136deb99afd9363b208fd6fe5684fe8c3b8)
Signed-off-by: Wenchen Fan <we...@databricks.com>
---
.../spark/sql/catalyst/util/RebaseDateTime.scala | 62 +++++++++-------------
1 file changed, 26 insertions(+), 36 deletions(-)
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala
index f4a98cf..50b552e 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala
@@ -64,8 +64,7 @@ object RebaseDateTime {
// The sorted days in Julian calendar when difference in days between Julian and
// Proleptic Gregorian calendars was changed.
// The starting point is the `0001-01-01` (-719164 days since the epoch in
- // Julian calendar). All dates before the staring point have the same difference
- // of 2 days in Julian and Proleptic Gregorian calendars.
+ // Julian calendar). This array is not applicable for dates before the staring point.
// Rebasing switch days and diffs `julianGregDiffSwitchDay` and `julianGregDiffs`
// was generated by the `localRebaseJulianToGregorianDays` function.
private val julianGregDiffSwitchDay = Array(
@@ -80,6 +79,15 @@ object RebaseDateTime {
* calendar, interprets the result as a local date in Proleptic Gregorian calendar, and takes the
* number of days since the epoch from the Gregorian local date.
*
+ * This is used to guarantee backward compatibility, as Spark 2.4 and earlier versions use
+ * Julian calendar for dates before 1582-10-15, while Spark 3.0 and later use Proleptic Gregorian
+ * calendar. See SPARK-26651.
+ *
+ * For example:
+ * Julian calendar: 1582-01-01 -> -141704
+ * Proleptic Gregorian calendar: 1582-01-01 -> -141714
+ * The code below converts -141704 to -141714.
+ *
* @param days The number of days since the epoch in Julian calendar. It can be negative.
* @return The rebased number of days in Gregorian calendar.
*/
@@ -105,22 +113,9 @@ object RebaseDateTime {
}
/**
- * For dates of Before Common Era, the function converts the given number of days since the epoch
- * day 1970-01-01 to a local date in Julian calendar, interprets the result as a local date in
- * Proleptic Gregorian calendar, and takes the number of days since the epoch from the Gregorian
- * local date.
- *
- * For dates of Common Era, the function performs rebasing via the pre-calculated array of diffs
- * between Julian and Proleptic Gregorian calendars.
- *
- * This is used to guarantee backward compatibility, as Spark 2.4 and earlier versions use
- * Julian calendar for dates before 1582-10-15, while Spark 3.0 and later use Proleptic Gregorian
- * calendar. See SPARK-26651.
- *
- * For example:
- * Julian calendar: 1582-01-01 -> -141704
- * Proleptic Gregorian calendar: 1582-01-01 -> -141714
- * The code below converts -141704 to -141714.
+ * An optimized version of [[localRebaseJulianToGregorianDays(Int)]]. This method leverages the
+ * pre-calculated rebasing array to save calculation. For dates of Before Common Era, the
+ * function falls back to the regular unoptimized version.
*
* @param days The number of days since the epoch in Julian calendar. It can be negative.
* @return The rebased number of days in Gregorian calendar.
@@ -140,8 +135,7 @@ object RebaseDateTime {
// The sorted days in Proleptic Gregorian calendar when difference in days between
// Proleptic Gregorian and Julian was changed.
// The starting point is the `0001-01-01` (-719162 days since the epoch in
- // Proleptic Gregorian calendar). All dates before the staring point have the same
- // difference of -2 days in Proleptic Gregorian and Julian calendars.
+ // Proleptic Gregorian calendar). This array is not applicable for dates before the staring point.
// Rebasing switch days and diffs `gregJulianDiffSwitchDay` and `gregJulianDiffs`
// was generated by the `localRebaseGregorianToJulianDays` function.
private val gregJulianDiffSwitchDay = Array(
@@ -157,6 +151,15 @@ object RebaseDateTime {
* Gregorian calendar, interprets the result as a local date in Julian calendar, and takes the
* number of days since the epoch from the Julian local date.
*
+ * This is used to guarantee backward compatibility, as Spark 2.4 and earlier versions use
+ * Julian calendar for dates before 1582-10-15, while Spark 3.0 and later use Proleptic Gregorian
+ * calendar. See SPARK-26651.
+ *
+ * For example:
+ * Proleptic Gregorian calendar: 1582-01-01 -> -141714
+ * Julian calendar: 1582-01-01 -> -141704
+ * The code below converts -141714 to -141704.
+ *
* @param days The number of days since the epoch in Proleptic Gregorian calendar.
* It can be negative.
* @return The rebased number of days in Julian calendar.
@@ -174,22 +177,9 @@ object RebaseDateTime {
}
/**
- * For dates of Before Common Era, Converts the given number of days since the epoch day
- * 1970-01-01 to a local date in Proleptic Gregorian calendar, interprets the result as a local
- * date in Julian calendar, and takes the number of days since the epoch from the Julian
- * local date.
- *
- * For dates of Common Era, the function performs rebasing via the pre-calculated array of diffs
- * between Proleptic Gregorian and Julian calendars.
- *
- * This is used to guarantee backward compatibility, as Spark 2.4 and earlier versions use
- * Julian calendar for dates before 1582-10-15, while Spark 3.0 and later use Proleptic Gregorian
- * calendar. See SPARK-26651.
- *
- * For example:
- * Proleptic Gregorian calendar: 1582-01-01 -> -141714
- * Julian calendar: 1582-01-01 -> -141704
- * The code below converts -141714 to -141704.
+ * An optimized version of [[localRebaseGregorianToJulianDays(Int)]]. This method leverages the
+ * pre-calculated rebasing array to save calculation. For dates of Before Common Era, the
+ * function falls back to the regular unoptimized version.
*
* @param days The number of days since the epoch in Gregorian calendar. It can be negative.
* @return The rebased number of days since the epoch in Julian calendar.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org