You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by we...@apache.org on 2020/04/14 08:08:00 UTC
[spark] branch master updated: [SPARK-31402][SQL][FOLLOWUP] Refine
code comments in RebaseDateTime
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 6b88d13 [SPARK-31402][SQL][FOLLOWUP] Refine code comments in RebaseDateTime
6b88d13 is described below
commit 6b88d136deb99afd9363b208fd6fe5684fe8c3b8
Author: Wenchen Fan <we...@databricks.com>
AuthorDate: Tue Apr 14 08:06:55 2020 +0000
[SPARK-31402][SQL][FOLLOWUP] Refine code comments in RebaseDateTime
### What changes were proposed in this pull request?
Refine the code comments of days rebasing, to be consistent with the micros rebasing. i.e. one method is the actual implementation and the other variant is the optimized version.
### Why are the changes needed?
improve code comments
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
N/A
Closes #28199 from cloud-fan/comment.
Authored-by: Wenchen Fan <we...@databricks.com>
Signed-off-by: Wenchen Fan <we...@databricks.com>
---
.../spark/sql/catalyst/util/RebaseDateTime.scala | 62 +++++++++-------------
1 file changed, 26 insertions(+), 36 deletions(-)
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala
index 37dd0d7..8ee6c87 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala
@@ -64,8 +64,7 @@ object RebaseDateTime {
// The sorted days in Julian calendar when difference in days between Julian and
// Proleptic Gregorian calendars was changed.
// The starting point is the `0001-01-01` (-719164 days since the epoch in
- // Julian calendar). All dates before the staring point have the same difference
- // of 2 days in Julian and Proleptic Gregorian calendars.
+ // Julian calendar). This array is not applicable for dates before the staring point.
// Rebasing switch days and diffs `julianGregDiffSwitchDay` and `julianGregDiffs`
// was generated by the `localRebaseJulianToGregorianDays` function.
private val julianGregDiffSwitchDay = Array(
@@ -80,6 +79,15 @@ object RebaseDateTime {
* calendar, interprets the result as a local date in Proleptic Gregorian calendar, and takes the
* number of days since the epoch from the Gregorian local date.
*
+ * This is used to guarantee backward compatibility, as Spark 2.4 and earlier versions use
+ * Julian calendar for dates before 1582-10-15, while Spark 3.0 and later use Proleptic Gregorian
+ * calendar. See SPARK-26651.
+ *
+ * For example:
+ * Julian calendar: 1582-01-01 -> -141704
+ * Proleptic Gregorian calendar: 1582-01-01 -> -141714
+ * The code below converts -141704 to -141714.
+ *
* @param days The number of days since the epoch in Julian calendar. It can be negative.
* @return The rebased number of days in Gregorian calendar.
*/
@@ -105,22 +113,9 @@ object RebaseDateTime {
}
/**
- * For dates of Before Common Era, the function converts the given number of days since the epoch
- * day 1970-01-01 to a local date in Julian calendar, interprets the result as a local date in
- * Proleptic Gregorian calendar, and takes the number of days since the epoch from the Gregorian
- * local date.
- *
- * For dates of Common Era, the function performs rebasing via the pre-calculated array of diffs
- * between Julian and Proleptic Gregorian calendars.
- *
- * This is used to guarantee backward compatibility, as Spark 2.4 and earlier versions use
- * Julian calendar for dates before 1582-10-15, while Spark 3.0 and later use Proleptic Gregorian
- * calendar. See SPARK-26651.
- *
- * For example:
- * Julian calendar: 1582-01-01 -> -141704
- * Proleptic Gregorian calendar: 1582-01-01 -> -141714
- * The code below converts -141704 to -141714.
+ * An optimized version of [[localRebaseJulianToGregorianDays(Int)]]. This method leverages the
+ * pre-calculated rebasing array to save calculation. For dates of Before Common Era, the
+ * function falls back to the regular unoptimized version.
*
* @param days The number of days since the epoch in Julian calendar. It can be negative.
* @return The rebased number of days in Gregorian calendar.
@@ -140,8 +135,7 @@ object RebaseDateTime {
// The sorted days in Proleptic Gregorian calendar when difference in days between
// Proleptic Gregorian and Julian was changed.
// The starting point is the `0001-01-01` (-719162 days since the epoch in
- // Proleptic Gregorian calendar). All dates before the staring point have the same
- // difference of -2 days in Proleptic Gregorian and Julian calendars.
+ // Proleptic Gregorian calendar). This array is not applicable for dates before the staring point.
// Rebasing switch days and diffs `gregJulianDiffSwitchDay` and `gregJulianDiffs`
// was generated by the `localRebaseGregorianToJulianDays` function.
private val gregJulianDiffSwitchDay = Array(
@@ -157,6 +151,15 @@ object RebaseDateTime {
* Gregorian calendar, interprets the result as a local date in Julian calendar, and takes the
* number of days since the epoch from the Julian local date.
*
+ * This is used to guarantee backward compatibility, as Spark 2.4 and earlier versions use
+ * Julian calendar for dates before 1582-10-15, while Spark 3.0 and later use Proleptic Gregorian
+ * calendar. See SPARK-26651.
+ *
+ * For example:
+ * Proleptic Gregorian calendar: 1582-01-01 -> -141714
+ * Julian calendar: 1582-01-01 -> -141704
+ * The code below converts -141714 to -141704.
+ *
* @param days The number of days since the epoch in Proleptic Gregorian calendar.
* It can be negative.
* @return The rebased number of days in Julian calendar.
@@ -174,22 +177,9 @@ object RebaseDateTime {
}
/**
- * For dates of Before Common Era, Converts the given number of days since the epoch day
- * 1970-01-01 to a local date in Proleptic Gregorian calendar, interprets the result as a local
- * date in Julian calendar, and takes the number of days since the epoch from the Julian
- * local date.
- *
- * For dates of Common Era, the function performs rebasing via the pre-calculated array of diffs
- * between Proleptic Gregorian and Julian calendars.
- *
- * This is used to guarantee backward compatibility, as Spark 2.4 and earlier versions use
- * Julian calendar for dates before 1582-10-15, while Spark 3.0 and later use Proleptic Gregorian
- * calendar. See SPARK-26651.
- *
- * For example:
- * Proleptic Gregorian calendar: 1582-01-01 -> -141714
- * Julian calendar: 1582-01-01 -> -141704
- * The code below converts -141714 to -141704.
+ * An optimized version of [[localRebaseGregorianToJulianDays(Int)]]. This method leverages the
+ * pre-calculated rebasing array to save calculation. For dates of Before Common Era, the
+ * function falls back to the regular unoptimized version.
*
* @param days The number of days since the epoch in Gregorian calendar. It can be negative.
* @return The rebased number of days since the epoch in Julian calendar.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org