You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/12 10:22:44 UTC
[GitHub] [spark] MaxGekk opened a new pull request #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
MaxGekk opened a new pull request #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890
### What changes were proposed in this pull request?
Move the code related to days rebasing from/to Julian calendar from `HiveInspectors` to new class `DaysWritable`.
### Why are the changes needed?
To improve maintainability of the `HiveInspectors` trait which is already pretty complex.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
By `HiveOrcHadoopFsRelationSuite`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598383978
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24452/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598151672
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598610103
**[Test build #119749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119749/testReport)** for PR 27890 at commit [`a017fbe`](https://github.com/apache/spark/commit/a017fbee3507125a484c90a47d21918018714497).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598782800
**[Test build #119761 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119761/testReport)** for PR 27890 at commit [`f606899`](https://github.com/apache/spark/commit/f6068992d16b3cd55f02658d0f9e2bcd52cb1be3).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598783428
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119761/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598151532
**[Test build #119713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119713/testReport)** for PR 27890 at commit [`2000387`](https://github.com/apache/spark/commit/2000387cccb7d12769a2538ee4cd96d8c37068e0).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class DaysWritable(var gregorianDays: Int = 0) extends DateWritable `
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598456409
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598610467
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598717245
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24490/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598610477
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24478/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598666495
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598297632
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#discussion_r392090623
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala
##########
@@ -646,14 +617,7 @@ private[hive] trait HiveInspectors {
case x: DateObjectInspector if x.preferWritable() =>
data: Any => {
if (data != null) {
- // Rebasing written days via conversion to local dates.
- // See the comment for `getDateWritable()`.
- val daysSinceEpoch = x.getPrimitiveWritableObject(data).getDays
- if (daysSinceEpoch < JULIAN_CUTOVER_DAY) {
- rebaseJulianToGregorianDays(daysSinceEpoch)
- } else {
- daysSinceEpoch
- }
+ x.getPrimitiveWritableObject(data).getDays
Review comment:
You are right, `DateWritable` can come here. I have fixed that by adding new constructor to `DaysWritable` to construct its instances from `DateWritable`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598717236
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#discussion_r392162971
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/DaysWritable.scala
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive
+
+import java.io.{DataInput, DataOutput, IOException}
+import java.sql.Date
+import java.time.LocalDate
+import java.util.Calendar
+
+import org.apache.hadoop.hive.serde2.io.DateWritable
+import org.apache.hadoop.io.WritableUtils
+
+import org.apache.spark.sql.catalyst.util.{DateTimeConstants, DateTimeUtils}
+
+/**
+ * The class accepts/returns days in Gregorian calendar and rebase them
+ * via conversion to local date in Julian calendar for dates before 1582-10-15
+ * in read/write for backward compatibility with Spark 2.4 and earlier versions.
+ *
+ * @param gregorianDays The number of days since the epoch 1970-01-01 in
+ * Gregorian calendar.
+ */
+private[hive] class DaysWritable(var gregorianDays: Int = 0) extends DateWritable {
+
+ def this(dateWritable: DateWritable) = {
+ this(dateWritable match {
+ case daysWritable: DaysWritable => daysWritable.gregorianDays
+ case dateWritable: DateWritable =>
+ DaysWritable.rebaseJulianToGregorianDays(dateWritable.getDays)
+ })
+ }
+
+ override def getDays: Int = gregorianDays
+ override def get(): Date = {
+ val days = DaysWritable.rebaseGregorianToJulianDays(gregorianDays)
+ new Date(DateWritable.daysToMillis(days))
+ }
+
+ @throws[IOException]
+ override def write(out: DataOutput): Unit = {
+ // Rebasing days since the epoch to store the same number of days
+ // as by Spark 2.4 and earlier versions. Spark 3.0 switched to
+ // Proleptic Gregorian calendar (see SPARK-26651), and as a consequence of that,
+ // this affects dates before 1582-10-15. Spark 2.4 and earlier versions use
+ // Julian calendar for dates before 1582-10-15. So, the same local date may
+ // be mapped to different number of days since the epoch in different calendars.
+ // For example:
+ // Proleptic Gregorian calendar: 1582-01-01 -> -141714
+ // Julian calendar: 1582-01-01 -> -141704
+ // The code below converts -141714 to -141704.
+ val writeDays = DaysWritable.rebaseGregorianToJulianDays(gregorianDays)
Review comment:
shall we cache the `julianDays`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598665915
**[Test build #119749 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119749/testReport)** for PR 27890 at commit [`a017fbe`](https://github.com/apache/spark/commit/a017fbee3507125a484c90a47d21918018714497).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598783416
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598296925
**[Test build #119720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119720/testReport)** for PR 27890 at commit [`af4549e`](https://github.com/apache/spark/commit/af4549edf4fdbd906e94a6d6a468853c6766fa3b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598456415
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119725/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598610477
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24478/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598717236
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598783428
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119761/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598574583
@cloud-fan @HyukjinKwon Please, take a look at this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598112452
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598112457
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24442/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598112457
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24442/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598362456
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598383978
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24452/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598455204
**[Test build #119725 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119725/testReport)** for PR 27890 at commit [`af4549e`](https://github.com/apache/spark/commit/af4549edf4fdbd906e94a6d6a468853c6766fa3b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598383972
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598783416
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598151672
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598383351
**[Test build #119725 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119725/testReport)** for PR 27890 at commit [`af4549e`](https://github.com/apache/spark/commit/af4549edf4fdbd906e94a6d6a468853c6766fa3b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598296925
**[Test build #119720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119720/testReport)** for PR 27890 at commit [`af4549e`](https://github.com/apache/spark/commit/af4549edf4fdbd906e94a6d6a468853c6766fa3b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-599422464
thanks, merging to master/3.0!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598666507
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119749/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598716633
**[Test build #119761 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119761/testReport)** for PR 27890 at commit [`f606899`](https://github.com/apache/spark/commit/f6068992d16b3cd55f02658d0f9e2bcd52cb1be3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#discussion_r392057460
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala
##########
@@ -646,14 +617,7 @@ private[hive] trait HiveInspectors {
case x: DateObjectInspector if x.preferWritable() =>
data: Any => {
if (data != null) {
- // Rebasing written days via conversion to local dates.
- // See the comment for `getDateWritable()`.
- val daysSinceEpoch = x.getPrimitiveWritableObject(data).getDays
- if (daysSinceEpoch < JULIAN_CUTOVER_DAY) {
- rebaseJulianToGregorianDays(daysSinceEpoch)
- } else {
- daysSinceEpoch
- }
+ x.getPrimitiveWritableObject(data).getDays
Review comment:
What if the returned `DateWritable` here is not our custom version?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598151676
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119713/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598362464
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119720/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598297640
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24448/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598716633
**[Test build #119761 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119761/testReport)** for PR 27890 at commit [`f606899`](https://github.com/apache/spark/commit/f6068992d16b3cd55f02658d0f9e2bcd52cb1be3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#discussion_r392162346
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/DaysWritable.scala
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive
+
+import java.io.{DataInput, DataOutput, IOException}
+import java.sql.Date
+import java.time.LocalDate
+import java.util.Calendar
+
+import org.apache.hadoop.hive.serde2.io.DateWritable
+import org.apache.hadoop.io.WritableUtils
+
+import org.apache.spark.sql.catalyst.util.{DateTimeConstants, DateTimeUtils}
+
+/**
+ * The class accepts/returns days in Gregorian calendar and rebase them
+ * via conversion to local date in Julian calendar for dates before 1582-10-15
+ * in read/write for backward compatibility with Spark 2.4 and earlier versions.
+ *
+ * @param gregorianDays The number of days since the epoch 1970-01-01 in
+ * Gregorian calendar.
+ */
+private[hive] class DaysWritable(var gregorianDays: Int = 0) extends DateWritable {
+
+ def this(dateWritable: DateWritable) = {
+ this(dateWritable match {
+ case daysWritable: DaysWritable => daysWritable.gregorianDays
+ case dateWritable: DateWritable =>
+ DaysWritable.rebaseJulianToGregorianDays(dateWritable.getDays)
+ })
+ }
+
+ override def getDays: Int = gregorianDays
Review comment:
should we return `julianDays` here to conform to the hive API?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598610467
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598297632
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598383351
**[Test build #119725 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119725/testReport)** for PR 27890 at commit [`af4549e`](https://github.com/apache/spark/commit/af4549edf4fdbd906e94a6d6a468853c6766fa3b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598112452
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#discussion_r392221888
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/DaysWritable.scala
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive
+
+import java.io.{DataInput, DataOutput, IOException}
+import java.sql.Date
+import java.time.LocalDate
+import java.util.Calendar
+
+import org.apache.hadoop.hive.serde2.io.DateWritable
+import org.apache.hadoop.io.WritableUtils
+
+import org.apache.spark.sql.catalyst.util.{DateTimeConstants, DateTimeUtils}
+
+/**
+ * The class accepts/returns days in Gregorian calendar and rebase them
+ * via conversion to local date in Julian calendar for dates before 1582-10-15
+ * in read/write for backward compatibility with Spark 2.4 and earlier versions.
+ *
+ * @param gregorianDays The number of days since the epoch 1970-01-01 in
+ * Gregorian calendar.
+ */
+private[hive] class DaysWritable(var gregorianDays: Int = 0) extends DateWritable {
+
+ def this(dateWritable: DateWritable) = {
+ this(dateWritable match {
+ case daysWritable: DaysWritable => daysWritable.gregorianDays
+ case dateWritable: DateWritable =>
+ DaysWritable.rebaseJulianToGregorianDays(dateWritable.getDays)
+ })
+ }
+
+ override def getDays: Int = gregorianDays
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598382901
jenkins, retest this, please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598610103
**[Test build #119749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119749/testReport)** for PR 27890 at commit [`a017fbe`](https://github.com/apache/spark/commit/a017fbee3507125a484c90a47d21918018714497).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598666507
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119749/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598111962
**[Test build #119713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119713/testReport)** for PR 27890 at commit [`2000387`](https://github.com/apache/spark/commit/2000387cccb7d12769a2538ee4cd96d8c37068e0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598362464
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119720/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598456409
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598456415
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119725/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-599424050
+1 Looks good to me too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#discussion_r392222004
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/DaysWritable.scala
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive
+
+import java.io.{DataInput, DataOutput, IOException}
+import java.sql.Date
+import java.time.LocalDate
+import java.util.Calendar
+
+import org.apache.hadoop.hive.serde2.io.DateWritable
+import org.apache.hadoop.io.WritableUtils
+
+import org.apache.spark.sql.catalyst.util.{DateTimeConstants, DateTimeUtils}
+
+/**
+ * The class accepts/returns days in Gregorian calendar and rebase them
+ * via conversion to local date in Julian calendar for dates before 1582-10-15
+ * in read/write for backward compatibility with Spark 2.4 and earlier versions.
+ *
+ * @param gregorianDays The number of days since the epoch 1970-01-01 in
+ * Gregorian calendar.
+ */
+private[hive] class DaysWritable(var gregorianDays: Int = 0) extends DateWritable {
+
+ def this(dateWritable: DateWritable) = {
+ this(dateWritable match {
+ case daysWritable: DaysWritable => daysWritable.gregorianDays
+ case dateWritable: DateWritable =>
+ DaysWritable.rebaseJulianToGregorianDays(dateWritable.getDays)
+ })
+ }
+
+ override def getDays: Int = gregorianDays
+ override def get(): Date = {
+ val days = DaysWritable.rebaseGregorianToJulianDays(gregorianDays)
+ new Date(DateWritable.daysToMillis(days))
+ }
+
+ @throws[IOException]
+ override def write(out: DataOutput): Unit = {
+ // Rebasing days since the epoch to store the same number of days
+ // as by Spark 2.4 and earlier versions. Spark 3.0 switched to
+ // Proleptic Gregorian calendar (see SPARK-26651), and as a consequence of that,
+ // this affects dates before 1582-10-15. Spark 2.4 and earlier versions use
+ // Julian calendar for dates before 1582-10-15. So, the same local date may
+ // be mapped to different number of days since the epoch in different calendars.
+ // For example:
+ // Proleptic Gregorian calendar: 1582-01-01 -> -141714
+ // Julian calendar: 1582-01-01 -> -141704
+ // The code below converts -141714 to -141704.
+ val writeDays = DaysWritable.rebaseGregorianToJulianDays(gregorianDays)
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598362456
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598717245
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24490/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598297640
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24448/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598111962
**[Test build #119713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119713/testReport)** for PR 27890 at commit [`2000387`](https://github.com/apache/spark/commit/2000387cccb7d12769a2538ee4cd96d8c37068e0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598383972
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598666495
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598362239
**[Test build #119720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119720/testReport)** for PR 27890 at commit [`af4549e`](https://github.com/apache/spark/commit/af4549edf4fdbd906e94a6d6a468853c6766fa3b).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27890:
[SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27890: [SPARK-31076][SQL][FOLLOWUP] Incapsulate date rebasing to `DaysWritable`
URL: https://github.com/apache/spark/pull/27890#issuecomment-598151676
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119713/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org