You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/02/08 20:17:55 UTC
[GitHub] [spark] MaxGekk opened a new pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
MaxGekk opened a new pull request #31529:
URL: https://github.com/apache/spark/pull/31529
### What changes were proposed in this pull request?
In the PR, I propose new option `datetimeRebaseMode` for the Avro datasource. The option influences on loading ancient dates and timestamps column values from avro files.
The option supports the same values as the SQL config `spark.sql.legacy.avro.datetimeRebaseModeInRead` namely;
- `"LEGACY"`, when an option is set to this value, Spark rebases dates/timestamps from the legacy hybrid calendar (Julian + Gregorian) to the Proleptic Gregorian calendar.
- `"CORRECTED"`, dates/timestamps are read AS IS from avro files.
- `"EXCEPTION"`, when it is set as an option value, Spark will fail the reading if it sees ancient dates/timestamps that are ambiguous between the two calendars.
### Why are the changes needed?
1. New options will allow to load avro files from at least two sources in different rebasing modes in the same query. For instance:
```scala
val df1 = spark.read.option("datetimeRebaseMode", "legacy").format("avro").load(folder1)
val df2 = spark.read.option("datetimeRebaseMode", "corrected").format("avro").load(folder2)
df1.join(df2, ...)
```
Before the changes, it is impossible because the SQL config `spark.sql.legacy.avro.datetimeRebaseModeInRead` influences on both reads.
2. Mixing of Dataset/DataFrame and RDD APIs should become possible. Since SQL configs are not propagated through RDDs, the following code fails on ancient timestamps:
```scala
spark.conf.set("spark.sql.legacy.avro.datetimeRebaseModeInRead", "legacy")
spark.read.format("avro").load(folder).distinct.rdd.collect()
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
By running the modified test suites:
```
$ build/sbt "test:testOnly *AvroV1Suite"
$ build/sbt "test:testOnly *AvroV2Suite"
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776460420
@cloud-fan @gengliangwang @HyukjinKwon This PR is a companion to https://github.com/apache/spark/pull/31489, and should solve the same issues. Any concerns about it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775517184
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39628/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775586246
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39634/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775555761
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39630/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775580868
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135051/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775468802
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135045/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776191660
**[Test build #135080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135080/testReport)** for PR 31529 at commit [`c930fd9`](https://github.com/apache/spark/commit/c930fd975575e9e4f42a180cd42da6a25e7322c6).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775571042
**[Test build #135051 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135051/testReport)** for PR 31529 at commit [`1e3e524`](https://github.com/apache/spark/commit/1e3e5246c476ce5b7dce1222b6caf44828984696).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776476030
thanks, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775503416
**[Test build #135047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135047/testReport)** for PR 31529 at commit [`1e3e524`](https://github.com/apache/spark/commit/1e3e5246c476ce5b7dce1222b6caf44828984696).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776234467
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39662/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776191660
**[Test build #135080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135080/testReport)** for PR 31529 at commit [`c930fd9`](https://github.com/apache/spark/commit/c930fd975575e9e4f42a180cd42da6a25e7322c6).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775580868
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135051/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776218688
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135080/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776252894
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39662/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775510620
**[Test build #135047 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135047/testReport)** for PR 31529 at commit [`1e3e524`](https://github.com/apache/spark/commit/1e3e5246c476ce5b7dce1222b6caf44828984696).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775553799
jenkins, retest this, please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775574877
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39634/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775436704
**[Test build #135045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135045/testReport)** for PR 31529 at commit [`79a8a59`](https://github.com/apache/spark/commit/79a8a597bec72778b9ad73fe0336d42548cad51e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775528859
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31529:
URL: https://github.com/apache/spark/pull/31529
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775561922
**[Test build #135051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135051/testReport)** for PR 31529 at commit [`1e3e524`](https://github.com/apache/spark/commit/1e3e5246c476ce5b7dce1222b6caf44828984696).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775503416
**[Test build #135047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135047/testReport)** for PR 31529 at commit [`1e3e524`](https://github.com/apache/spark/commit/1e3e5246c476ce5b7dce1222b6caf44828984696).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775424148
@cloud-fan @HyukjinKwon Could you review this PR, please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775528858
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775605752
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39634/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775469874
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39628/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775561922
**[Test build #135051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135051/testReport)** for PR 31529 at commit [`1e3e524`](https://github.com/apache/spark/commit/1e3e5246c476ce5b7dce1222b6caf44828984696).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776217578
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39662/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776218688
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135080/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775605752
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39634/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775449509
**[Test build #135045 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135045/testReport)** for PR 31529 at commit [`79a8a59`](https://github.com/apache/spark/commit/79a8a597bec72778b9ad73fe0336d42548cad51e).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775555761
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39630/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776252894
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39662/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk edited a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
MaxGekk edited a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775424148
@cloud-fan @gengliangwang @HyukjinKwon Could you review this PR, please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-776205646
**[Test build #135080 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135080/testReport)** for PR 31529 at commit [`c930fd9`](https://github.com/apache/spark/commit/c930fd975575e9e4f42a180cd42da6a25e7322c6).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775436704
**[Test build #135045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135045/testReport)** for PR 31529 at commit [`79a8a59`](https://github.com/apache/spark/commit/79a8a597bec72778b9ad73fe0336d42548cad51e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31529:
URL: https://github.com/apache/spark/pull/31529#issuecomment-775468802
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135045/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org