You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/11/18 13:22:24 UTC
[GitHub] [spark] beliefer opened a new pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
beliefer opened a new pull request #34647:
URL: https://github.com/apache/spark/pull/34647
### What changes were proposed in this pull request?
Spark not supports TimestampNTZ type in Hive.
```
[info] Caused by: java.lang.IllegalArgumentException: Error: type expected at the position 0 of 'timestamp_ntz:timestamp' but 'timestamp_ntz' is found.[info] Caused by: java.lang.IllegalArgumentException: Error: type expected at the position 0 of 'timestamp_ntz:timestamp' but 'timestamp_ntz' is found.
[info] at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils$TypeInfoParser.expect(TypeInfoUtils.java:372)
[info] at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils$TypeInfoParser.expect(TypeInfoUtils.java:355)
[info] at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils$TypeInfoParser.parseType(TypeInfoUtils.java:416)
[info] at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils$TypeInfoParser.parseTypeInfos(TypeInfoUtils.java:329)
[info] at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils.getTypeInfosFromTypeString(TypeInfoUtils.java:814)
[info] at org.apache.hadoop.hive.serde2.lazy.LazySerDeParameters.extractColumnInfo(LazySerDeParameters.java:162)[info] at org.apache.hadoop.hive.serde2.lazy.LazySerDeParameters.<init>(LazySerDeParameters.java:91)
[info] at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.initialize(LazySimpleSerDe.java:116)
[info] at org.apache.hadoop.hive.serde2.AbstractSerDe.initialize(AbstractSerDe.java:54)
[info] at org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:533)
[info] at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:453)
[info] at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:440)
[info] at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:281)
[info] at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:199)
[info] at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:842)
...
```
Hive only providers the timestamp type, so Spark should write both timestamp_ltz and timestamp_ntz as Hive' timestamp.
When Spark read schema form Hive, We should restore the timestamp_ntz type.
### Why are the changes needed?
The hive 2.3.9 does not have 2 timestamp or a type named timestamp_ntz.
FYI, In hive 3.0, the will be a timestamp with local timezone added.
### Does this PR introduce _any_ user-facing change?
'No'. timestamp_ntz is new and not public yet
### How was this patch tested?
New test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-976149044
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-976210466
**[Test build #145524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145524/testReport)** for PR 34647 at commit [`99b603f`](https://github.com/apache/spark/commit/99b603f7a5b4a52b44e9ebde94c8b3e526e866c2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973382858
**[Test build #145396 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145396/testReport)** for PR 34647 at commit [`195e43c`](https://github.com/apache/spark/commit/195e43c5a4b5470ee211f2a795a2c4ffb8699fdc).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-976111313
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972876007
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145391/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972978812
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49863/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973017066
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49869/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972875975
**[Test build #145391 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145391/testReport)** for PR 34647 at commit [`ed9f858`](https://github.com/apache/spark/commit/ed9f858297374659729811c0f33a9178bfba40a7).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973653633
**[Test build #145427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145427/testReport)** for PR 34647 at commit [`195e43c`](https://github.com/apache/spark/commit/195e43c5a4b5470ee211f2a795a2c4ffb8699fdc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973704785
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49901/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-976211655
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145524/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754268757
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##########
@@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase extends QueryTest with SQLTestUtils with TestHi
}
}
}
+
+ test("SPARK-36180: Support TimestampNTZ type in Hive") {
+ withTable("tb") {
+ val dt = "2018-11-17 13:33:33.0"
+ val ddl =
+ s"CREATE TABLE tb as SELECT TIMESTAMP_LTZ'$dt' as c0, TIMESTAMP_NTZ '$dt' as c1"
Review comment:
Yeah, it's a Hive's limitation. So, Spark can't support `timestamp_ntz` for Hive provider.
If user need to store a table which contains `timestamp_ntz` as a Hive table, the user should manually convert `timestamp_ntz` to `timestamp` if it's really necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754456952
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##########
@@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase extends QueryTest with SQLTestUtils with TestHi
}
}
}
+
+ test("SPARK-36180: Support TimestampNTZ type in Hive") {
+ withTable("tb") {
+ val dt = "2018-11-17 13:33:33.0"
+ val ddl =
+ s"CREATE TABLE tb as SELECT TIMESTAMP_LTZ'$dt' as c0, TIMESTAMP_NTZ '$dt' as c1"
Review comment:
I am +1 with option 2. The benefit is that users can create/read tables with TimestampNTZ columns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972955471
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49863/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973830073
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145427/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-976149044
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973728751
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49901/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973818242
**[Test build #145427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145427/testReport)** for PR 34647 at commit [`195e43c`](https://github.com/apache/spark/commit/195e43c5a4b5470ee211f2a795a2c4ffb8699fdc).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972915920
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49863/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-976098359
**[Test build #145524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145524/testReport)** for PR 34647 at commit [`99b603f`](https://github.com/apache/spark/commit/99b603f7a5b4a52b44e9ebde94c8b3e526e866c2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973037746
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49869/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973451419
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145396/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973653633
**[Test build #145427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145427/testReport)** for PR 34647 at commit [`195e43c`](https://github.com/apache/spark/commit/195e43c5a4b5470ee211f2a795a2c4ffb8699fdc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754252588
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##########
@@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase extends QueryTest with SQLTestUtils with TestHi
}
}
}
+
+ test("SPARK-36180: Support TimestampNTZ type in Hive") {
+ withTable("tb") {
+ val dt = "2018-11-17 13:33:33.0"
+ val ddl =
+ s"CREATE TABLE tb as SELECT TIMESTAMP_LTZ'$dt' as c0, TIMESTAMP_NTZ '$dt' as c1"
Review comment:
Hive serde table is very special, as it includes both data source and metastore. For metastore, the current decision is to store the schema as it is in a hive-incompatible way. For the data source part, this PR proposes to store NTZ as Hive's timestamp.
I think there are 2 options
1. do not support NTZ at all for hive serde tables
2. revisit the decision for metastore. We can still store table with ntz in a hive-compatible way, by storing both ntz and ltz as hive timestamp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754236576
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DataType.scala
##########
@@ -392,6 +392,8 @@ object DataType {
equalsIgnoreCaseAndNullability(l.dataType, r.dataType)
}
+ case (TimestampNTZType, TimestampType) => true
Review comment:
This is definitely not OK. We can't change the semantic of `equalsIgnoreCaseAndNullability`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-976211655
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145524/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754460682
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -1002,15 +1002,19 @@ private[hive] class HiveClientImpl(
private[hive] object HiveClientImpl extends Logging {
/** Converts the native StructField to Hive's FieldSchema. */
def toHiveColumn(c: StructField): FieldSchema = {
+ val field = c.dataType match {
+ case TimestampNTZType => c.copy(dataType = TimestampType)
Review comment:
I find this line confusing. Could you add comments here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754756036
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -1002,15 +1002,19 @@ private[hive] class HiveClientImpl(
private[hive] object HiveClientImpl extends Logging {
/** Converts the native StructField to Hive's FieldSchema. */
def toHiveColumn(c: StructField): FieldSchema = {
+ val field = c.dataType match {
+ case TimestampNTZType => c.copy(dataType = TimestampType)
Review comment:
Store timestamp_ntz in a hive-compatible way, by storing both ntz and ltz as hive timestamp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-976098359
**[Test build #145524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145524/testReport)** for PR 34647 at commit [`99b603f`](https://github.com/apache/spark/commit/99b603f7a5b4a52b44e9ebde94c8b3e526e866c2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754268757
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##########
@@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase extends QueryTest with SQLTestUtils with TestHi
}
}
}
+
+ test("SPARK-36180: Support TimestampNTZ type in Hive") {
+ withTable("tb") {
+ val dt = "2018-11-17 13:33:33.0"
+ val ddl =
+ s"CREATE TABLE tb as SELECT TIMESTAMP_LTZ'$dt' as c0, TIMESTAMP_NTZ '$dt' as c1"
Review comment:
~~Yeah, it's a Hive's limitation. So, Spark can't support `timestamp_ntz` for Hive provider.~~
~~If a user needs to store a table which contains `timestamp_ntz` as a Hive table, the user should manually convert `timestamp_ntz` to `timestamp` if it's really necessary.~~
I misunderstood that Hive's timestamp is ltz. The option 1 seems better in that it keeps Spark simple. But how much is the benefit of option 2 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972978812
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49863/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973652823
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972931556
**[Test build #145396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145396/testReport)** for PR 34647 at commit [`195e43c`](https://github.com/apache/spark/commit/195e43c5a4b5470ee211f2a795a2c4ffb8699fdc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972975616
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49869/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973830073
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145427/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972876007
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145391/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754237594
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##########
@@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase extends QueryTest with SQLTestUtils with TestHi
}
}
}
+
+ test("SPARK-36180: Support TimestampNTZ type in Hive") {
+ withTable("tb") {
+ val dt = "2018-11-17 13:33:33.0"
+ val ddl =
+ s"CREATE TABLE tb as SELECT TIMESTAMP_LTZ'$dt' as c0, TIMESTAMP_NTZ '$dt' as c1"
Review comment:
I think the only problem is we can't create hive-compatible table with timestamp ntz. This seems like an acceptable limitation. What do you think? @sarutak @MaxGekk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972873808
**[Test build #145391 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145391/testReport)** for PR 34647 at commit [`ed9f858`](https://github.com/apache/spark/commit/ed9f858297374659729811c0f33a9178bfba40a7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973037746
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49869/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973728751
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49901/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754268757
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##########
@@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase extends QueryTest with SQLTestUtils with TestHi
}
}
}
+
+ test("SPARK-36180: Support TimestampNTZ type in Hive") {
+ withTable("tb") {
+ val dt = "2018-11-17 13:33:33.0"
+ val ddl =
+ s"CREATE TABLE tb as SELECT TIMESTAMP_LTZ'$dt' as c0, TIMESTAMP_NTZ '$dt' as c1"
Review comment:
Yeah, it's a Hive's limitation. So, Spark can't support `timestamp_ntz` for Hive provider.
If a user needs to store a table which contains `timestamp_ntz` as a Hive table, the user should manually convert `timestamp_ntz` to `timestamp` if it's really necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754268757
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##########
@@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase extends QueryTest with SQLTestUtils with TestHi
}
}
}
+
+ test("SPARK-36180: Support TimestampNTZ type in Hive") {
+ withTable("tb") {
+ val dt = "2018-11-17 13:33:33.0"
+ val ddl =
+ s"CREATE TABLE tb as SELECT TIMESTAMP_LTZ'$dt' as c0, TIMESTAMP_NTZ '$dt' as c1"
Review comment:
~~Yeah, it's a Hive's limitation. So, Spark can't support `timestamp_ntz` for Hive provider.~~
~~If a user needs to store a table which contains `timestamp_ntz` as a Hive table, the user should manually convert `timestamp_ntz` to `timestamp` if it's really necessary.~~
I misunderstood that Hive's timestamp is ltz. The option 1 seems better in that it makes Spark keep simple. But how much is the benefit of option 2 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-976133330
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49996/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972873808
**[Test build #145391 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145391/testReport)** for PR 34647 at commit [`ed9f858`](https://github.com/apache/spark/commit/ed9f858297374659729811c0f33a9178bfba40a7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-972931556
**[Test build #145396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145396/testReport)** for PR 34647 at commit [`195e43c`](https://github.com/apache/spark/commit/195e43c5a4b5470ee211f2a795a2c4ffb8699fdc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973726082
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49901/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973884852
ping @gengliangwang @MaxGekk
cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-973451419
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145396/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #34647:
URL: https://github.com/apache/spark/pull/34647#issuecomment-975306888
Thanks for the ping. I will take a look later today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org