You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/12/08 00:26:36 UTC
[GitHub] [spark] wangyum opened a new pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
wangyum opened a new pull request #30657:
URL: https://github.com/apache/spark/pull/30657
### What changes were proposed in this pull request?
Hive 2.3.8 changes:
HIVE-19662: Upgrade Avro to 1.8.2
HIVE-24324: Remove deprecated API usage from Avro
HIVE-23980: Shade Guava from hive-exec in Hive 2.3
HIVE-24436: Fix Avro NULL_DEFAULT_VALUE compatibility issue
### Why are the changes needed?
Upgrade Avro and Parquet to latest version.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
// TODO
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756646001
retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-757106854
**[Test build #133861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133861/testReport)** for PR 30657 at commit [`2248a9c`](https://github.com/apache/spark/commit/2248a9c58757bc3b27d1568a9bc79ba011309344).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761932948
**[Test build #134170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134170/testReport)** for PR 30657 at commit [`bb418bb`](https://github.com/apache/spark/commit/bb418bbf3a76e2c5ff47e0e4fcea318e6b2eec69).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745245995
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37425/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-757109462
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133861/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-757093046
**[Test build #133861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133861/testReport)** for PR 30657 at commit [`2248a9c`](https://github.com/apache/spark/commit/2248a9c58757bc3b27d1568a9bc79ba011309344).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741485504
for comparison, the binary distribution from Hive master branch doesn't include any calcite jars, but I'm seeing:
```
calcite-core-1.10.0.jar
calcite-druid-1.10.0.jar
calcite-linq4j-1.10.0.jar
```
in 2.3.8 binary distribution.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742855756
**[Test build #132596 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132596/testReport)** for PR 30657 at commit [`8a4e20e`](https://github.com/apache/spark/commit/8a4e20e7fa4b697e91a9b38e8c3cc93762018b11).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-757098860
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38450/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742515965
retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761978693
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134170/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742912145
**[Test build #132608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132608/testReport)** for PR 30657 at commit [`127cb2d`](https://github.com/apache/spark/commit/127cb2dd900144742e471cb32447d3c7d5d2f2fe).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-740779728
> ```
> [info] - 2.3: loadTable *** FAILED *** (180 milliseconds)
> [info] java.lang.reflect.InvocationTargetException:
> ```
Yea, this was we encountered in internal test. So as https://github.com/apache/spark/pull/30657#discussion_r537961979, there must be a separated `HiveVersion` for 2.3.8 to solve these errors.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #30657:
URL: https://github.com/apache/spark/pull/30657#discussion_r537961979
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala
##########
@@ -102,7 +102,7 @@ package object client {
// Since HIVE-14496, Hive materialized view need calcite-core.
// For spark, only VersionsSuite currently creates a hive materialized view for testing.
- case object v2_3 extends HiveVersion("2.3.7",
+ case object v2_3 extends HiveVersion("2.3.8",
Review comment:
Per local test, this needs to be changed to pass all tests. But let's see the Jenkins result first.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756675525
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761978693
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134170/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742319757
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37153/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756776884
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38429/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-740276137
Yey!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742912145
**[Test build #132608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132608/testReport)** for PR 30657 at commit [`127cb2d`](https://github.com/apache/spark/commit/127cb2dd900144742e471cb32447d3c7d5d2f2fe).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761292563
**[Test build #134133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134133/testReport)** for PR 30657 at commit [`bb418bb`](https://github.com/apache/spark/commit/bb418bbf3a76e2c5ff47e0e4fcea318e6b2eec69).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745186570
**[Test build #132823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132823/testReport)** for PR 30657 at commit [`127cb2d`](https://github.com/apache/spark/commit/127cb2dd900144742e471cb32447d3c7d5d2f2fe).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742855756
**[Test build #132596 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132596/testReport)** for PR 30657 at commit [`8a4e20e`](https://github.com/apache/spark/commit/8a4e20e7fa4b697e91a9b38e8c3cc93762018b11).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745245995
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37425/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-757096934
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38450/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742952217
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132608/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756808690
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133840/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742172991
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37127/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742284885
**[Test build #132549 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132549/testReport)** for PR 30657 at commit [`34a4da2`](https://github.com/apache/spark/commit/34a4da281be90514a5d879fcf86ddf54a64b01ef).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756646001
retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-757098860
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38450/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741375885
@viirya @sunchao Can you reproduce this issue? Hadoop 2.7.7 + Hive 2.3.8:
```
[root@spark-3267648 apache-hive-2.3.8-bin]# bin/hive
which: no hbase in (/usr/lib/jdk1.8.0_221/bin:/usr/lib/maven-3.6.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/apache-hive-2.3.8-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Logging initialized using configuration in jar:file:/root/apache-hive-2.3.8-bin/lib/hive-common-2.3.8.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive> CREATE TABLE tbl AS SELECT 1 AS a;
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = root_20201208181102_11753a84-c915-41ae-8865-1d2240e004b1
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
Job running in-process (local Hadoop)
2020-12-08 18:11:08,462 Stage-1 map = 100%, reduce = 0%
Ended Job = job_local904330736_0001
Stage-4 is selected by condition resolver.
Stage-3 is filtered out by condition resolver.
Stage-5 is filtered out by condition resolver.
Moving data to directory file:/user/hive/warehouse/.hive-staging_hive_2020-12-08_18-11-02_638_1954554869364101742-1/-ext-10002
Moving data to directory file:/user/hive/warehouse/tbl
MapReduce Jobs Launched:
Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 0 msec
OK
Time taken: 6.484 seconds
hive> select * from tbl;
Exception in thread "main" java.lang.AssertionError: Internal error: While invoking method 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
at org.apache.calcite.util.Util.newInternal(Util.java:792)
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
at org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
at org.apache.calcite.sql2rel.RelFieldTrimmer.trim(RelFieldTrimmer.java:160)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1331)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1261)
at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113)
at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:997)
at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149)
at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1069)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1085)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:364)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11138)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:226)
at org.apache.hadoop.util.RunJar.main(RunJar.java:141)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
... 31 more
Caused by: java.lang.NoSuchMethodError: org.apache.calcite.rel.RelCollationImpl.<init>(Lorg/apache/hive/com/google/common/collect/ImmutableList;)V
at org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelCollation.<init>(HiveRelCollation.java:29)
at org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getCollationList(RelOptHiveTable.java:181)
at org.apache.calcite.rel.metadata.RelMdCollation.table(RelMdCollation.java:175)
at org.apache.calcite.rel.metadata.RelMdCollation.collations(RelMdCollation.java:122)
at GeneratedMetadataHandler_Collation.collations_$(Unknown Source)
at GeneratedMetadataHandler_Collation.collations(Unknown Source)
at org.apache.calcite.rel.metadata.RelMetadataQuery.collations(RelMetadataQuery.java:482)
at org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:189)
at org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:273)
... 36 more
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #30657: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #30657:
URL: https://github.com/apache/spark/pull/30657
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742204022
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132524/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756686137
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38421/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742870761
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37201/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742731781
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132560/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-757093046
**[Test build #133861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133861/testReport)** for PR 30657 at commit [`2248a9c`](https://github.com/apache/spark/commit/2248a9c58757bc3b27d1568a9bc79ba011309344).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #30657:
URL: https://github.com/apache/spark/pull/30657#discussion_r537961979
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala
##########
@@ -102,7 +102,7 @@ package object client {
// Since HIVE-14496, Hive materialized view need calcite-core.
// For spark, only VersionsSuite currently creates a hive materialized view for testing.
- case object v2_3 extends HiveVersion("2.3.7",
+ case object v2_3 extends HiveVersion("2.3.8",
Review comment:
Per the test internally, this needs to be changed to pass all tests. But let's see the Jenkins result first.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756672893
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38421/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742898220
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132596/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-740305969
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36989/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756808690
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133840/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742441754
**[Test build #132560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132560/testReport)** for PR 30657 at commit [`8a4e20e`](https://github.com/apache/spark/commit/8a4e20e7fa4b697e91a9b38e8c3cc93762018b11).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761699361
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761962810
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38754/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745186570
**[Test build #132823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132823/testReport)** for PR 30657 at commit [`127cb2d`](https://github.com/apache/spark/commit/127cb2dd900144742e471cb32447d3c7d5d2f2fe).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dbtsai commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
dbtsai commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745050668
Thank you all for working on this!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-740322999
**[Test build #132389 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132389/testReport)** for PR 30657 at commit [`fc9f735`](https://github.com/apache/spark/commit/fc9f73538378033f148903d43dfff21c0fb81112).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum edited a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum edited a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742398492
@sunchao @viirya Could we do not include `com.fasterxml.jackson`?
Hive 2.3.8 | Hive 2.3.7
--- | ---
 | 
Hive 2.3.8 load class from `hive-exec-2.3.8.jar`:
```
hive class: com.fasterxml.jackson.annotation.JsonFormat$Value - jar:file:/root/opensource/spark/sql/hive/target/tmp/org.apache.spark.sql.hive.client.VersionsSuite/hive-v2_3-059f3d91-4a54-43cb-8382-6446765c2446/org.apache.hive_hive-exec-2.3.8.jar!/com/fasterxml/jackson/annotation/JsonFormat$Value.class
```
Hive 2.3.7 load class from `jackson-annotations-2.9.5.jar`:
```
hive class: com.fasterxml.jackson.annotation.JsonFormat$Value - jar:file:/root/opensource/spark/sql/hive/target/tmp/org.apache.spark.sql.hive.client.VersionsSuite/hive-v2_3-768ce706-3000-4b11-a8a3-6dd721494904/com.fasterxml.jackson.core_jackson-annotations-2.9.5.jar!/com/fasterxml/jackson/annotation/JsonFormat$Value.class
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742952217
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132608/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741466091
@wangyum @viirya yes I was able to reproduce the same issue. I think it could be cause we have `RelCollationImpl` included in both `hive-exec` jar and `calcite-core` jar. Perhaps we should exclude the latter from the Hive binary distribution and classpath.
I'll update the release vote email thread and start a new RC once this is fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745732487
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37451/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756680731
**[Test build #133832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133832/testReport)** for PR 30657 at commit [`e0632ca`](https://github.com/apache/spark/commit/e0632cac19c403a31ba16490fa90e6742889d9a1).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class SparkPod(pod: Pod, container: Container) `
* `trait KubernetesFeatureConfigStep `
* `public class Distributions `
* `case class Decode(params: Seq[Expression], child: Expression) extends RuntimeReplaceable `
* `case class StringDecode(bin: Expression, charset: Expression)`
* `case class AlterTableRecoverPartitions(child: LogicalPlan) extends Command `
* `case class AlterViewAs(`
* `case class CacheTable(`
* `case class CacheTableAsSelect(`
* `case class SubqueryExec(name: String, child: SparkPlan, maxNumRows: Option[Int] = None)`
* `trait BaseCacheTableExec extends V2CommandExec `
* `case class CacheTableExec(`
* `case class CacheTableAsSelectExec(`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741476642
Thank you @sunchao .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745750363
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37451/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745274515
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132823/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761685669
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745716241
**[Test build #132849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132849/testReport)** for PR 30657 at commit [`e0632ca`](https://github.com/apache/spark/commit/e0632cac19c403a31ba16490fa90e6742889d9a1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756729755
**[Test build #133840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133840/testReport)** for PR 30657 at commit [`fce63a1`](https://github.com/apache/spark/commit/fce63a1293bb44f19a1f849b793647ef24bbe4f7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742144532
retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742158223
**[Test build #132524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132524/testReport)** for PR 30657 at commit [`fc9f735`](https://github.com/apache/spark/commit/fc9f73538378033f148903d43dfff21c0fb81112).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742948995
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37212/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742940127
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37212/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741476895
> @wangyum @viirya yes I was able to reproduce the same issue. I think it could be cause we have `RelCollationImpl` included in both `hive-exec` jar and `calcite-core` jar. Perhaps we should exclude the latter from the Hive binary distribution and classpath.
>
> I'll update the release vote email thread and start a new RC once this is fixed.
We have included `org.apache.calcite:*` in Hive shaded jar. The following error looks like there is other calcite jar in the classpath so it conflicts with the shaded calcite.
```
Caused by: java.lang.NoSuchMethodError: org.apache.calcite.rel.RelCollationImpl.<init>(Lorg/apache/hive/com/google/common/collect/ImmutableList;)V
at org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelCollation.<init>(HiveRelCollation.java:29)
at org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getCollationList(RelOptHiveTable.java:181)
at org.apache.calcite.rel.metadata.RelMdCollation.table(RelMdCollation.java:175)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742470305
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37164/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742531846
**[Test build #132571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132571/testReport)** for PR 30657 at commit [`8a4e20e`](https://github.com/apache/spark/commit/8a4e20e7fa4b697e91a9b38e8c3cc93762018b11).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742854293
retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on a change in pull request #30657:
URL: https://github.com/apache/spark/pull/30657#discussion_r553920965
##########
File path: pom.xml
##########
@@ -1844,6 +1855,22 @@
<groupId>org.apache.logging.log4j</groupId>
<artifactId>*</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>net.hydromatic</groupId>
+ <artifactId>eigenbase-properties</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.codehaus.janino</groupId>
+ <artifactId>commons-compiler</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.codehaus.janino</groupId>
+ <artifactId>janino</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.pentaho</groupId>
+ <artifactId>pentaho-aggdesigner-algorithm</artifactId>
+ </exclusion>
Review comment:
These dependencies added by https://github.com/apache/hive/commit/52a4ab843503567a18219247eeaec840414a3376
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] jaceklaskowski commented on a change in pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
jaceklaskowski commented on a change in pull request #30657:
URL: https://github.com/apache/spark/pull/30657#discussion_r559161731
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/test/TestHive.scala
##########
@@ -666,7 +666,7 @@ private[sql] class TestHiveSessionStateBuilder(
}
private[hive] object HiveTestJars {
- private val repository = SQLConf.ADDITIONAL_REMOTE_REPOSITORIES.defaultValueString.split(",").last
+ private val repository = SQLConf.ADDITIONAL_REMOTE_REPOSITORIES.defaultValueString.split(",")(0)
Review comment:
nit: `.head`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742431266
Oh, sorry. We can add it to exclusions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745274515
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132823/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745918127
Thank you so much, @wangyum !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742848079
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132571/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761298775
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756675525
**[Test build #133832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133832/testReport)** for PR 30657 at commit [`e0632ca`](https://github.com/apache/spark/commit/e0632cac19c403a31ba16490fa90e6742889d9a1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-757097427
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38450/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745264414
**[Test build #132823 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132823/testReport)** for PR 30657 at commit [`127cb2d`](https://github.com/apache/spark/commit/127cb2dd900144742e471cb32447d3c7d5d2f2fe).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `trait CheckAnalysis extends PredicateHelper with LookupCatalog `
* `case class UnresolvedView(`
* `case class NoopCommand(`
* `case class ShowTableExtended(`
* `case class DropView(`
* `case class RepairTable(child: LogicalPlan) extends Command `
* `case class AlterViewSetProperties(`
* `case class AlterViewUnsetProperties(`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742182319
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37127/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-757109462
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133861/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-740305969
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36989/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #30657:
URL: https://github.com/apache/spark/pull/30657#discussion_r539945362
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala
##########
@@ -100,11 +100,13 @@ package object client {
"org.apache.curator:*",
"org.pentaho:pentaho-aggdesigner-algorithm"))
- // Since HIVE-14496, Hive materialized view need calcite-core.
+ // Since HIVE-23980, calcite-core included in Hive package jar.
// For spark, only VersionsSuite currently creates a hive materialized view for testing.
- case object v2_3 extends HiveVersion("2.3.7",
- exclusions = Seq("org.apache.calcite:calcite-druid",
+ case object v2_3 extends HiveVersion("2.3.8",
+ exclusions = Seq("org.apache.calcite:calcite-core",
+ "org.apache.calcite:calcite-druid",
"org.apache.calcite.avatica:avatica",
+ "com.google.guava:guava",
Review comment:
Just curious. Why do we need to exclude it? `Hive 2.3.8` is supposed to shade it, isn't it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741385920
let me try to reproduce this as well locally.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756672893
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745758359
**[Test build #132849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132849/testReport)** for PR 30657 at commit [`e0632ca`](https://github.com/apache/spark/commit/e0632cac19c403a31ba16490fa90e6742889d9a1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class SparkPod(pod: Pod, container: Container) `
* `trait KubernetesFeatureConfigStep `
* `public class Distributions `
* `case class Decode(params: Seq[Expression], child: Expression) extends RuntimeReplaceable `
* `case class StringDecode(bin: Expression, charset: Expression)`
* `case class AlterTableRecoverPartitions(child: LogicalPlan) extends Command `
* `case class AlterViewAs(`
* `case class CacheTable(`
* `case class CacheTableAsSelect(`
* `case class SubqueryExec(name: String, child: SparkPlan, maxNumRows: Option[Int] = None)`
* `trait BaseCacheTableExec extends V2CommandExec `
* `case class CacheTableExec(`
* `case class CacheTableAsSelectExec(`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742898220
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132596/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742158223
**[Test build #132524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132524/testReport)** for PR 30657 at commit [`fc9f735`](https://github.com/apache/spark/commit/fc9f73538378033f148903d43dfff21c0fb81112).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742574760
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37175/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742731776
**[Test build #132560 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132560/testReport)** for PR 30657 at commit [`8a4e20e`](https://github.com/apache/spark/commit/8a4e20e7fa4b697e91a9b38e8c3cc93762018b11).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741497467
> for comparison, the binary distribution from Hive master branch doesn't include any calcite jars, but I'm seeing:
>
> ```
> calcite-core-1.10.0.jar
> calcite-druid-1.10.0.jar
> calcite-linq4j-1.10.0.jar
> ```
>
> in 2.3.8 binary distribution.
I only see `hive-druid-handler` pulls `calcite-druid` in dependency tree. Other `calcite-core`, `calcite-druid` are all from hive-exec.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742531846
**[Test build #132571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132571/testReport)** for PR 30657 at commit [`8a4e20e`](https://github.com/apache/spark/commit/8a4e20e7fa4b697e91a9b38e8c3cc93762018b11).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742879673
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37201/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on a change in pull request #30657:
URL: https://github.com/apache/spark/pull/30657#discussion_r553920965
##########
File path: pom.xml
##########
@@ -1844,6 +1855,22 @@
<groupId>org.apache.logging.log4j</groupId>
<artifactId>*</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>net.hydromatic</groupId>
+ <artifactId>eigenbase-properties</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.codehaus.janino</groupId>
+ <artifactId>commons-compiler</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.codehaus.janino</groupId>
+ <artifactId>janino</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.pentaho</groupId>
+ <artifactId>pentaho-aggdesigner-algorithm</artifactId>
+ </exclusion>
Review comment:
These dependencies added by https://github.com/apache/hive/commit/52a4ab843503567a18219247eeaec840414a3376
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742848079
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132571/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-740269042
**[Test build #132389 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132389/testReport)** for PR 30657 at commit [`fc9f735`](https://github.com/apache/spark/commit/fc9f73538378033f148903d43dfff21c0fb81112).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-744844134
> @wangyum do you still see any issue with Spark on Hive 2.3.8?
There are no other issues. Thank you @sunchao.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756802037
**[Test build #133840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133840/testReport)** for PR 30657 at commit [`fce63a1`](https://github.com/apache/spark/commit/fce63a1293bb44f19a1f849b793647ef24bbe4f7).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class SparkPod(pod: Pod, container: Container) `
* `trait KubernetesFeatureConfigStep `
* `public class Distributions `
* `case class Decode(params: Seq[Expression], child: Expression) extends RuntimeReplaceable `
* `case class StringDecode(bin: Expression, charset: Expression)`
* `case class AlterTableRecoverPartitions(child: LogicalPlan) extends Command `
* `case class AlterViewAs(`
* `case class CacheTable(`
* `case class CacheTableAsSelect(`
* `case class SubqueryExec(name: String, child: SparkPlan, maxNumRows: Option[Int] = None)`
* `trait BaseCacheTableExec extends V2CommandExec `
* `case class CacheTableExec(`
* `case class CacheTableAsSelectExec(`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756700567
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745181076
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756700564
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761298775
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742948995
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37212/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742731781
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132560/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761928444
retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742284885
**[Test build #132549 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132549/testReport)** for PR 30657 at commit [`34a4da2`](https://github.com/apache/spark/commit/34a4da281be90514a5d879fcf86ddf54a64b01ef).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742203497
**[Test build #132524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132524/testReport)** for PR 30657 at commit [`fc9f735`](https://github.com/apache/spark/commit/fc9f73538378033f148903d43dfff21c0fb81112).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756729755
**[Test build #133840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133840/testReport)** for PR 30657 at commit [`fce63a1`](https://github.com/apache/spark/commit/fce63a1293bb44f19a1f849b793647ef24bbe4f7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761920537
Artifacts have been published to Maven now - could you try this again?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742378704
**[Test build #132549 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132549/testReport)** for PR 30657 at commit [`34a4da2`](https://github.com/apache/spark/commit/34a4da281be90514a5d879fcf86ddf54a64b01ef).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742182319
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37127/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761293951
**[Test build #134133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134133/testReport)** for PR 30657 at commit [`bb418bb`](https://github.com/apache/spark/commit/bb418bbf3a76e2c5ff47e0e4fcea318e6b2eec69).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761949169
Great! Could you remove the `[WIP]` in the title and convert from `Draft` to `Normal` PR, @wangyum ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761962810
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38754/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742699753
yay! seems all tests passed (test failures are not related?)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742373180
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37153/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742398492
@sunchao @viirya Could we do not include `com.fasterxml.jackson`?
Hive 2.3.8 | Hive 2.3.7
--- | ---
 | 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742373180
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37153/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-740323358
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132389/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742379257
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132549/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-740323358
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132389/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742441754
**[Test build #132560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132560/testReport)** for PR 30657 at commit [`8a4e20e`](https://github.com/apache/spark/commit/8a4e20e7fa4b697e91a9b38e8c3cc93762018b11).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-744652858
@wangyum do you still see any issue with Spark on Hive 2.3.8?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742879688
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37201/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742930942
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37212/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741552409
I think we need something like https://issues.apache.org/jira/browse/HIVE-23593 and exclude calcite from `bin.xml`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745766966
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132849/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742470305
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37164/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742897943
**[Test build #132596 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132596/testReport)** for PR 30657 at commit [`8a4e20e`](https://github.com/apache/spark/commit/8a4e20e7fa4b697e91a9b38e8c3cc93762018b11).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761699361
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #30657: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on a change in pull request #30657:
URL: https://github.com/apache/spark/pull/30657#discussion_r559314335
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/test/TestHive.scala
##########
@@ -666,7 +666,7 @@ private[sql] class TestHiveSessionStateBuilder(
}
private[hive] object HiveTestJars {
- private val repository = SQLConf.ADDITIONAL_REMOTE_REPOSITORIES.defaultValueString.split(",").last
+ private val repository = SQLConf.ADDITIONAL_REMOTE_REPOSITORIES.defaultValueString.split(",")(0)
Review comment:
This is a revert.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742182302
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37127/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756675525
**[Test build #133832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133832/testReport)** for PR 30657 at commit [`e0632ca`](https://github.com/apache/spark/commit/e0632cac19c403a31ba16490fa90e6742889d9a1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761693124
**[Test build #134151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134151/testReport)** for PR 30657 at commit [`bb418bb`](https://github.com/apache/spark/commit/bb418bbf3a76e2c5ff47e0e4fcea318e6b2eec69).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742354181
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37153/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742574760
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37175/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761974204
**[Test build #134170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134170/testReport)** for PR 30657 at commit [`bb418bb`](https://github.com/apache/spark/commit/bb418bbf3a76e2c5ff47e0e4fcea318e6b2eec69).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761932948
**[Test build #134170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134170/testReport)** for PR 30657 at commit [`bb418bb`](https://github.com/apache/spark/commit/bb418bbf3a76e2c5ff47e0e4fcea318e6b2eec69).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741478922
@viirya yes exactly - it is from `calcite-core`, perhaps we need to find out which dependency pulls in `calcite-core` and exclude it from there.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745766966
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132849/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742879688
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37201/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745716241
**[Test build #132849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132849/testReport)** for PR 30657 at commit [`e0632ca`](https://github.com/apache/spark/commit/e0632cac19c403a31ba16490fa90e6742889d9a1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741380859
Hmm yes I used to see this failure in Hive UTs but I think it should have been resolved via https://github.com/apache/hive/pull/1356, since we also shade calcite in `hive-exec` fat jar.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761694337
**[Test build #134151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134151/testReport)** for PR 30657 at commit [`bb418bb`](https://github.com/apache/spark/commit/bb418bbf3a76e2c5ff47e0e4fcea318e6b2eec69).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
wangyum commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-740525189
```
[info] - 2.3: loadTable *** FAILED *** (180 milliseconds)
[info] java.lang.reflect.InvocationTargetException:
[info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[info] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info] at java.lang.reflect.Method.invoke(Method.java:498)
[info] at org.apache.spark.sql.hive.client.Shim_v2_1.loadTable(HiveShim.scala:1267)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$loadTable$1(HiveClientImpl.scala:880)
[info] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$withHiveState$1(HiveClientImpl.scala:289)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:222)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:221)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:271)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.loadTable(HiveClientImpl.scala:875)
[info] at org.apache.spark.sql.hive.client.VersionsSuite.$anonfun$new$22(VersionsSuite.scala:288)
[info] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[info] at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] at org.scalatest.Transformer.apply(Transformer.scala:20)
[info] at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:190)
[info] at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:176)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:188)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:200)
[info] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:200)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:182)
[info] at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:61)
[info] at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
[info] at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
[info] at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:61)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:233)
[info] at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
[info] at scala.collection.immutable.List.foreach(List.scala:392)
[info] at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info] at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
[info] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:233)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:232)
[info] at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1563)
[info] at org.scalatest.Suite.run(Suite.scala:1112)
[info] at org.scalatest.Suite.run$(Suite.scala:1094)
[info] at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1563)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:237)
[info] at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:237)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:236)
[info] at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:61)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
[info] Cause: java.lang.NoSuchMethodError: com.fasterxml.jackson.annotation.JsonFormat$Value.empty()Lcom/fasterxml/jackson/annotation/JsonFormat$Value;
[info] at com.fasterxml.jackson.databind.cfg.MapperConfig.<clinit>(MapperConfig.java:50)
[info] at com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:565)
[info] at com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:480)
[info] at org.apache.hadoop.hive.common.StatsSetupConst$ColumnStatsAccurate.<clinit>(StatsSetupConst.java:164)
[info] at org.apache.hadoop.hive.common.StatsSetupConst.parseStatsAcc(StatsSetupConst.java:297)
[info] at org.apache.hadoop.hive.common.StatsSetupConst.clearColumnStatsState(StatsSetupConst.java:261)
[info] at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:2032)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[info] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[info] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info] at java.lang.reflect.Method.invoke(Method.java:498)
[info] at org.apache.spark.sql.hive.client.Shim_v2_1.loadTable(HiveShim.scala:1267)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$loadTable$1(HiveClientImpl.scala:880)
[info] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$withHiveState$1(HiveClientImpl.scala:289)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:222)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:221)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:271)
[info] at org.apache.spark.sql.hive.client.HiveClientImpl.loadTable(HiveClientImpl.scala:875)
[info] at org.apache.spark.sql.hive.client.VersionsSuite.$anonfun$new$22(VersionsSuite.scala:288)
[info] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[info] at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] at org.scalatest.Transformer.apply(Transformer.scala:20)
[info] at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:190)
[info] at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:176)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:188)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:200)
[info] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:200)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:182)
[info] at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:61)
[info] at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
[info] at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
[info] at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:61)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:233)
[info] at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
[info] at scala.collection.immutable.List.foreach(List.scala:392)
[info] at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info] at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
[info] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:233)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:232)
[info] at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1563)
[info] at org.scalatest.Suite.run(Suite.scala:1112)
[info] at org.scalatest.Suite.run$(Suite.scala:1094)
[info] at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1563)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:237)
[info] at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:237)
[info] at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:236)
[info] at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:61)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
[info] at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] at java.lang.Thread.run(Thread.java:748)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761693124
**[Test build #134151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134151/testReport)** for PR 30657 at commit [`bb418bb`](https://github.com/apache/spark/commit/bb418bbf3a76e2c5ff47e0e4fcea318e6b2eec69).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742204022
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132524/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756776884
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38429/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756700567
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742831098
**[Test build #132571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132571/testReport)** for PR 30657 at commit [`8a4e20e`](https://github.com/apache/spark/commit/8a4e20e7fa4b697e91a9b38e8c3cc93762018b11).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742379257
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132549/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya edited a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-741476895
> @wangyum @viirya yes I was able to reproduce the same issue. I think it could be cause we have `RelCollationImpl` included in both `hive-exec` jar and `calcite-core` jar. Perhaps we should exclude the latter from the Hive binary distribution and classpath.
>
> I'll update the release vote email thread and start a new RC once this is fixed.
We have included `org.apache.calcite:*` in Hive shaded jar. The following error looks like there is other calcite jar in the classpath so it conflicts with the shaded calcite.
```
Caused by: java.lang.NoSuchMethodError: org.apache.calcite.rel.RelCollationImpl.<init>(Lorg/apache/hive/com/google/common/collect/ImmutableList;)V
at org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelCollation.<init>(HiveRelCollation.java:29)
at org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getCollationList(RelOptHiveTable.java:181)
at org.apache.calcite.rel.metadata.RelMdCollation.table(RelMdCollation.java:175)
```
Do we know where it comes from?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745750363
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37451/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-745743477
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37451/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-761292563
**[Test build #134133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134133/testReport)** for PR 30657 at commit [`bb418bb`](https://github.com/apache/spark/commit/bb418bbf3a76e2c5ff47e0e4fcea318e6b2eec69).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-756700564
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-742951770
**[Test build #132608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132608/testReport)** for PR 30657 at commit [`127cb2d`](https://github.com/apache/spark/commit/127cb2dd900144742e471cb32447d3c7d5d2f2fe).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30657: [WIP][SPARK-33696][SQL] Upgrade built-in Hive to 2.3.8
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30657:
URL: https://github.com/apache/spark/pull/30657#issuecomment-740269042
**[Test build #132389 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132389/testReport)** for PR 30657 at commit [`fc9f735`](https://github.com/apache/spark/commit/fc9f73538378033f148903d43dfff21c0fb81112).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org