You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/09 02:17:53 UTC
[GitHub] [spark] yhuai commented on a change in pull request #26619:
[SPARK-29981][BUILD] Add hive-1.2/2.3 profiles
yhuai commented on a change in pull request #26619: [SPARK-29981][BUILD] Add hive-1.2/2.3 profiles
URL: https://github.com/apache/spark/pull/26619#discussion_r376748954
##########
File path: pom.xml
##########
@@ -128,19 +128,19 @@
<zookeeper.version>3.4.14</zookeeper.version>
<curator.version>2.7.1</curator.version>
<okapi.version>0.4.2</okapi.version>
- <hive.group>org.spark-project.hive</hive.group>
- <hive.classifier></hive.classifier>
+ <hive.group>org.apache.hive</hive.group>
+ <hive.classifier>core</hive.classifier>
<!-- Version used in Maven Hive dependency -->
- <hive.version>1.2.1.spark2</hive.version>
+ <hive.version>2.3.6</hive.version>
<hive23.version>2.3.6</hive23.version>
<!-- Version used for internal directory structure -->
- <hive.version.short>1.2.1</hive.version.short>
+ <hive.version.short>2.3.5</hive.version.short>
Review comment:
@dongjoon-hyun i found hive storage api and hive common have the following common class files
```
org/apache/hadoop/hive/common/ValidReadTxnList.class
org/apache/hadoop/hive/common/ValidTxnList.class
org/apache/hadoop/hive/common/ValidTxnList$RangeResponse.class
```
For example, https://github.com/apache/hive/blob/rel/storage-release-2.6.0/storage-api/src/java/org/apache/hadoop/hive/common/ValidReadTxnList.java (pulled in by orc 1.5.8) and https://github.com/apache/hive/blob/rel/release-2.3.6/common/src/java/org/apache/hadoop/hive/common/ValidReadTxnList.java (from hive-common 2.3.6) both are in the classpath and they are different. I am worried that users may hit issues due to classloading order. I think it will be safe to still use nohive classifier for orc, which shades storage api.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org