You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2020/02/11 17:16:00 UTC
[jira] [Resolved] (SPARK-30783) Hive 2.3 profile should exclude
hive-service-rpc
[ https://issues.apache.org/jira/browse/SPARK-30783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan resolved SPARK-30783.
---------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 27533
[https://github.com/apache/spark/pull/27533]
> Hive 2.3 profile should exclude hive-service-rpc
> ------------------------------------------------
>
> Key: SPARK-30783
> URL: https://issues.apache.org/jira/browse/SPARK-30783
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Yin Huai
> Assignee: Yin Huai
> Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: hive-service-rpc-2.3.6-classes, spark-hive-thriftserver_2.12-3.0.0-20200207.021914-364-classes
>
>
> hive-service-rpc 2.3.6 and spark sql's thrift server module have duplicate classes. Leaving hive-service-rpc 2.3.6 in the class path means that spark can pick up classes defined in hive instead of its thrift server module, which can cause hard to debug runtime errors due to class loading order and compilation errors for applications depend on spark.
>
> If you compare hive-service-rpc 2.3.6's jar ([https://search.maven.org/remotecontent?filepath=org/apache/hive/hive-service-rpc/2.3.6/hive-service-rpc-2.3.6.jar]) and spark thrift server's jar (e.g. [https://repository.apache.org/content/groups/snapshots/org/apache/spark/spark-hive-thriftserver_2.12/3.0.0-SNAPSHOT/spark-hive-thriftserver_2.12-3.0.0-20200207.021914-364.jar),] you will see that all of classes provided by hive-service-rpc-2.3.6.jar are covered by spark thrift server's jar. I am attaching the list of jar contents for your reference.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org