You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Marcelo Vanzin (JIRA)" <ji...@apache.org> on 2017/03/03 02:06:46 UTC
[jira] [Commented] (SPARK-19804) HiveClientImpl does not work with
Hive 2.2.0 metastore
[ https://issues.apache.org/jira/browse/SPARK-19804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893579#comment-15893579 ]
Marcelo Vanzin commented on SPARK-19804:
----------------------------------------
For posterity, the error you get looks like this:
{noformat}
java.lang.ExceptionInInitializerError: null
at java.lang.Class.getConstructor0(Class.java:2892)
at java.lang.Class.getDeclaredConstructor(Class.java:2058)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1541)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:67)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3220)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3239)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3464)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:226)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:210)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:294)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:269)
at org.apache.spark.sql.hive.client.ClientWrapper.client(ClientWrapper.scala:272)
{noformat}
Which is rather cryptic; it's caused by one of the classes in the constructor being loaded by two different class loaders, so {{getDeclaredConstructor}} fails to find the right constructor and returns null.
> HiveClientImpl does not work with Hive 2.2.0 metastore
> ------------------------------------------------------
>
> Key: SPARK-19804
> URL: https://issues.apache.org/jira/browse/SPARK-19804
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Marcelo Vanzin
> Priority: Minor
>
> I know that Spark currently does not officially support Hive 2.2 (perhaps because it hasn't been released yet); but we have some 2.2 patches in CDH and the current code in the isolated client fails. The most probably culprit are changes added in HIVE-13149.
> The fix is simple, and here's the patch we applied in CDH:
> https://github.com/cloudera/spark/commit/954f060afe6ed469e85d656abd02790a79ec07a0
> Fixing that doesn't affect any existing Hive version support, but will make it easier to support 2.2 when it's out.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org