You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Zineng Yuan (JIRA)" <ji...@apache.org> on 2018/08/08 21:31:00 UTC
[jira] [Commented] (SPARK-25061) Spark SQL Thrift Server fails to
not pick up hiveconf passing parameter
[ https://issues.apache.org/jira/browse/SPARK-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573911#comment-16573911 ]
Zineng Yuan commented on SPARK-25061:
-------------------------------------
A further investigation starting from thrift server source points out that the problem comes from HiveClientImpl( Line 154):: after HiveConf is created (with hiveconf property), these confs got overwritten by hadooConf object (with hive-site.xml).
[https://github.com/apache/spark/blob/branch-2.2/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala]
[https://github.com/apache/spark/blob/branch-2.2/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala]
Don't understand why passing parameters got overwritten by hive-site.xml. Passing parameters should override others.
> Spark SQL Thrift Server fails to not pick up hiveconf passing parameter
> ------------------------------------------------------------------------
>
> Key: SPARK-25061
> URL: https://issues.apache.org/jira/browse/SPARK-25061
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Zineng Yuan
> Priority: Critical
>
> Spark thrift server should pick up passing parameter value and overwrites the conf value rom hive-site.xml. For example, the server should overwrite what exists in hive-site.xml.
> ./sbin/start-thriftserver.sh --master yarn-client ...
> --hiveconf "hive.server2.authentication.kerberos.principal=<client_principal>" ...
> <property>
> <name>hive.server2.authentication.kerberos.principal</name>
> <value>hive/_HOST@<server_domain></value>
> </property>
> However, the server takes what in hive-site.xml.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org