You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Park Chan Min (JIRA)" <ji...@apache.org> on 2018/04/05 02:05:00 UTC
[jira] [Created] (SPARK-23872) You can not connect to another
metastore uri using two Spark sessions
Park Chan Min created SPARK-23872:
-------------------------------------
Summary: You can not connect to another metastore uri using two Spark sessions
Key: SPARK-23872
URL: https://issues.apache.org/jira/browse/SPARK-23872
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.3.0
Environment: OS :CentOS release 6.8 (Final)
JAVA : build 1.8.0_101-b13
SPARK : 2.3.0
Reporter: Park Chan Min
In Spark 2.1.0, two sessions worked normally In 2.3.0, the first session metasore information is used when the second session is run.
*------------------- Code ----------------------*
val spark_1 = SparkSession.builder()
.enableHiveSupport()
.config("hive.metastore.uris", "thrift://HOST_A:9083")
.getOrCreate()
spark_1.sql("SELECT A_FIELD FROM TABLE_A").show()
SparkSession.clearActiveSession()
SparkSession.clearDefaultSession()
val spark_2 = SparkSession.builder()
.enableHiveSupport()
.config("hive.metastore.uris", "thrift://HOST_B:9083")
.getOrCreate()
spark_2.sql("SELECT B_FIELD FROM TABLE_B").show()
*------------------- spark 2.1.0 result ------------------*
INFO metastore: Trying to connect to metastore with URI thrift://{color:#d04437}*HOST_A*{color}:9083
......
INFO DAGScheduler: Job 3 finished: show at SparkHDFSTest.scala:20, took 0.807905 s
+-------+
|A_FIELD|
+-------+
| A |
+-------+
......
INFO metastore: Trying to connect to metastore with URI thrift://*{color:#d04437}HOST_B{color}*:9083
INFO DAGScheduler: Job 3 finished: show at SparkHDFSTest.scala:20, took 0.807905 s
+-------+
|B_FIELD|
+-------+
| B |
+-------+
......
*------------------- spark 2.3.0 result ------------------*
INFO metastore: Trying to connect to metastore with URI thrift://*{color:#d04437}HOST_A{color}*:9083
......
INFO DAGScheduler: Job 3 finished: show at SparkHDFSTest.scala:20, took 0.807905 s
+-------+
|A_FIELD|
+-------+
| A |
+-------+
......
INFO metastore: Trying to connect to metastore with URI thrift://*{color:#d04437}HOST_A{color}*:9083
INFO DAGScheduler: Job 3 finished: show at SparkHDFSTest.scala:20, took 0.807905 s
......
Exception in thread "main" org.apache.spark.sql.AnalysisException: Table or view not found: `default`.`TABLE_B`; line 1 pos 19;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org