You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ma...@apache.org on 2015/10/21 23:12:27 UTC
spark git commit: [SPARK-11208][SQL] Filter out
'hive.metastore.rawstore.impl' from executionHive temporary config
Repository: spark
Updated Branches:
refs/heads/master 3afe448d3 -> a37cd8704
[SPARK-11208][SQL] Filter out 'hive.metastore.rawstore.impl' from executionHive temporary config
The executionHive assumed to be a standard meta store located in temporary directory as a derby db. But hive.metastore.rawstore.impl was not filtered out so any custom implementation of the metastore with other storage properties (not JDO) will persist that temporary functions. CassandraHiveMetaStore from DataStax Enterprise is one of examples.
Author: Artem Aliev <ar...@datastax.com>
Closes #9178 from artem-aliev/SPARK-11208.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a37cd870
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a37cd870
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a37cd870
Branch: refs/heads/master
Commit: a37cd870489d7ff85455bf9b57d9aa5329514058
Parents: 3afe448
Author: Artem Aliev <ar...@datastax.com>
Authored: Wed Oct 21 14:12:00 2015 -0700
Committer: Michael Armbrust <mi...@databricks.com>
Committed: Wed Oct 21 14:12:07 2015 -0700
----------------------------------------------------------------------
.../src/main/scala/org/apache/spark/sql/hive/HiveContext.scala | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/a37cd870/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
----------------------------------------------------------------------
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
index 61f6116..c7460d4 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
@@ -725,7 +725,8 @@ private[hive] object HiveContext {
// We have to mask all properties in hive-site.xml that relates to metastore data source
// as we used a local metastore here.
HiveConf.ConfVars.values().foreach { confvar =>
- if (confvar.varname.contains("datanucleus") || confvar.varname.contains("jdo")) {
+ if (confvar.varname.contains("datanucleus") || confvar.varname.contains("jdo")
+ || confvar.varname.contains("hive.metastore.rawstore.impl")) {
propMap.put(confvar.varname, confvar.getDefaultExpr())
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org