You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by rx...@apache.org on 2016/07/19 19:58:07 UTC
spark git commit: [SPARK-15705][SQL] Change the default value of
spark.sql.hive.convertMetastoreOrc to false.
Repository: spark
Updated Branches:
refs/heads/master 162d04a30 -> 2ae7b88a0
[SPARK-15705][SQL] Change the default value of spark.sql.hive.convertMetastoreOrc to false.
## What changes were proposed in this pull request?
In 2.0, we add a new logic to convert HiveTableScan on ORC tables to Spark's native code path. However, during this conversion, we drop the original metastore schema (https://issues.apache.org/jira/browse/SPARK-15705). Because of this regression, I am changing the default value of `spark.sql.hive.convertMetastoreOrc` to false.
Author: Yin Huai <yh...@databricks.com>
Closes #14267 from yhuai/SPARK-15705-changeDefaultValue.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2ae7b88a
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2ae7b88a
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/2ae7b88a
Branch: refs/heads/master
Commit: 2ae7b88a07140e012b6c60db3c4a2a8ca360c684
Parents: 162d04a
Author: Yin Huai <yh...@databricks.com>
Authored: Tue Jul 19 12:58:08 2016 -0700
Committer: Reynold Xin <rx...@databricks.com>
Committed: Tue Jul 19 12:58:08 2016 -0700
----------------------------------------------------------------------
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/2ae7b88a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
----------------------------------------------------------------------
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
index 9ed357c..bdec611 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
@@ -97,10 +97,11 @@ private[spark] object HiveUtils extends Logging {
.createWithDefault(false)
val CONVERT_METASTORE_ORC = SQLConfigBuilder("spark.sql.hive.convertMetastoreOrc")
+ .internal()
.doc("When set to false, Spark SQL will use the Hive SerDe for ORC tables instead of " +
"the built in support.")
.booleanConf
- .createWithDefault(true)
+ .createWithDefault(false)
val HIVE_METASTORE_SHARED_PREFIXES = SQLConfigBuilder("spark.sql.hive.metastore.sharedPrefixes")
.doc("A comma separated list of class prefixes that should be loaded using the classloader " +
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org