You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/01/05 09:47:06 UTC
[GitHub] [spark] dh20 commented on a change in pull request #31030: [SPARK-34006][SQL][DOC] Add descriptions about `spark.sql.hive.convertMetastoreOrc` in the SQL documents
dh20 commented on a change in pull request #31030:
URL: https://github.com/apache/spark/pull/31030#discussion_r551822676
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
##########
@@ -130,7 +130,10 @@ private[spark] object HiveUtils extends Logging {
val CONVERT_METASTORE_ORC = buildConf("spark.sql.hive.convertMetastoreOrc")
.doc("When set to true, the built-in ORC reader and writer are used to process " +
- "ORC tables created by using the HiveQL syntax, instead of Hive serde.")
+ "ORC tables created by using the HiveQL syntax, instead of Hive serde." +
+ "Enabling this parameter will cause insert overwrite when reading the table," +
+ "and the file format of this table is ORC This error will be thrown[], " +
Review comment:
@HyukjinKwon When I query an orc format hive table and overwrite the query result into this orc table, sparksql will throw this error -> [org.apache.spark.sql.AnalysisException: Cannot overwrite a path that is also being read from], when I set this parameter to false, Hql will execute normally
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org