You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by vi...@apache.org on 2020/02/10 18:46:21 UTC
[spark] branch branch-3.0 updated: [SPARK-27946][SQL][FOLLOW-UP]
Change doc and error message for SHOW CREATE TABLE
This is an automated email from the ASF dual-hosted git repository.
viirya pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new d5e4f2e [SPARK-27946][SQL][FOLLOW-UP] Change doc and error message for SHOW CREATE TABLE
d5e4f2e is described below
commit d5e4f2e262e78e4a22e6b881bb721c7a8c0a5823
Author: Liang-Chi Hsieh <vi...@gmail.com>
AuthorDate: Mon Feb 10 10:45:00 2020 -0800
[SPARK-27946][SQL][FOLLOW-UP] Change doc and error message for SHOW CREATE TABLE
### What changes were proposed in this pull request?
This is a follow-up for #24938 to tweak error message and migration doc.
### Why are the changes needed?
Making user know workaround if SHOW CREATE TABLE doesn't work for some Hive tables.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
Existing unit tests.
Closes #27505 from viirya/SPARK-27946-followup.
Authored-by: Liang-Chi Hsieh <vi...@gmail.com>
Signed-off-by: Liang-Chi Hsieh <li...@uber.com>
(cherry picked from commit acfdb46a60fc06dac0af55951492d74b7073f546)
Signed-off-by: Liang-Chi Hsieh <li...@uber.com>
---
docs/sql-migration-guide.md | 2 +-
.../scala/org/apache/spark/sql/execution/command/tables.scala | 8 ++++++--
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index be0fe32..26eb583 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -326,7 +326,7 @@ license: |
- Since Spark 3.0, `SHOW TBLPROPERTIES` will cause `AnalysisException` if the table does not exist. In Spark version 2.4 and earlier, this scenario caused `NoSuchTableException`. Also, `SHOW TBLPROPERTIES` on a temporary view will cause `AnalysisException`. In Spark version 2.4 and earlier, it returned an empty result.
- - Since Spark 3.0, `SHOW CREATE TABLE` will always return Spark DDL, even when the given table is a Hive serde table. For Hive DDL, please use `SHOW CREATE TABLE AS SERDE` command instead.
+ - Since Spark 3.0, `SHOW CREATE TABLE` will always return Spark DDL, even when the given table is a Hive serde table. For generating Hive DDL, please use `SHOW CREATE TABLE AS SERDE` command instead.
- Since Spark 3.0, we upgraded the built-in Hive from 1.2 to 2.3. This may need to set `spark.sql.hive.metastore.version` and `spark.sql.hive.metastore.jars` according to the version of the Hive metastore.
For example: set `spark.sql.hive.metastore.version` to `1.2.1` and `spark.sql.hive.metastore.jars` to `maven` if your Hive metastore version is 1.2.1.
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
index 468ca50..90dbdf5 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
@@ -1076,7 +1076,9 @@ case class ShowCreateTableCommand(table: TableIdentifier)
"Failed to execute SHOW CREATE TABLE against table " +
s"${tableMetadata.identifier}, which is created by Hive and uses the " +
"following unsupported feature(s)\n" +
- tableMetadata.unsupportedFeatures.map(" - " + _).mkString("\n")
+ tableMetadata.unsupportedFeatures.map(" - " + _).mkString("\n") + ". " +
+ s"Please use `SHOW CREATE TABLE ${tableMetadata.identifier} AS SERDE` " +
+ "to show Hive DDL instead."
)
}
@@ -1086,7 +1088,9 @@ case class ShowCreateTableCommand(table: TableIdentifier)
if ("true".equalsIgnoreCase(tableMetadata.properties.getOrElse("transactional", "false"))) {
throw new AnalysisException(
- "SHOW CREATE TABLE doesn't support transactional Hive table")
+ "SHOW CREATE TABLE doesn't support transactional Hive table. " +
+ s"Please use `SHOW CREATE TABLE ${tableMetadata.identifier} AS SERDE` " +
+ "to show Hive DDL instead.")
}
convertTableMetadata(tableMetadata)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org