You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2021/09/07 02:40:59 UTC
[spark] branch branch-3.2 updated:
[SPARK-36153][SQL][DOCS][FOLLOWUP] Fix the description about the possible
values of `spark.sql.catalogImplementation` property
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new a41dc45 [SPARK-36153][SQL][DOCS][FOLLOWUP] Fix the description about the possible values of `spark.sql.catalogImplementation` property
a41dc45 is described below
commit a41dc4516ed0f8d596747ea65d0e2b028ee466b5
Author: Kousuke Saruta <sa...@oss.nttdata.com>
AuthorDate: Tue Sep 7 11:39:45 2021 +0900
[SPARK-36153][SQL][DOCS][FOLLOWUP] Fix the description about the possible values of `spark.sql.catalogImplementation` property
### What changes were proposed in this pull request?
This PR fixes the description about the possible values of `spark.sql.catalogImplementation` property.
It was added in SPARK-36153 (#33362) but the possible values are `hive` or `in-memory` rather than `true` or `false`.
### Why are the changes needed?
To fix wrong description.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
I just confirmed `in-memory` and `hive` are the valid values with SparkShell.
Closes #33923 from sarutak/fix-doc-about-catalogImplementation.
Authored-by: Kousuke Saruta <sa...@oss.nttdata.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
(cherry picked from commit a5fe5d368c56c7015458e1aa690e4eadde33224c)
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
docs/sql-ref-syntax-qry-select-transform.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/sql-ref-syntax-qry-select-transform.md b/docs/sql-ref-syntax-qry-select-transform.md
index 5a38e14..41c0928 100644
--- a/docs/sql-ref-syntax-qry-select-transform.md
+++ b/docs/sql-ref-syntax-qry-select-transform.md
@@ -26,10 +26,10 @@ to transform the inputs by running a user-specified command or script.
Spark's script transform supports two modes:
- 1. Hive support disabled: Spark script transform can run without `spark.sql.catalogImplementation=true`
- or `SparkSession.builder.enableHiveSupport()`. In this case, now Spark only uses the script transform with
+ 1. Hive support disabled: Spark script transform can run with `spark.sql.catalogImplementation=in-memory`
+ or without `SparkSession.builder.enableHiveSupport()`. In this case, now Spark only uses the script transform with
`ROW FORMAT DELIMITED` and treats all values passed to the script as strings.
- 2. Hive support enabled: When Spark is run with `spark.sql.catalogImplementation=true` or Spark SQL is started
+ 2. Hive support enabled: When Spark is run with `spark.sql.catalogImplementation=hive` or Spark SQL is started
with `SparkSession.builder.enableHiveSupport()`, Spark can use the script transform with both Hive SerDe and
`ROW FORMAT DELIMITED`.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org