You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by we...@apache.org on 2021/09/16 07:52:00 UTC
[spark] branch branch-3.2 updated: [SPARK-36775][DOCS] Add
documentation for ANSI store assignment rules
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new d20ed03 [SPARK-36775][DOCS] Add documentation for ANSI store assignment rules
d20ed03 is described below
commit d20ed030a8cb4b72aedf2ce6dc806f5d089748ef
Author: Gengliang Wang <ge...@apache.org>
AuthorDate: Thu Sep 16 15:50:40 2021 +0800
[SPARK-36775][DOCS] Add documentation for ANSI store assignment rules
### What changes were proposed in this pull request?
Add documentation for ANSI store assignment rules for
- the valid source/target type combinations
- runtime error will happen on numberic overflow
### Why are the changes needed?
Better docs
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Build docs and preview:
Closes #34014 from gengliangwang/addStoreAssignDoc.
Authored-by: Gengliang Wang <ge...@apache.org>
Signed-off-by: Wenchen Fan <we...@databricks.com>
(cherry picked from commit ff7705ad2ad5f4b9dfbeda83e93a0db676e1ffd9)
Signed-off-by: Wenchen Fan <we...@databricks.com>
---
docs/sql-ref-ansi-compliance.md | 29 +++++++++++++++++++++++++++--
1 file changed, 27 insertions(+), 2 deletions(-)
diff --git a/docs/sql-ref-ansi-compliance.md b/docs/sql-ref-ansi-compliance.md
index f0e1066..7eb8025 100644
--- a/docs/sql-ref-ansi-compliance.md
+++ b/docs/sql-ref-ansi-compliance.md
@@ -78,7 +78,7 @@ The `CAST` clause of Spark ANSI mode follows the syntax rules of section 6.13 "c
* MapType => String
* StructType => String
- The valid combinations of target data type and source data type in a `CAST` expression are given by the following table.
+ The valid combinations of source and target data type in a `CAST` expression are given by the following table.
“Y” indicates that the combination is syntactically valid without restriction and “N” indicates that the combination is not valid.
| Source\Target | Numeric | String | Date | Timestamp | Interval | Boolean | Binary | Array | Map | Struct |
@@ -156,7 +156,32 @@ SELECT * FROM t;
```
### Store assignment
-As mentioned at the beginning, when `spark.sql.storeAssignmentPolicy` is set to `ANSI`(which is the default value), Spark SQL complies with the ANSI store assignment rules. During table insertion, Spark will throw exception on numeric value overflow or the source value can't be stored as the target type.
+As mentioned at the beginning, when `spark.sql.storeAssignmentPolicy` is set to `ANSI`(which is the default value), Spark SQL complies with the ANSI store assignment rules on table insertions. The valid combinations of source and target data type in table insertions are given by the following table.
+
+| Source\Target | Numeric | String | Date | Timestamp | Interval | Boolean | Binary | Array | Map | Struct |
+|:-------------:|:-------:|:------:|:----:|:---------:|:--------:|:-------:|:------:|:-----:|:---:|:------:|
+| Numeric | Y | Y | N | N | N | N | N | N | N | N |
+| String | N | Y | N | N | N | N | N | N | N | N |
+| Date | N | Y | Y | Y | N | N | N | N | N | N |
+| Timestamp | N | Y | Y | Y | N | N | N | N | N | N |
+| Interval | N | Y | N | N | N* | N | N | N | N | N |
+| Boolean | N | Y | N | N | N | Y | N | N | N | N |
+| Binary | N | Y | N | N | N | N | Y | N | N | N |
+| Array | N | N | N | N | N | N | N | Y** | N | N |
+| Map | N | N | N | N | N | N | N | N | Y** | N |
+| Struct | N | N | N | N | N | N | N | N | N | Y** |
+
+\* Spark doesn't support interval type table column.
+
+\*\* For Array/Map/Struct types, the data type check rule applies recursively to its component elements.
+
+During table insertion, Spark will throw exception on numeric value overflow.
+```sql
+CREATE TABLE test(i INT);
+
+INSERT INTO test VALUES (2147483648L);
+java.lang.ArithmeticException: Casting 2147483648 to int causes overflow
+```
### Type coercion
#### Type Promotion and Precedence
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org