You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ya...@apache.org on 2020/05/05 06:22:59 UTC
[spark] branch branch-3.0 updated: [SPARK-31030][DOCS][FOLLOWUP]
Replace HTML Table by Markdown Table
This is an automated email from the ASF dual-hosted git repository.
yamamuro pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new ccde0a1 [SPARK-31030][DOCS][FOLLOWUP] Replace HTML Table by Markdown Table
ccde0a1 is described below
commit ccde0a1ae2d880585cb554cc67f75ef972a78c67
Author: Dilip Biswal <dk...@gmail.com>
AuthorDate: Tue May 5 15:21:14 2020 +0900
[SPARK-31030][DOCS][FOLLOWUP] Replace HTML Table by Markdown Table
### What changes were proposed in this pull request?
This PR is to clean up the markdown file in remaining pages in sql reference. The first one was done by gatorsmile in [28415](https://github.com/apache/spark/pull/28415)
- Replace HTML table by MD table
- **sql-ref-ansi-compliance.md**
<img width="967" alt="Screen Shot 2020-05-01 at 4 36 35 PM" src="https://user-images.githubusercontent.com/14225158/80848981-1cbca080-8bca-11ea-8a5d-63174b31c800.png">
- **sql-ref-datatypes.md (Scala)**
<img width="967" alt="Screen Shot 2020-05-01 at 4 37 30 PM" src="https://user-images.githubusercontent.com/14225158/80849057-6a390d80-8bca-11ea-8866-ab08bab31432.png">
<img width="967" alt="Screen Shot 2020-05-01 at 4 39 18 PM" src="https://user-images.githubusercontent.com/14225158/80849061-6c9b6780-8bca-11ea-834c-eb93d3ab47ae.png">
- **sql-ref-datatypes.md (Java)**
<img width="967" alt="Screen Shot 2020-05-01 at 4 41 24 PM" src="https://user-images.githubusercontent.com/14225158/80849138-b3895d00-8bca-11ea-9d3b-555acad2086c.png">
<img width="967" alt="Screen Shot 2020-05-01 at 4 41 39 PM" src="https://user-images.githubusercontent.com/14225158/80849140-b6844d80-8bca-11ea-9ca9-1812b6a76c02.png">
- **sql-ref-datatypes.md (Python)**
<img width="967" alt="Screen Shot 2020-05-01 at 4 43 36 PM" src="https://user-images.githubusercontent.com/14225158/80849202-0400ba80-8bcb-11ea-96a5-7caecbf9dbbf.png">
<img width="967" alt="Screen Shot 2020-05-01 at 4 43 54 PM" src="https://user-images.githubusercontent.com/14225158/80849205-06fbab00-8bcb-11ea-8f00-6df52b151684.png">
- **sql-ref-datatypes.md (R)**
<img width="967" alt="Screen Shot 2020-05-01 at 4 45 16 PM" src="https://user-images.githubusercontent.com/14225158/80849288-5fcb4380-8bcb-11ea-8277-8589b5bb31bc.png">
<img width="967" alt="Screen Shot 2020-05-01 at 4 45 36 PM" src="https://user-images.githubusercontent.com/14225158/80849294-62c63400-8bcb-11ea-9438-b4f1193bc757.png">
- **sql-ref-datatypes.md (SQL)**
<img width="967" alt="Screen Shot 2020-05-01 at 4 48 02 PM" src="https://user-images.githubusercontent.com/14225158/80849336-986b1d00-8bcb-11ea-9736-5fb40496b681.png">
- **sql-ref-syntax-qry-select-tvf.md**
<img width="967" alt="Screen Shot 2020-05-01 at 4 49 32 PM" src="https://user-images.githubusercontent.com/14225158/80849399-d10af680-8bcb-11ea-8dc2-e3e750e21a59.png">
### Why are the changes needed?
Make the doc cleaner and easily editable by MD editors
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Manually using jekyll serve
Closes #28433 from dilipbiswal/sql-doc-table-cleanup.
Authored-by: Dilip Biswal <dk...@gmail.com>
Signed-off-by: Takeshi Yamamuro <ya...@apache.org>
(cherry picked from commit 5052d9557d964c07d0b8bd2e2b08ede7c6958118)
Signed-off-by: Takeshi Yamamuro <ya...@apache.org>
---
docs/sql-ref-ansi-compliance.md | 542 +++++++++++++-------------
docs/sql-ref-datatypes.md | 695 +++++-----------------------------
docs/sql-ref-datetime-pattern.md | 8 +-
docs/sql-ref-null-semantics.md | 131 ++-----
docs/sql-ref-syntax-qry-select-tvf.md | 33 +-
5 files changed, 388 insertions(+), 1021 deletions(-)
diff --git a/docs/sql-ref-ansi-compliance.md b/docs/sql-ref-ansi-compliance.md
index 6cf1653..93fb10b 100644
--- a/docs/sql-ref-ansi-compliance.md
+++ b/docs/sql-ref-ansi-compliance.md
@@ -27,35 +27,10 @@ The casting behaviours are defined as store assignment rules in the standard.
When `spark.sql.storeAssignmentPolicy` is set to `ANSI`, Spark SQL complies with the ANSI store assignment rules. This is a separate configuration because its default value is `ANSI`, while the configuration `spark.sql.ansi.enabled` is disabled by default.
-<table class="table">
-<tr><th>Property Name</th><th>Default</th><th>Meaning</th><th>Since Version</th></tr>
-<tr>
- <td><code>spark.sql.ansi.enabled</code></td>
- <td>false</td>
- <td>
- (Experimental) When true, Spark tries to conform to the ANSI SQL specification:
- 1. Spark will throw a runtime exception if an overflow occurs in any operation on integral/decimal field.
- 2. Spark will forbid using the reserved keywords of ANSI SQL as identifiers in the SQL parser.
- </td>
- <td>3.0.0</td>
-</tr>
-<tr>
- <td><code>spark.sql.storeAssignmentPolicy</code></td>
- <td>ANSI</td>
- <td>
- (Experimental) When inserting a value into a column with different data type, Spark will perform type coercion.
- Currently, we support 3 policies for the type coercion rules: ANSI, legacy and strict. With ANSI policy,
- Spark performs the type coercion as per ANSI SQL. In practice, the behavior is mostly the same as PostgreSQL.
- It disallows certain unreasonable type conversions such as converting string to int or double to boolean.
- With legacy policy, Spark allows the type coercion as long as it is a valid Cast, which is very loose.
- e.g. converting string to int or double to boolean is allowed.
- It is also the only behavior in Spark 2.x and it is compatible with Hive.
- With strict policy, Spark doesn't allow any possible precision loss or data truncation in type coercion,
- e.g. converting double to int or decimal to double is not allowed.
- </td>
- <td>3.0.0</td>
-</tr>
-</table>
+|Property Name|Default|Meaning|Since Version|
+|-------------|-------|-------|-------------|
+|`spark.sql.ansi.enabled`|false|(Experimental) When true, Spark tries to conform to the ANSI SQL specification: <br/> 1. Spark will throw a runtime exception if an overflow occurs in any operation on integral/decimal field. <br/> 2. Spark will forbid using the reserved keywords of ANSI SQL as identifiers in the SQL parser.|3.0.0|
+|`spark.sql.storeAssignmentPolicy`|ANSI|(Experimental) When inserting a value into a column with different data type, Spark will perform type coercion. Currently, we support 3 policies for the type coercion rules: ANSI, legacy and strict. With ANSI policy, Spark performs the type coercion as per ANSI SQL. In practice, the behavior is mostly the same as PostgreSQL. It disallows certain unreasonable type conversions such as converting string to int or double to boolean. With legacy poli [...]
The following subsections present behaviour changes in arithmetic operations, type conversions, and SQL parsing when the ANSI mode enabled.
@@ -152,259 +127,256 @@ By default `spark.sql.ansi.enabled` is false.
Below is a list of all the keywords in Spark SQL.
-<table class="table">
- <tr><th rowspan="2" style="vertical-align: middle;"><b>Keyword</b></th><th colspan="2"><b>Spark SQL</b></th><th rowspan="2" style="vertical-align: middle;"><b>SQL-2011</b></th></tr>
- <tr><th><b>ANSI mode</b></th><th><b>default mode</b></th></tr>
- <tr><td>ADD</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>AFTER</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>ALL</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ALTER</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ANALYZE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>AND</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ANTI</td><td>reserved</td><td>strict-non-reserved</td><td>non-reserved</td></tr>
- <tr><td>ANY</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ARCHIVE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>ARRAY</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>AS</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ASC</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>AT</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>AUTHORIZATION</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>BETWEEN</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>BOTH</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>BUCKET</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>BUCKETS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>BY</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CACHE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>CASCADE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CASE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CAST</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CHANGE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>CHECK</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CLEAR</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>CLUSTER</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>CLUSTERED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>CODEGEN</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>COLLATE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>COLLECTION</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>COLUMN</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>COLUMNS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>COMMENT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>COMMIT</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>COMPACT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>COMPACTIONS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>COMPUTE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>CONCATENATE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>CONSTRAINT</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>COST</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>CREATE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CROSS</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>CUBE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CURRENT</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CURRENT_DATE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CURRENT_TIME</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CURRENT_TIMESTAMP</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>CURRENT_USER</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>DATA</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DATABASE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DATABASES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DAY</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>DBPROPERTIES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DEFINED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DELETE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>DELIMITED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DESC</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DESCRIBE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>DFS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DIRECTORIES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DIRECTORY</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DISTINCT</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>DISTRIBUTE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DIV</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>DROP</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ELSE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>END</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ESCAPE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ESCAPED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>EXCEPT</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>EXCHANGE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>EXISTS</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>EXPLAIN</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>EXPORT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>EXTENDED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>EXTERNAL</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>EXTRACT</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>FALSE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>FETCH</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>FIELDS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>FILTER</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>FILEFORMAT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>FIRST</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>FOLLOWING</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>FOR</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>FOREIGN</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>FORMAT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>FORMATTED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>FROM</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>FULL</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>FUNCTION</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>FUNCTIONS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>GLOBAL</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>GRANT</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>GROUP</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>GROUPING</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>HAVING</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>HOUR</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>IF</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>IGNORE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>IMPORT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>IN</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>INDEX</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>INDEXES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>INNER</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>INPATH</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>INPUTFORMAT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>INSERT</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>INTERSECT</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>INTERVAL</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>INTO</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>IS</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ITEMS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>JOIN</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>KEYS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>LAST</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>LATERAL</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>LAZY</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>LEADING</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>LEFT</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>LIKE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>LIMIT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>LINES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>LIST</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>LOAD</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>LOCAL</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>LOCATION</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>LOCK</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>LOCKS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>LOGICAL</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>MACRO</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>MAP</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>MATCHED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>MERGE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>MINUS</td><td>reserved</td><td>strict-non-reserved</td><td>non-reserved</td></tr>
- <tr><td>MINUTE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>MONTH</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>MSCK</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>NAMESPACE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>NAMESPACES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>NATURAL</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>NO</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>NOT</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>NULL</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>NULLS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>OF</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ON</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>ONLY</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>OPTION</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>OPTIONS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>OR</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ORDER</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>OUT</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>OUTER</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>OUTPUTFORMAT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>OVER</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>OVERLAPS</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>OVERLAY</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>OVERWRITE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>PARTITION</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>PARTITIONED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>PARTITIONS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>PERCENT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>PIVOT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>PLACING</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>POSITION</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>PRECEDING</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>PRIMARY</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>PRINCIPALS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>PROPERTIES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>PURGE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>QUERY</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>RANGE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>RECORDREADER</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>RECORDWRITER</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>RECOVER</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>REDUCE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>REFERENCES</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>REFRESH</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>RENAME</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>REPAIR</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>REPLACE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>RESET</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>RESTRICT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>REVOKE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>RIGHT</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>RLIKE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>ROLE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>ROLES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>ROLLBACK</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ROLLUP</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ROW</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>ROWS</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>SCHEMA</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SECOND</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>SELECT</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>SEMI</td><td>reserved</td><td>strict-non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SEPARATED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SERDE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SERDEPROPERTIES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SESSION_USER</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>SET</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>SETS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SHOW</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SKEWED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SOME</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>SORT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SORTED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>START</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>STATISTICS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>STORED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>STRATIFY</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>STRUCT</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SUBSTR</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>SUBSTRING</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>TABLE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>TABLES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>TABLESAMPLE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>TBLPROPERTIES</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>TEMPORARY</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>TERMINATED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>THEN</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>TO</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>TOUCH</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>TRAILING</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>TRANSACTION</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>TRANSACTIONS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>TRANSFORM</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>TRIM</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>TRUE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>TRUNCATE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>UNARCHIVE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>UNBOUNDED</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>UNCACHE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>UNION</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>UNIQUE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>UNKNOWN</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>UNLOCK</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>UNSET</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>UPDATE</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>USE</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>USER</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>USING</td><td>reserved</td><td>strict-non-reserved</td><td>reserved</td></tr>
- <tr><td>VALUES</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>VIEW</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>VIEWS</td><td>non-reserved</td><td>non-reserved</td><td>non-reserved</td></tr>
- <tr><td>WHEN</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>WHERE</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>WINDOW</td><td>non-reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>WITH</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
- <tr><td>YEAR</td><td>reserved</td><td>non-reserved</td><td>reserved</td></tr>
-</table>
+|Keyword|Spark SQL<br/>ANSI Mode|Spark SQL<br/>Default Mode|SQL-2011|
+|-------|----------------------|-------------------------|--------|
+|ADD|non-reserved|non-reserved|non-reserved|
+|AFTER|non-reserved|non-reserved|non-reserved|
+|ALL|reserved|non-reserved|reserved|
+|ALTER|non-reserved|non-reserved|reserved|
+|ANALYZE|non-reserved|non-reserved|non-reserved|
+|AND|reserved|non-reserved|reserved|
+|ANTI|reserved|strict-non-reserved|non-reserved|
+|ANY|reserved|non-reserved|reserved|
+|ARCHIVE|non-reserved|non-reserved|non-reserved|
+|ARRAY|non-reserved|non-reserved|reserved|
+|AS|reserved|non-reserved|reserved|
+|ASC|non-reserved|non-reserved|non-reserved|
+|AT|non-reserved|non-reserved|reserved|
+|AUTHORIZATION|reserved|non-reserved|reserved|
+|BETWEEN|non-reserved|non-reserved|reserved|
+|BOTH|reserved|non-reserved|reserved|
+|BUCKET|non-reserved|non-reserved|non-reserved|
+|BUCKETS|non-reserved|non-reserved|non-reserved|
+|BY|non-reserved|non-reserved|reserved|
+|CACHE|non-reserved|non-reserved|non-reserved|
+|CASCADE|non-reserved|non-reserved|reserved|
+|CASE|reserved|non-reserved|reserved|
+|CAST|reserved|non-reserved|reserved|
+|CHANGE|non-reserved|non-reserved|non-reserved|
+|CHECK|reserved|non-reserved|reserved|
+|CLEAR|non-reserved|non-reserved|non-reserved|
+|CLUSTER|non-reserved|non-reserved|non-reserved|
+|CLUSTERED|non-reserved|non-reserved|non-reserved|
+|CODEGEN|non-reserved|non-reserved|non-reserved|
+|COLLATE|reserved|non-reserved|reserved|
+|COLLECTION|non-reserved|non-reserved|non-reserved|
+|COLUMN|reserved|non-reserved|reserved|
+|COLUMNS|non-reserved|non-reserved|non-reserved|
+|COMMENT|non-reserved|non-reserved|non-reserved|
+|COMMIT|non-reserved|non-reserved|reserved|
+|COMPACT|non-reserved|non-reserved|non-reserved|
+|COMPACTIONS|non-reserved|non-reserved|non-reserved|
+|COMPUTE|non-reserved|non-reserved|non-reserved|
+|CONCATENATE|non-reserved|non-reserved|non-reserved|
+|CONSTRAINT|reserved|non-reserved|reserved|
+|COST|non-reserved|non-reserved|non-reserved|
+|CREATE|reserved|non-reserved|reserved|
+|CROSS|reserved|strict-non-reserved|reserved|
+|CUBE|non-reserved|non-reserved|reserved|
+|CURRENT|non-reserved|non-reserved|reserved|
+|CURRENT_DATE|reserved|non-reserved|reserved|
+|CURRENT_TIME|reserved|non-reserved|reserved|
+|CURRENT_TIMESTAMP|reserved|non-reserved|reserved|
+|CURRENT_USER|reserved|non-reserved|reserved|
+|DATA|non-reserved|non-reserved|non-reserved|
+|DATABASE|non-reserved|non-reserved|non-reserved|
+|DATABASES|non-reserved|non-reserved|non-reserved|
+|DAY|reserved|non-reserved|reserved|
+|DBPROPERTIES|non-reserved|non-reserved|non-reserved|
+|DEFINED|non-reserved|non-reserved|non-reserved|
+|DELETE|non-reserved|non-reserved|reserved|
+|DELIMITED|non-reserved|non-reserved|non-reserved|
+|DESC|non-reserved|non-reserved|non-reserved|
+|DESCRIBE|non-reserved|non-reserved|reserved|
+|DFS|non-reserved|non-reserved|non-reserved|
+|DIRECTORIES|non-reserved|non-reserved|non-reserved|
+|DIRECTORY|non-reserved|non-reserved|non-reserved|
+|DISTINCT|reserved|non-reserved|reserved|
+|DISTRIBUTE|non-reserved|non-reserved|non-reserved|
+|DIV|non-reserved|non-reserved|non-reserved|
+|DROP|non-reserved|non-reserved|reserved|
+|ELSE|reserved|non-reserved|reserved|
+|END|reserved|non-reserved|reserved|
+|ESCAPE|reserved|non-reserved|reserved|
+|ESCAPED|non-reserved|non-reserved|non-reserved|
+|EXCEPT|reserved|strict-non-reserved|reserved|
+|EXCHANGE|non-reserved|non-reserved|non-reserved|
+|EXISTS|non-reserved|non-reserved|reserved|
+|EXPLAIN|non-reserved|non-reserved|non-reserved|
+|EXPORT|non-reserved|non-reserved|non-reserved|
+|EXTENDED|non-reserved|non-reserved|non-reserved|
+|EXTERNAL|non-reserved|non-reserved|reserved|
+|EXTRACT|non-reserved|non-reserved|reserved|
+|FALSE|reserved|non-reserved|reserved|
+|FETCH|reserved|non-reserved|reserved|
+|FIELDS|non-reserved|non-reserved|non-reserved|
+|FILTER|reserved|non-reserved|reserved|
+|FILEFORMAT|non-reserved|non-reserved|non-reserved|
+|FIRST|non-reserved|non-reserved|non-reserved|
+|FOLLOWING|non-reserved|non-reserved|non-reserved|
+|FOR|reserved|non-reserved|reserved|
+|FOREIGN|reserved|non-reserved|reserved|
+|FORMAT|non-reserved|non-reserved|non-reserved|
+|FORMATTED|non-reserved|non-reserved|non-reserved|
+|FROM|reserved|non-reserved|reserved|
+|FULL|reserved|strict-non-reserved|reserved|
+|FUNCTION|non-reserved|non-reserved|reserved|
+|FUNCTIONS|non-reserved|non-reserved|non-reserved|
+|GLOBAL|non-reserved|non-reserved|reserved|
+|GRANT|reserved|non-reserved|reserved|
+|GROUP|reserved|non-reserved|reserved|
+|GROUPING|non-reserved|non-reserved|reserved|
+|HAVING|reserved|non-reserved|reserved|
+|HOUR|reserved|non-reserved|reserved|
+|IF|non-reserved|non-reserved|reserved|
+|IGNORE|non-reserved|non-reserved|non-reserved|
+|IMPORT|non-reserved|non-reserved|non-reserved|
+|IN|reserved|non-reserved|reserved|
+|INDEX|non-reserved|non-reserved|non-reserved|
+|INDEXES|non-reserved|non-reserved|non-reserved|
+|INNER|reserved|strict-non-reserved|reserved|
+|INPATH|non-reserved|non-reserved|non-reserved|
+|INPUTFORMAT|non-reserved|non-reserved|non-reserved|
+|INSERT|non-reserved|non-reserved|reserved|
+|INTERSECT|reserved|strict-non-reserved|reserved|
+|INTERVAL|non-reserved|non-reserved|reserved|
+|INTO|reserved|non-reserved|reserved|
+|IS|reserved|non-reserved|reserved|
+|ITEMS|non-reserved|non-reserved|non-reserved|
+|JOIN|reserved|strict-non-reserved|reserved|
+|KEYS|non-reserved|non-reserved|non-reserved|
+|LAST|non-reserved|non-reserved|non-reserved|
+|LATERAL|non-reserved|non-reserved|reserved|
+|LAZY|non-reserved|non-reserved|non-reserved|
+|LEADING|reserved|non-reserved|reserved|
+|LEFT|reserved|strict-non-reserved|reserved|
+|LIKE|non-reserved|non-reserved|reserved|
+|LIMIT|non-reserved|non-reserved|non-reserved|
+|LINES|non-reserved|non-reserved|non-reserved|
+|LIST|non-reserved|non-reserved|non-reserved|
+|LOAD|non-reserved|non-reserved|non-reserved|
+|LOCAL|non-reserved|non-reserved|reserved|
+|LOCATION|non-reserved|non-reserved|non-reserved|
+|LOCK|non-reserved|non-reserved|non-reserved|
+|LOCKS|non-reserved|non-reserved|non-reserved|
+|LOGICAL|non-reserved|non-reserved|non-reserved|
+|MACRO|non-reserved|non-reserved|non-reserved|
+|MAP|non-reserved|non-reserved|non-reserved|
+|MATCHED|non-reserved|non-reserved|non-reserved|
+|MERGE|non-reserved|non-reserved|non-reserved|
+|MINUS|reserved|strict-non-reserved|non-reserved|
+|MINUTE|reserved|non-reserved|reserved|
+|MONTH|reserved|non-reserved|reserved|
+|MSCK|non-reserved|non-reserved|non-reserved|
+|NAMESPACE|non-reserved|non-reserved|non-reserved|
+|NAMESPACES|non-reserved|non-reserved|non-reserved|
+|NATURAL|reserved|strict-non-reserved|reserved|
+|NO|non-reserved|non-reserved|reserved|
+|NOT|reserved|non-reserved|reserved|
+|NULL|reserved|non-reserved|reserved|
+|NULLS|non-reserved|non-reserved|non-reserved|
+|OF|non-reserved|non-reserved|reserved|
+|ON|reserved|strict-non-reserved|reserved|
+|ONLY|reserved|non-reserved|reserved|
+|OPTION|non-reserved|non-reserved|non-reserved|
+|OPTIONS|non-reserved|non-reserved|non-reserved|
+|OR|reserved|non-reserved|reserved|
+|ORDER|reserved|non-reserved|reserved|
+|OUT|non-reserved|non-reserved|reserved|
+|OUTER|reserved|non-reserved|reserved|
+|OUTPUTFORMAT|non-reserved|non-reserved|non-reserved|
+|OVER|non-reserved|non-reserved|non-reserved|
+|OVERLAPS|reserved|non-reserved|reserved|
+|OVERLAY|non-reserved|non-reserved|non-reserved|
+|OVERWRITE|non-reserved|non-reserved|non-reserved|
+|PARTITION|non-reserved|non-reserved|reserved|
+|PARTITIONED|non-reserved|non-reserved|non-reserved|
+|PARTITIONS|non-reserved|non-reserved|non-reserved|
+|PERCENT|non-reserved|non-reserved|non-reserved|
+|PIVOT|non-reserved|non-reserved|non-reserved|
+|PLACING|non-reserved|non-reserved|non-reserved|
+|POSITION|non-reserved|non-reserved|reserved|
+|PRECEDING|non-reserved|non-reserved|non-reserved|
+|PRIMARY|reserved|non-reserved|reserved|
+|PRINCIPALS|non-reserved|non-reserved|non-reserved|
+|PROPERTIES|non-reserved|non-reserved|non-reserved|
+|PURGE|non-reserved|non-reserved|non-reserved|
+|QUERY|non-reserved|non-reserved|non-reserved|
+|RECORDREADER|non-reserved|non-reserved|non-reserved|
+|RECORDWRITER|non-reserved|non-reserved|non-reserved|
+|RECOVER|non-reserved|non-reserved|non-reserved|
+|REDUCE|non-reserved|non-reserved|non-reserved|
+|REFERENCES|reserved|non-reserved|reserved|
+|REFRESH|non-reserved|non-reserved|non-reserved|
+|RENAME|non-reserved|non-reserved|non-reserved|
+|REPAIR|non-reserved|non-reserved|non-reserved|
+|REPLACE|non-reserved|non-reserved|non-reserved|
+|RESET|non-reserved|non-reserved|non-reserved|
+|RESTRICT|non-reserved|non-reserved|non-reserved|
+|REVOKE|non-reserved|non-reserved|reserved|
+|RIGHT|reserved|strict-non-reserved|reserved|
+|RLIKE|non-reserved|non-reserved|non-reserved|
+|ROLE|non-reserved|non-reserved|non-reserved|
+|ROLES|non-reserved|non-reserved|non-reserved|
+|ROLLBACK|non-reserved|non-reserved|reserved|
+|ROLLUP|non-reserved|non-reserved|reserved|
+|ROW|non-reserved|non-reserved|reserved|
+|ROWS|non-reserved|non-reserved|reserved|
+|SCHEMA|non-reserved|non-reserved|non-reserved|
+|SECOND|reserved|non-reserved|reserved|
+|SELECT|reserved|non-reserved|reserved|
+|SEMI|reserved|strict-non-reserved|non-reserved|
+|SEPARATED|non-reserved|non-reserved|non-reserved|
+|SERDE|non-reserved|non-reserved|non-reserved|
+|SERDEPROPERTIES|non-reserved|non-reserved|non-reserved|
+|SESSION_USER|reserved|non-reserved|reserved|
+|SET|non-reserved|non-reserved|reserved|
+|SETS|non-reserved|non-reserved|non-reserved|
+|SHOW|non-reserved|non-reserved|non-reserved|
+|SKEWED|non-reserved|non-reserved|non-reserved|
+|SOME|reserved|non-reserved|reserved|
+|SORT|non-reserved|non-reserved|non-reserved|
+|SORTED|non-reserved|non-reserved|non-reserved|
+|START|non-reserved|non-reserved|reserved|
+|STATISTICS|non-reserved|non-reserved|non-reserved|
+|STORED|non-reserved|non-reserved|non-reserved|
+|STRATIFY|non-reserved|non-reserved|non-reserved|
+|STRUCT|non-reserved|non-reserved|non-reserved|
+|SUBSTR|non-reserved|non-reserved|non-reserved|
+|SUBSTRING|non-reserved|non-reserved|non-reserved|
+|TABLE|reserved|non-reserved|reserved|
+|TABLES|non-reserved|non-reserved|non-reserved|
+|TABLESAMPLE|non-reserved|non-reserved|reserved|
+|TBLPROPERTIES|non-reserved|non-reserved|non-reserved|
+|TEMPORARY|non-reserved|non-reserved|non-reserved|
+|TERMINATED|non-reserved|non-reserved|non-reserved|
+|THEN|reserved|non-reserved|reserved|
+|TO|reserved|non-reserved|reserved|
+|TOUCH|non-reserved|non-reserved|non-reserved|
+|TRAILING|reserved|non-reserved|reserved|
+|TRANSACTION|non-reserved|non-reserved|non-reserved|
+|TRANSACTIONS|non-reserved|non-reserved|non-reserved|
+|TRANSFORM|non-reserved|non-reserved|non-reserved|
+|TRIM|non-reserved|non-reserved|non-reserved|
+|TRUE|non-reserved|non-reserved|reserved|
+|TRUNCATE|non-reserved|non-reserved|reserved|
+|UNARCHIVE|non-reserved|non-reserved|non-reserved|
+|UNBOUNDED|non-reserved|non-reserved|non-reserved|
+|UNCACHE|non-reserved|non-reserved|non-reserved|
+|UNION|reserved|strict-non-reserved|reserved|
+|UNIQUE|reserved|non-reserved|reserved|
+|UNKNOWN|reserved|non-reserved|reserved|
+|UNLOCK|non-reserved|non-reserved|non-reserved|
+|UNSET|non-reserved|non-reserved|non-reserved|
+|UPDATE|non-reserved|non-reserved|reserved|
+|USE|non-reserved|non-reserved|non-reserved|
+|USER|reserved|non-reserved|reserved|
+|USING|reserved|strict-non-reserved|reserved|
+|VALUES|non-reserved|non-reserved|reserved|
+|VIEW|non-reserved|non-reserved|non-reserved|
+|VIEWS|non-reserved|non-reserved|non-reserved|
+|WHEN|reserved|non-reserved|reserved|
+|WHERE|reserved|non-reserved|reserved|
+|WINDOW|non-reserved|non-reserved|reserved|
+|WITH|reserved|non-reserved|reserved|
+|YEAR|reserved|non-reserved|reserved|
diff --git a/docs/sql-ref-datatypes.md b/docs/sql-ref-datatypes.md
index 0d49f6f..3f6b6b5 100644
--- a/docs/sql-ref-datatypes.md
+++ b/docs/sql-ref-datatypes.md
@@ -71,130 +71,24 @@ You can access them by doing
{% include_example data_types scala/org/apache/spark/examples/sql/SparkSQLExample.scala %}
-<table class="table">
-<tr>
- <th style="width:20%">Data type</th>
- <th style="width:40%">Value type in Scala</th>
- <th>API to access or create a data type</th></tr>
-<tr>
- <td> <b>ByteType</b> </td>
- <td> Byte </td>
- <td>
- ByteType
- </td>
-</tr>
-<tr>
- <td> <b>ShortType</b> </td>
- <td> Short </td>
- <td>
- ShortType
- </td>
-</tr>
-<tr>
- <td> <b>IntegerType</b> </td>
- <td> Int </td>
- <td>
- IntegerType
- </td>
-</tr>
-<tr>
- <td> <b>LongType</b> </td>
- <td> Long </td>
- <td>
- LongType
- </td>
-</tr>
-<tr>
- <td> <b>FloatType</b> </td>
- <td> Float </td>
- <td>
- FloatType
- </td>
-</tr>
-<tr>
- <td> <b>DoubleType</b> </td>
- <td> Double </td>
- <td>
- DoubleType
- </td>
-</tr>
-<tr>
- <td> <b>DecimalType</b> </td>
- <td> java.math.BigDecimal </td>
- <td>
- DecimalType
- </td>
-</tr>
-<tr>
- <td> <b>StringType</b> </td>
- <td> String </td>
- <td>
- StringType
- </td>
-</tr>
-<tr>
- <td> <b>BinaryType</b> </td>
- <td> Array[Byte] </td>
- <td>
- BinaryType
- </td>
-</tr>
-<tr>
- <td> <b>BooleanType</b> </td>
- <td> Boolean </td>
- <td>
- BooleanType
- </td>
-</tr>
-<tr>
- <td> <b>TimestampType</b> </td>
- <td> java.sql.Timestamp </td>
- <td>
- TimestampType
- </td>
-</tr>
-<tr>
- <td> <b>DateType</b> </td>
- <td> java.sql.Date </td>
- <td>
- DateType
- </td>
-</tr>
-<tr>
- <td> <b>ArrayType</b> </td>
- <td> scala.collection.Seq </td>
- <td>
- ArrayType(<i>elementType</i>, [<i>containsNull</i>])<br />
- <b>Note:</b> The default value of <i>containsNull</i> is <i>true</i>.
- </td>
-</tr>
-<tr>
- <td> <b>MapType</b> </td>
- <td> scala.collection.Map </td>
- <td>
- MapType(<i>keyType</i>, <i>valueType</i>, [<i>valueContainsNull</i>])<br />
- <b>Note:</b> The default value of <i>valueContainsNull</i> is <i>true</i>.
- </td>
-</tr>
-<tr>
- <td> <b>StructType</b> </td>
- <td> org.apache.spark.sql.Row </td>
- <td>
- StructType(<i>fields</i>)<br />
- <b>Note:</b> <i>fields</i> is a Seq of StructFields. Also, two fields with the same
- name are not allowed.
- </td>
-</tr>
-<tr>
- <td> <b>StructField</b> </td>
- <td> The value type in Scala of the data type of this field
- (For example, Int for a StructField with the data type IntegerType) </td>
- <td>
- StructField(<i>name</i>, <i>dataType</i>, [<i>nullable</i>])<br />
- <b>Note:</b> The default value of <i>nullable</i> is <i>true</i>.
- </td>
-</tr>
-</table>
+|Data type|Value type in Scala|API to access or create a data type|
+|---------|-------------------|-----------------------------------|
+|**ByteType**|Byte|ByteType|
+|**ShortType**|Short|ShortType|
+|**IntegerType**|Int|IntegerType|
+|**LongType**|Long|LongType|
+|**FloatType**|Float|FloatType|
+|**DoubleType**|Double|DoubleType|
+|**DecimalType**|java.math.BigDecimal|DecimalType|
+|**StringType**|String|StringType|
+|**BinaryType**|Array[Byte]|BinaryType|
+|**BooleanType**|Boolean|BooleanType|
+|**TimestampType**|java.sql.Timestamp|TimestampType|
+|**DateType**|java.sql.Date|DateType|
+|**ArrayType**|scala.collection.Seq|ArrayType(*elementType*, [*containsNull]*)<br/>**Note:** The default value of *containsNull* is true.|
+|**MapType**|scala.collection.Map|MapType(*keyType*, *valueType*, [*valueContainsNull]*)<br/>**Note:** The default value of *valueContainsNull* is true.|
+|**StructType**|org.apache.spark.sql.Row|StructType(*fields*)<br/>**Note:** *fields* is a Seq of StructFields. Also, two fields with the same name are not allowed.|
+|**StructField**|The value type in Scala of the data type of this field(For example, Int for a StructField with the data type IntegerType)|StructField(*name*, *dataType*, [*nullable*])<br/>**Note:** The default value of *nullable* is true.|
</div>
@@ -205,132 +99,24 @@ All data types of Spark SQL are located in the package of
please use factory methods provided in
`org.apache.spark.sql.types.DataTypes`.
-<table class="table">
-<tr>
- <th style="width:20%">Data type</th>
- <th style="width:40%">Value type in Java</th>
- <th>API to access or create a data type</th></tr>
-<tr>
- <td> <b>ByteType</b> </td>
- <td> byte or Byte </td>
- <td>
- DataTypes.ByteType
- </td>
-</tr>
-<tr>
- <td> <b>ShortType</b> </td>
- <td> short or Short </td>
- <td>
- DataTypes.ShortType
- </td>
-</tr>
-<tr>
- <td> <b>IntegerType</b> </td>
- <td> int or Integer </td>
- <td>
- DataTypes.IntegerType
- </td>
-</tr>
-<tr>
- <td> <b>LongType</b> </td>
- <td> long or Long </td>
- <td>
- DataTypes.LongType
- </td>
-</tr>
-<tr>
- <td> <b>FloatType</b> </td>
- <td> float or Float </td>
- <td>
- DataTypes.FloatType
- </td>
-</tr>
-<tr>
- <td> <b>DoubleType</b> </td>
- <td> double or Double </td>
- <td>
- DataTypes.DoubleType
- </td>
-</tr>
-<tr>
- <td> <b>DecimalType</b> </td>
- <td> java.math.BigDecimal </td>
- <td>
- DataTypes.createDecimalType()<br />
- DataTypes.createDecimalType(<i>precision</i>, <i>scale</i>).
- </td>
-</tr>
-<tr>
- <td> <b>StringType</b> </td>
- <td> String </td>
- <td>
- DataTypes.StringType
- </td>
-</tr>
-<tr>
- <td> <b>BinaryType</b> </td>
- <td> byte[] </td>
- <td>
- DataTypes.BinaryType
- </td>
-</tr>
-<tr>
- <td> <b>BooleanType</b> </td>
- <td> boolean or Boolean </td>
- <td>
- DataTypes.BooleanType
- </td>
-</tr>
-<tr>
- <td> <b>TimestampType</b> </td>
- <td> java.sql.Timestamp </td>
- <td>
- DataTypes.TimestampType
- </td>
-</tr>
-<tr>
- <td> <b>DateType</b> </td>
- <td> java.sql.Date </td>
- <td>
- DataTypes.DateType
- </td>
-</tr>
-<tr>
- <td> <b>ArrayType</b> </td>
- <td> java.util.List </td>
- <td>
- DataTypes.createArrayType(<i>elementType</i>)<br />
- <b>Note:</b> The value of <i>containsNull</i> will be <i>true</i><br />
- DataTypes.createArrayType(<i>elementType</i>, <i>containsNull</i>).
- </td>
-</tr>
-<tr>
- <td> <b>MapType</b> </td>
- <td> java.util.Map </td>
- <td>
- DataTypes.createMapType(<i>keyType</i>, <i>valueType</i>)<br />
- <b>Note:</b> The value of <i>valueContainsNull</i> will be <i>true</i>.<br />
- DataTypes.createMapType(<i>keyType</i>, <i>valueType</i>, <i>valueContainsNull</i>)<br />
- </td>
-</tr>
-<tr>
- <td> <b>StructType</b> </td>
- <td> org.apache.spark.sql.Row </td>
- <td>
- DataTypes.createStructType(<i>fields</i>)<br />
- <b>Note:</b> <i>fields</i> is a List or an array of StructFields.
- Also, two fields with the same name are not allowed.
- </td>
-</tr>
-<tr>
- <td> <b>StructField</b> </td>
- <td> The value type in Java of the data type of this field
- (For example, int for a StructField with the data type IntegerType) </td>
- <td>
- DataTypes.createStructField(<i>name</i>, <i>dataType</i>, <i>nullable</i>)
- </td>
-</tr>
-</table>
+|Data type|Value type in Java|API to access or create a data type|
+|---------|------------------|-----------------------------------|
+|**ByteType**|byte or Byte|DataTypes.ByteType|
+|**ShortType**|short or Short|DataTypes.ShortType|
+|**IntegerType**|int or Integer|DataTypes.IntegerType|
+|**LongType**|long or Long|DataTypes.LongType|
+|**FloatType**|float or Float|DataTypes.FloatType|
+|**DoubleType**|double or Double|DataTypes.DoubleType|
+|**DecimalType**|java.math.BigDecimal|DataTypes.createDecimalType()<br/>DataTypes.createDecimalType(*precision*, *scale*).|
+|**StringType**|String|DataTypes.StringType|
+|**BinaryType**|byte[]|DataTypes.BinaryType|
+|**BooleanType**|boolean or Boolean|DataTypes.BooleanType|
+|**TimestampType**|java.sql.Timestamp|DataTypes.TimestampType|
+|**DateType**|java.sql.Date|DataTypes.DateType|
+|**ArrayType**|java.util.List|DataTypes.createArrayType(*elementType*)<br/>**Note:** The value of *containsNull* will be true.<br/>DataTypes.createArrayType(*elementType*, *containsNull*).|
+|**MapType**|java.util.Map|DataTypes.createMapType(*keyType*, *valueType*)<br/>**Note:** The value of *valueContainsNull* will be true.<br/>DataTypes.createMapType(*keyType*, *valueType*, *valueContainsNull*)|
+|**StructType**|org.apache.spark.sql.Row|DataTypes.createStructType(*fields*)<br/>**Note:** *fields* is a List or an array of StructFields.Also, two fields with the same name are not allowed.|
+|**StructField**|The value type in Java of the data type of this field (For example, int for a StructField with the data type IntegerType)|DataTypes.createStructField(*name*, *dataType*, *nullable*)|
</div>
@@ -342,370 +128,73 @@ You can access them by doing
from pyspark.sql.types import *
{% endhighlight %}
-<table class="table">
-<tr>
- <th style="width:20%">Data type</th>
- <th style="width:40%">Value type in Python</th>
- <th>API to access or create a data type</th></tr>
-<tr>
- <td> <b>ByteType</b> </td>
- <td>
- int or long <br />
- <b>Note:</b> Numbers will be converted to 1-byte signed integer numbers at runtime.
- Please make sure that numbers are within the range of -128 to 127.
- </td>
- <td>
- ByteType()
- </td>
-</tr>
-<tr>
- <td> <b>ShortType</b> </td>
- <td>
- int or long <br />
- <b>Note:</b> Numbers will be converted to 2-byte signed integer numbers at runtime.
- Please make sure that numbers are within the range of -32768 to 32767.
- </td>
- <td>
- ShortType()
- </td>
-</tr>
-<tr>
- <td> <b>IntegerType</b> </td>
- <td> int or long </td>
- <td>
- IntegerType()
- </td>
-</tr>
-<tr>
- <td> <b>LongType</b> </td>
- <td>
- long <br />
- <b>Note:</b> Numbers will be converted to 8-byte signed integer numbers at runtime.
- Please make sure that numbers are within the range of
- -9223372036854775808 to 9223372036854775807.
- Otherwise, please convert data to decimal.Decimal and use DecimalType.
- </td>
- <td>
- LongType()
- </td>
-</tr>
-<tr>
- <td> <b>FloatType</b> </td>
- <td>
- float <br />
- <b>Note:</b> Numbers will be converted to 4-byte single-precision floating
- point numbers at runtime.
- </td>
- <td>
- FloatType()
- </td>
-</tr>
-<tr>
- <td> <b>DoubleType</b> </td>
- <td> float </td>
- <td>
- DoubleType()
- </td>
-</tr>
-<tr>
- <td> <b>DecimalType</b> </td>
- <td> decimal.Decimal </td>
- <td>
- DecimalType()
- </td>
-</tr>
-<tr>
- <td> <b>StringType</b> </td>
- <td> string </td>
- <td>
- StringType()
- </td>
-</tr>
-<tr>
- <td> <b>BinaryType</b> </td>
- <td> bytearray </td>
- <td>
- BinaryType()
- </td>
-</tr>
-<tr>
- <td> <b>BooleanType</b> </td>
- <td> bool </td>
- <td>
- BooleanType()
- </td>
-</tr>
-<tr>
- <td> <b>TimestampType</b> </td>
- <td> datetime.datetime </td>
- <td>
- TimestampType()
- </td>
-</tr>
-<tr>
- <td> <b>DateType</b> </td>
- <td> datetime.date </td>
- <td>
- DateType()
- </td>
-</tr>
-<tr>
- <td> <b>ArrayType</b> </td>
- <td> list, tuple, or array </td>
- <td>
- ArrayType(<i>elementType</i>, [<i>containsNull</i>])<br />
- <b>Note:</b> The default value of <i>containsNull</i> is <i>True</i>.
- </td>
-</tr>
-<tr>
- <td> <b>MapType</b> </td>
- <td> dict </td>
- <td>
- MapType(<i>keyType</i>, <i>valueType</i>, [<i>valueContainsNull</i>])<br />
- <b>Note:</b> The default value of <i>valueContainsNull</i> is <i>True</i>.
- </td>
-</tr>
-<tr>
- <td> <b>StructType</b> </td>
- <td> list or tuple </td>
- <td>
- StructType(<i>fields</i>)<br />
- <b>Note:</b> <i>fields</i> is a Seq of StructFields. Also, two fields with the same
- name are not allowed.
- </td>
-</tr>
-<tr>
- <td> <b>StructField</b> </td>
- <td> The value type in Python of the data type of this field
- (For example, Int for a StructField with the data type IntegerType) </td>
- <td>
- StructField(<i>name</i>, <i>dataType</i>, [<i>nullable</i>])<br />
- <b>Note:</b> The default value of <i>nullable</i> is <i>True</i>.
- </td>
-</tr>
-</table>
+|Data type|Value type in Python|API to access or create a data type|
+|---------|--------------------|-----------------------------------|
+|**ByteType**|int or long<br/>**Note:** Numbers will be converted to 1-byte signed integer numbers at runtime. Please make sure that numbers are within the range of -128 to 127.|ByteType()|
+|**ShortType**|int or long<br/>**Note:** Numbers will be converted to 2-byte signed integer numbers at runtime. Please make sure that numbers are within the range of -32768 to 32767.|ShortType()|
+|**IntegerType**|int or long|IntegerType()|
+|**LongType**|long<br/>**Note:** Numbers will be converted to 8-byte signed integer numbers at runtime. Please make sure that numbers are within the range of -9223372036854775808 to 9223372036854775807.Otherwise, please convert data to decimal.Decimal and use DecimalType.|LongType()|
+|**FloatType**|float<br/>**Note:** Numbers will be converted to 4-byte single-precision floating point numbers at runtime.|FloatType()|
+|**DoubleType**|float|DoubleType()|
+|**DecimalType**|decimal.Decimal|DecimalType()|
+|**StringType**|string|StringType()|
+|**BinaryType**|bytearray|BinaryType()|
+|**BooleanType**|bool|BooleanType()|
+|**TimestampType**|datetime.datetime|TimestampType()|
+|**DateType**|datetime.date|DateType()|
+|**ArrayType**|list, tuple, or array|ArrayType(*elementType*, [*containsNull*])<br/>**Note:**The default value of *containsNull* is True.|
+|**MapType**|dict|MapType(*keyType*, *valueType*, [*valueContainsNull]*)<br/>**Note:**The default value of *valueContainsNull* is True.|
+|**StructType**|list or tuple|StructType(*fields*)<br/>**Note:** *fields* is a Seq of StructFields. Also, two fields with the same name are not allowed.|
+|**StructField**|The value type in Python of the data type of this field<br/>(For example, Int for a StructField with the data type IntegerType)|StructField(*name*, *dataType*, [*nullable*])<br/>**Note:** The default value of *nullable* is True.|
</div>
<div data-lang="r" markdown="1">
-<table class="table">
-<tr>
- <th style="width:20%">Data type</th>
- <th style="width:40%">Value type in R</th>
- <th>API to access or create a data type</th></tr>
-<tr>
- <td> <b>ByteType</b> </td>
- <td>
- integer <br />
- <b>Note:</b> Numbers will be converted to 1-byte signed integer numbers at runtime.
- Please make sure that numbers are within the range of -128 to 127.
- </td>
- <td>
- "byte"
- </td>
-</tr>
-<tr>
- <td> <b>ShortType</b> </td>
- <td>
- integer <br />
- <b>Note:</b> Numbers will be converted to 2-byte signed integer numbers at runtime.
- Please make sure that numbers are within the range of -32768 to 32767.
- </td>
- <td>
- "short"
- </td>
-</tr>
-<tr>
- <td> <b>IntegerType</b> </td>
- <td> integer </td>
- <td>
- "integer"
- </td>
-</tr>
-<tr>
- <td> <b>LongType</b> </td>
- <td>
- integer <br />
- <b>Note:</b> Numbers will be converted to 8-byte signed integer numbers at runtime.
- Please make sure that numbers are within the range of
- -9223372036854775808 to 9223372036854775807.
- Otherwise, please convert data to decimal.Decimal and use DecimalType.
- </td>
- <td>
- "long"
- </td>
-</tr>
-<tr>
- <td> <b>FloatType</b> </td>
- <td>
- numeric <br />
- <b>Note:</b> Numbers will be converted to 4-byte single-precision floating
- point numbers at runtime.
- </td>
- <td>
- "float"
- </td>
-</tr>
-<tr>
- <td> <b>DoubleType</b> </td>
- <td> numeric </td>
- <td>
- "double"
- </td>
-</tr>
-<tr>
- <td> <b>DecimalType</b> </td>
- <td> Not supported </td>
- <td>
- Not supported
- </td>
-</tr>
-<tr>
- <td> <b>StringType</b> </td>
- <td> character </td>
- <td>
- "string"
- </td>
-</tr>
-<tr>
- <td> <b>BinaryType</b> </td>
- <td> raw </td>
- <td>
- "binary"
- </td>
-</tr>
-<tr>
- <td> <b>BooleanType</b> </td>
- <td> logical </td>
- <td>
- "bool"
- </td>
-</tr>
-<tr>
- <td> <b>TimestampType</b> </td>
- <td> POSIXct </td>
- <td>
- "timestamp"
- </td>
-</tr>
-<tr>
- <td> <b>DateType</b> </td>
- <td> Date </td>
- <td>
- "date"
- </td>
-</tr>
-<tr>
- <td> <b>ArrayType</b> </td>
- <td> vector or list </td>
- <td>
- list(type="array", elementType=<i>elementType</i>, containsNull=[<i>containsNull</i>])<br />
- <b>Note:</b> The default value of <i>containsNull</i> is <i>TRUE</i>.
- </td>
-</tr>
-<tr>
- <td> <b>MapType</b> </td>
- <td> environment </td>
- <td>
- list(type="map", keyType=<i>keyType</i>, valueType=<i>valueType</i>, valueContainsNull=[<i>valueContainsNull</i>])<br />
- <b>Note:</b> The default value of <i>valueContainsNull</i> is <i>TRUE</i>.
- </td>
-</tr>
-<tr>
- <td> <b>StructType</b> </td>
- <td> named list</td>
- <td>
- list(type="struct", fields=<i>fields</i>)<br />
- <b>Note:</b> <i>fields</i> is a Seq of StructFields. Also, two fields with the same
- name are not allowed.
- </td>
-</tr>
-<tr>
- <td> <b>StructField</b> </td>
- <td> The value type in R of the data type of this field
- (For example, integer for a StructField with the data type IntegerType) </td>
- <td>
- list(name=<i>name</i>, type=<i>dataType</i>, nullable=[<i>nullable</i>])<br />
- <b>Note:</b> The default value of <i>nullable</i> is <i>TRUE</i>.
- </td>
-</tr>
-</table>
+|Data type|Value type in R|API to access or create a data type|
+|---------|---------------|-----------------------------------|
+|**ByteType**|integer <br/>**Note:** Numbers will be converted to 1-byte signed integer numbers at runtime. Please make sure that numbers are within the range of -128 to 127.|"byte"|
+|**ShortType**|integer <br/>**Note:** Numbers will be converted to 2-byte signed integer numbers at runtime. Please make sure that numbers are within the range of -32768 to 32767.|"short"|
+|**IntegerType**|integer|"integer"|
+|**LongType**|integer <br/>**Note:** Numbers will be converted to 8-byte signed integer numbers at runtime. Please make sure that numbers are within the range of -9223372036854775808 to 9223372036854775807. Otherwise, please convert data to decimal.Decimal and use DecimalType.|"long"|
+|**FloatType**|numeric <br/>**Note:** Numbers will be converted to 4-byte single-precision floating point numbers at runtime.|"float"|
+|**DoubleType**|numeric|"double"|
+|**DecimalType**|Not supported|Not supported|
+|**StringType**|character|"string"|
+|**BinaryType**|raw|"binary"|
+|**BooleanType**|logical|"bool"|
+|**TimestampType**|POSIXct|"timestamp"|
+|**DateType**|Date|"date"|
+|**ArrayType**|vector or list|list(type="array", elementType=*elementType*, containsNull=[*containsNull*])<br/>**Note:** The default value of *containsNull* is TRUE.|
+|**MapType**|environment|list(type="map", keyType=*keyType*, valueType=*valueType*, valueContainsNull=[*valueContainsNull*])<br/> **Note:** The default value of *valueContainsNull* is TRUE.|
+|**StructType**|named list|list(type="struct", fields=*fields*)<br/> **Note:** *fields* is a Seq of StructFields. Also, two fields with the same name are not allowed.|
+|**StructField**|The value type in R of the data type of this field (For example, integer for a StructField with the data type IntegerType)|list(name=*name*, type=*dataType*, nullable=[*nullable*])<br/> **Note:** The default value of *nullable* is TRUE.|
+
</div>
<div data-lang="SQL" markdown="1">
The following table shows the type names as well as aliases used in Spark SQL parser for each data type.
-<table class="table">
-<tr>
- <th style="width:40%">Data type</th>
- <th>SQL name</th></tr>
-<tr>
- <td> <b>BooleanType</b> </td>
- <td> BOOLEAN </td>
-</tr>
-<tr>
- <td> <b>ByteType</b> </td>
- <td> BYTE, TINYINT </td>
-</tr>
-<tr>
- <td> <b>ShortType</b> </td>
- <td> SHORT, SMALLINT </td>
-</tr>
-<tr>
- <td> <b>IntegerType</b> </td>
- <td> INT, INTEGER </td>
-</tr>
-<tr>
- <td> <b>LongType</b> </td>
- <td> LONG, BIGINT </td>
-</tr>
-<tr>
- <td> <b>FloatType</b> </td>
- <td> FLOAT, REAL </td>
-</tr>
-<tr>
- <td> <b>DoubleType</b> </td>
- <td> DOUBLE </td>
-</tr>
-<tr>
- <td> <b>DateType</b> </td>
- <td> DATE </td>
-</tr>
-<tr>
- <td> <b>TimestampType</b> </td>
- <td> TIMESTAMP </td>
-</tr>
-<tr>
- <td> <b>StringType</b> </td>
- <td> STRING </td>
-</tr>
-<tr>
- <td> <b>BinaryType</b> </td>
- <td> BINARY </td>
-</tr>
-<tr>
- <td> <b>DecimalType</b> </td>
- <td> DECIMAL, DEC, NUMERIC </td>
-</tr>
-<tr>
- <td> <b>CalendarIntervalType</b> </td>
- <td> INTERVAL </td>
-</tr>
-<tr>
- <td> <b>ArrayType</b> </td>
- <td> ARRAY<element_type> </td>
-</tr>
-<tr>
- <td> <b>StructType</b> </td>
- <td> STRUCT<field1_name: field1_type, field2_name: field2_type, ...> </td>
-</tr>
-<tr>
- <td> <b>MapType</b> </td>
- <td> MAP<key_type, value_type> </td>
-</tr>
-</table>
+|Data type|SQL name|
+|---------|--------|
+|**BooleanType**|BOOLEAN|
+|**ByteType**|BYTE, TINYINT|
+|**ShortType**|SHORT, SMALLINT|
+|**IntegerType**|INT, INTEGER|
+|**LongType**|LONG, BIGINT|
+|**FloatType**|FLOAT, REAL|
+|**DoubleType**|DOUBLE|
+|**DateType**|DATE|
+|**TimestampType**|TIMESTAMP|
+|**StringType**|STRING|
+|**BinaryType**|BINARY|
+|**DecimalType**|DECIMAL, DEC, NUMERIC|
+|**CalendarIntervalType**|INTERVAL|
+|**ArrayType**|ARRAY<element_type>|
+|**StructType**|STRUCT<field1_name: field1_type, field2_name: field2_type, ...>|
+|**MapType**|MAP<key_type, value_type>|
+
</div>
</div>
diff --git a/docs/sql-ref-datetime-pattern.md b/docs/sql-ref-datetime-pattern.md
index e606383..b65bb13 100644
--- a/docs/sql-ref-datetime-pattern.md
+++ b/docs/sql-ref-datetime-pattern.md
@@ -29,7 +29,7 @@ There are several common scenarios for datetime usage in Spark:
Spark uses pattern letters in the following table for date and timestamp parsing and formatting:
|Symbol|Meaning|Presentation|Examples|
-|--- |--- |--- |--- |
+|------|-------|------------|--------|
|**G**|era|text|AD; Anno Domini; A|
|**y**|year|year|2020; 20|
|**D**|day-of-year|number|189|
@@ -56,10 +56,10 @@ Spark uses pattern letters in the following table for date and timestamp parsing
|**X**|zone-offset 'Z' for zero|offset-X|Z; -08; -0830; -08:30; -083015; -08:30:15;|
|**x**|zone-offset|offset-x|+0000; -08; -0830; -08:30; -083015; -08:30:15;|
|**Z**|zone-offset|offset-Z|+0000; -0800; -08:00;|
-|**'**|escape for text|delimiter||
+|**'**|escape for text|delimiter| |
|**''**|single quote|literal|'|
-|**[**|optional section start|||
-|**]**|optional section end|||
+|**[**|optional section start| | |
+|**]**|optional section end| | |
The count of pattern letters determines the format.
diff --git a/docs/sql-ref-null-semantics.md b/docs/sql-ref-null-semantics.md
index d1f4f17..56b5cde 100644
--- a/docs/sql-ref-null-semantics.md
+++ b/docs/sql-ref-null-semantics.md
@@ -49,18 +49,18 @@ other `SQL` constructs.
</style>
The following illustrates the schema layout and data of a table named `person`. The data contains `NULL` values in
-the `age` column and this table will be used in various examples in the sections below.
+the `age` column and this table will be used in various examples in the sections below.<br/>
**<u>TABLE: person</u>**
-<table class="tsclass" border="1">
-<tr><th>Id</th><th>Name</th><th>Age</th></tr>
-<tr><td>100</td><td>Joe</td><td>30</td></tr>
-<tr><td>200</td><td>Marry</td><td>NULL</td></tr>
-<tr><td>300</td><td>Mike</td><td>18</td></tr>
-<tr><td>400</td><td>Fred</td><td>50</td></tr>
-<tr><td>500</td><td>Albert</td><td>NULL</td></tr>
-<tr><td>600</td><td>Michelle</td><td>30</td></tr>
-<tr><td>700</td><td>Dan</td><td>50</td></tr>
-</table>
+
+|Id |Name|Age|
+|---|----|---|
+|100|Joe|30|
+|200|Marry|NULL|
+|300|Mike|18|
+|400|Fred|50|
+|500|Albert|NULL|
+|600|Michelle|30|
+|700|Dan|50|
### Comparison Operators <a name="comp-operators"></a>
@@ -71,48 +71,11 @@ equal operator ('<=>'), which returns `False` when one of the operand is `NULL`
both the operands are `NULL`. The following table illustrates the behaviour of comparison operators when
one or both operands are `NULL`:
-<table class="tsclass" border="1">
- <tr>
- <th>Left Operand</th>
- <th>Right Operand</th>
- <th>></th>
- <th>>=</th>
- <th>=</th>
- <th><</th>
- <th><=</th>
- <th><=></th>
- </tr>
- <tr>
- <td>NULL</td>
- <td>Any value</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>False</td>
- </tr>
- <tr>
- <td>Any value</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>False</td>
- </tr>
- <tr>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>True</td>
- </tr>
-</table>
+|Left Operand|Right Operand|> |>= |= |< |<= |<=>|
+|------------|--------------|---|---|---|---|---|---|
+|NULL|Any value|NULL|NULL|NULL|NULL|NULL|False|
+|Any value|NULL|NULL|NULL|NULL|NULL|NULL|False|
+|NULL|NULL|NULL|NULL|NULL|NULL|NULL|True|
### Examples
@@ -157,55 +120,17 @@ as the arguments and return a `Boolean` value.
The following tables illustrate the behavior of logical operators when one or both operands are `NULL`.
-<table class="tsclass" border="1">
- <tr>
- <th>Left Operand</th>
- <th>Right Operand</th>
- <th>OR</th>
- <th>AND</th>
- </tr>
- <tr>
- <td>True</td>
- <td>NULL</td>
- <td>True</td>
- <td>NULL</td>
- </tr>
- <tr>
- <td>False</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>False</td>
- </tr>
- <tr>
- <td>NULL</td>
- <td>True</td>
- <td>True</td>
- <td>NULL</td>
- </tr>
- <tr>
- <td>NULL</td>
- <td>False</td>
- <td>NULL</td>
- <td>NULL</td>
- </tr>
- <tr>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- <td>NULL</td>
- </tr>
-</table>
-<br>
-<table class="tsclass" border="1">
- <tr>
- <th>operand</th>
- <th>NOT</th>
- </tr>
- <tr>
- <td>NULL</td>
- <td>NULL</td>
- </tr>
-</table>
+|Left Operand|Right Operand|OR |AND|
+|------------|-------------|---|---|
+|True|NULL|True|NULL|
+|False|NULL|NULL|False|
+|NULL|True|True|NULL|
+|NULL|False|NULL|NULL|
+|NULL|NULL|NULL|NULL|
+
+|operand|NOT|
+|-------|---|
+|NULL|NULL|
### Examples
@@ -335,7 +260,7 @@ the rules of how `NULL` values are handled by aggregate functions.
- `NULL` values are ignored from processing by all the aggregate functions.
- Only exception to this rule is COUNT(*) function.
- Some aggregate functions return `NULL` when all input values are `NULL` or the input data set
- is empty.<br> The list of these functions is:
+ is empty.<br/> The list of these functions is:
- MAX
- MIN
- SUM
diff --git a/docs/sql-ref-syntax-qry-select-tvf.md b/docs/sql-ref-syntax-qry-select-tvf.md
index ac8feca..1d9505a 100644
--- a/docs/sql-ref-syntax-qry-select-tvf.md
+++ b/docs/sql-ref-syntax-qry-select-tvf.md
@@ -40,7 +40,7 @@ function_name ( expression [ , ... ] ) [ table_alias ]
<dl>
<dt><code><em>table_alias</em></code></dt>
<dd>
- Specifies a temporary name with an optional column name list. <br><br>
+ Specifies a temporary name with an optional column name list. <br/><br/>
<b>Syntax:</b>
<code>
[ AS ] table_name [ ( column_name [ , ... ] ) ]
@@ -50,31 +50,12 @@ function_name ( expression [ , ... ] ) [ table_alias ]
### Supported Table-valued Functions
-<table class="table">
- <thead>
- <tr><th style="width:25%">Function</th><th>Argument Type(s)</th><th>Description</th></tr>
- </thead>
- <tr>
- <td><b> range </b>( <i>end</i> )</td>
- <td> Long </td>
- <td>Creates a table with a single <code>LongType</code> column named <code>id</code>, containing rows in a range from 0 to <code>end</code> (exclusive) with step value 1.</td>
- </tr>
- <tr>
- <td><b> range </b>( <i> start, end</i> )</td>
- <td> Long, Long </td>
- <td width="60%">Creates a table with a single <code>LongType</code> column named <code>id</code>, containing rows in a range from <code>start</code> to <code>end</code> (exclusive) with step value 1.</td>
- </tr>
- <tr>
- <td><b> range </b>( <i> start, end, step</i> )</td>
- <td> Long, Long, Long </td>
- <td width="60%">Creates a table with a single <code>LongType</code> column named <code>id</code>, containing rows in a range from <code>start</code> to <code>end</code> (exclusive) with <code>step</code> value.</td>
- </tr>
- <tr>
- <td><b> range </b>( <i> start, end, step, numPartitions</i> )</td>
- <td> Long, Long, Long, Int </td>
- <td width="60%">Creates a table with a single <code>LongType</code> column named <code>id</code>, containing rows in a range from <code>start</code> to <code>end</code> (exclusive) with <code>step</code> value, with partition number <code>numPartitions</code> specified. </td>
- </tr>
-</table>
+|Function|Argument Type(s)|Description|
+|--------|----------------|-----------|
+|**range** ( *end* )|Long|Creates a table with a single *LongType* column named *id*, <br/> containing rows in a range from 0 to *end* (exclusive) with step value 1.|
+|**range** ( *start, end* )|Long, Long|Creates a table with a single *LongType* column named *id*, <br/> containing rows in a range from *start* to *end* (exclusive) with step value 1.|
+|**range** ( *start, end, step* )|Long, Long, Long|Creates a table with a single *LongType* column named *id*, <br/> containing rows in a range from *start* to *end* (exclusive) with *step* value.|
+|**range** ( *start, end, step, numPartitions* )|Long, Long, Long, Int|Creates a table with a single *LongType* column named *id*, <br/> containing rows in a range from *start* to *end* (exclusive) with *step* value, with partition number *numPartitions* specified.|
### Examples
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org