You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ya...@apache.org on 2020/10/28 02:29:45 UTC
[spark] branch branch-3.0 updated: [SPARK-33264][SQL][DOCS] Add a
dedicated page for SQL-on-file in SQL documents
This is an automated email from the ASF dual-hosted git repository.
yamamuro pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new ba2a113 [SPARK-33264][SQL][DOCS] Add a dedicated page for SQL-on-file in SQL documents
ba2a113 is described below
commit ba2a11330038771d085f2ea9b50940ec3b8b97c5
Author: Takeshi Yamamuro <ya...@apache.org>
AuthorDate: Wed Oct 28 11:21:35 2020 +0900
[SPARK-33264][SQL][DOCS] Add a dedicated page for SQL-on-file in SQL documents
### What changes were proposed in this pull request?
This PR intends to add a dedicated page for SQL-on-file in SQL documents.
This comes from the comment: https://github.com/apache/spark/pull/30095/files#r508965149
### Why are the changes needed?
For better documentations.
### Does this PR introduce _any_ user-facing change?
<img width="544" alt="Screen Shot 2020-10-28 at 9 56 59" src="https://user-images.githubusercontent.com/692303/97378051-c1fbcb80-1904-11eb-86c0-a88c5269d41c.png">
### How was this patch tested?
N/A
Closes #30165 from maropu/DocForFile.
Authored-by: Takeshi Yamamuro <ya...@apache.org>
Signed-off-by: Takeshi Yamamuro <ya...@apache.org>
(cherry picked from commit c2bea045e3628081bca1ba752669a5bc009ebd00)
Signed-off-by: Takeshi Yamamuro <ya...@apache.org>
---
docs/_data/menu-sql.yaml | 2 +
docs/sql-ref-syntax-qry-select-file.md | 76 ++++++++++++++++++++++++++++++++++
docs/sql-ref-syntax-qry-select.md | 3 +-
docs/sql-ref-syntax-qry.md | 1 +
docs/sql-ref-syntax.md | 1 +
5 files changed, 82 insertions(+), 1 deletion(-)
diff --git a/docs/_data/menu-sql.yaml b/docs/_data/menu-sql.yaml
index 22fae0c..581fd1d 100644
--- a/docs/_data/menu-sql.yaml
+++ b/docs/_data/menu-sql.yaml
@@ -175,6 +175,8 @@
url: sql-ref-syntax-qry-select-hints.html
- text: Inline Table
url: sql-ref-syntax-qry-select-inline-table.html
+ - text: File
+ url: sql-ref-syntax-qry-select-file.html
- text: JOIN
url: sql-ref-syntax-qry-select-join.html
- text: LIKE Predicate
diff --git a/docs/sql-ref-syntax-qry-select-file.md b/docs/sql-ref-syntax-qry-select-file.md
new file mode 100644
index 0000000..c3dc406
--- /dev/null
+++ b/docs/sql-ref-syntax-qry-select-file.md
@@ -0,0 +1,76 @@
+---
+layout: global
+title: File
+displayTitle: File
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+### Description
+
+You can query a file with a specified format directly with SQL.
+
+### Syntax
+
+```sql
+file_format.`file_path`
+```
+
+### Parameters
+
+* **file_format**
+
+ Specifies a file format for a given file path, could be TEXTFILE, ORC, PARQUET, etc.
+
+* **file_path**
+
+ Specifies a file path with a given format.
+
+### Examples
+
+```sql
+-- PARQUET file
+SELECT * FROM parquet.`examples/src/main/resources/users.parquet`;
++------+--------------+----------------+
+| name|favorite_color|favorite_numbers|
++------+--------------+----------------+
+|Alyssa| null| [3, 9, 15, 20]|
+| Ben| red| []|
++------+--------------+----------------+
+
+-- ORC file
+SELECT * FROM orc.`examples/src/main/resources/users.orc`;
++------+--------------+----------------+
+| name|favorite_color|favorite_numbers|
++------+--------------+----------------+
+|Alyssa| null| [3, 9, 15, 20]|
+| Ben| red| []|
++------+--------------+----------------+
+
+-- JSON file
+SELECT * FROM json.`examples/src/main/resources/people.json`;
++----+-------+
+| age| name|
++----+-------+
+|null|Michael|
+| 30| Andy|
+| 19| Justin|
++----+-------+
+```
+
+### Related Statements
+
+* [SELECT](sql-ref-syntax-qry-select.html)
diff --git a/docs/sql-ref-syntax-qry-select.md b/docs/sql-ref-syntax-qry-select.md
index 655766d..bac7c2b 100644
--- a/docs/sql-ref-syntax-qry-select.md
+++ b/docs/sql-ref-syntax-qry-select.md
@@ -85,7 +85,7 @@ SELECT [ hints , ... ] [ ALL | DISTINCT ] { named_expression [ , ... ] }
* [Table-value function](sql-ref-syntax-qry-select-tvf.html)
* [Inline table](sql-ref-syntax-qry-select-inline-table.html)
* Subquery
- * [File](sql-data-sources-load-save-functions.html#run-sql-on-files-directly)
+ * [File](sql-ref-syntax-qry-select-file.html)
* **PIVOT**
@@ -165,6 +165,7 @@ SELECT [ hints , ... ] [ ALL | DISTINCT ] { named_expression [ , ... ] }
* [Common Table Expression](sql-ref-syntax-qry-select-cte.html)
* [Hints](sql-ref-syntax-qry-select-hints.html)
* [Inline Table](sql-ref-syntax-qry-select-inline-table.html)
+* [File](sql-ref-syntax-qry-select-file.html)
* [JOIN](sql-ref-syntax-qry-select-join.html)
* [LIKE Predicate](sql-ref-syntax-qry-select-like.html)
* [Set Operators](sql-ref-syntax-qry-select-setops.html)
diff --git a/docs/sql-ref-syntax-qry.md b/docs/sql-ref-syntax-qry.md
index d55ea43..6751b90 100644
--- a/docs/sql-ref-syntax-qry.md
+++ b/docs/sql-ref-syntax-qry.md
@@ -39,6 +39,7 @@ ability to generate logical and physical plan for a given query using
* [Common Table Expression](sql-ref-syntax-qry-select-cte.html)
* [Hints](sql-ref-syntax-qry-select-hints.html)
* [Inline Table](sql-ref-syntax-qry-select-inline-table.html)
+ * [File](sql-ref-syntax-qry-select-file.html)
* [JOIN](sql-ref-syntax-qry-select-join.html)
* [LIKE Predicate](sql-ref-syntax-qry-select-like.html)
* [Set Operators](sql-ref-syntax-qry-select-setops.html)
diff --git a/docs/sql-ref-syntax.md b/docs/sql-ref-syntax.md
index 4318cd6..148ce10 100644
--- a/docs/sql-ref-syntax.md
+++ b/docs/sql-ref-syntax.md
@@ -56,6 +56,7 @@ Spark SQL is Apache Spark's module for working with structured data. The SQL Syn
* [HAVING Clause](sql-ref-syntax-qry-select-having.html)
* [Hints](sql-ref-syntax-qry-select-hints.html)
* [Inline Table](sql-ref-syntax-qry-select-inline-table.html)
+ * [File](sql-ref-syntax-qry-select-file.html)
* [JOIN](sql-ref-syntax-qry-select-join.html)
* [LIKE Predicate](sql-ref-syntax-qry-select-like.html)
* [LIMIT Clause](sql-ref-syntax-qry-select-limit.html)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org