You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by li...@apache.org on 2017/09/05 21:35:12 UTC
spark git commit: [MINOR][DOC] Update `Partition Discovery` section
to enumerate all available file sources
Repository: spark
Updated Branches:
refs/heads/master fd60d4fa6 -> 9e451bcf3
[MINOR][DOC] Update `Partition Discovery` section to enumerate all available file sources
## What changes were proposed in this pull request?
All built-in data sources support `Partition Discovery`. We had better update the document to give the users more benefit clearly.
**AFTER**
<img width="906" alt="1" src="https://user-images.githubusercontent.com/9700541/30083628-14278908-9244-11e7-98dc-9ad45fe233a9.png">
## How was this patch tested?
```
SKIP_API=1 jekyll serve --watch
```
Author: Dongjoon Hyun <do...@apache.org>
Closes #19139 from dongjoon-hyun/partitiondiscovery.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9e451bcf
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9e451bcf
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/9e451bcf
Branch: refs/heads/master
Commit: 9e451bcf36151bf401f72dcd66001b9ceb079738
Parents: fd60d4f
Author: Dongjoon Hyun <do...@apache.org>
Authored: Tue Sep 5 14:35:09 2017 -0700
Committer: gatorsmile <ga...@gmail.com>
Committed: Tue Sep 5 14:35:09 2017 -0700
----------------------------------------------------------------------
docs/sql-programming-guide.md | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/9e451bcf/docs/sql-programming-guide.md
----------------------------------------------------------------------
diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md
index ee231a9..032073b 100644
--- a/docs/sql-programming-guide.md
+++ b/docs/sql-programming-guide.md
@@ -733,8 +733,9 @@ SELECT * FROM parquetTable
Table partitioning is a common optimization approach used in systems like Hive. In a partitioned
table, data are usually stored in different directories, with partitioning column values encoded in
-the path of each partition directory. The Parquet data source is now able to discover and infer
-partitioning information automatically. For example, we can store all our previously used
+the path of each partition directory. All built-in file sources (including Text/CSV/JSON/ORC/Parquet)
+are able to discover and infer partitioning information automatically.
+For example, we can store all our previously used
population data into a partitioned table using the following directory structure, with two extra
columns, `gender` and `country` as partitioning columns:
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org