You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ma...@apache.org on 2014/10/09 02:16:55 UTC
git commit: [SQL][Doc] Keep Spark SQL README.md up to date
Repository: spark
Updated Branches:
refs/heads/master a42cc08d2 -> 00b779172
[SQL][Doc] Keep Spark SQL README.md up to date
marmbrus
Update README.md to be consistent with Spark 1.1
Author: Liquan Pei <li...@gmail.com>
Closes #2706 from Ishiihara/SparkSQL-readme and squashes the following commits:
33b9d4b [Liquan Pei] keep README.md up to date
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/00b77917
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/00b77917
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/00b77917
Branch: refs/heads/master
Commit: 00b7791720e50119a98084b2e8755e1b593ca55f
Parents: a42cc08
Author: Liquan Pei <li...@gmail.com>
Authored: Wed Oct 8 17:16:54 2014 -0700
Committer: Michael Armbrust <mi...@databricks.com>
Committed: Wed Oct 8 17:16:54 2014 -0700
----------------------------------------------------------------------
sql/README.md | 31 +++++++++++++++----------------
1 file changed, 15 insertions(+), 16 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/00b77917/sql/README.md
----------------------------------------------------------------------
diff --git a/sql/README.md b/sql/README.md
index 31f9152..c84534d 100644
--- a/sql/README.md
+++ b/sql/README.md
@@ -44,38 +44,37 @@ Type in expressions to have them evaluated.
Type :help for more information.
scala> val query = sql("SELECT * FROM (SELECT * FROM src) a")
-query: org.apache.spark.sql.ExecutedQuery =
-SELECT * FROM (SELECT * FROM src) a
-=== Query Plan ===
-Project [key#6:0.0,value#7:0.1]
- HiveTableScan [key#6,value#7], (MetastoreRelation default, src, None), None
+query: org.apache.spark.sql.SchemaRDD =
+== Query Plan ==
+== Physical Plan ==
+HiveTableScan [key#10,value#11], (MetastoreRelation default, src, None), None
```
Query results are RDDs and can be operated as such.
```
scala> query.collect()
-res8: Array[org.apache.spark.sql.execution.Row] = Array([238,val_238], [86,val_86], [311,val_311]...
+res2: Array[org.apache.spark.sql.Row] = Array([238,val_238], [86,val_86], [311,val_311], [27,val_27]...
```
You can also build further queries on top of these RDDs using the query DSL.
```
-scala> query.where('key === 100).toRdd.collect()
-res11: Array[org.apache.spark.sql.execution.Row] = Array([100,val_100], [100,val_100])
+scala> query.where('key === 100).collect()
+res3: Array[org.apache.spark.sql.Row] = Array([100,val_100], [100,val_100])
```
-From the console you can even write rules that transform query plans. For example, the above query has redundant project operators that aren't doing anything. This redundancy can be eliminated using the `transform` function that is available on all [`TreeNode`](http://databricks.github.io/catalyst/latest/api/#catalyst.trees.TreeNode) objects.
+From the console you can even write rules that transform query plans. For example, the above query has redundant project operators that aren't doing anything. This redundancy can be eliminated using the `transform` function that is available on all [`TreeNode`](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala) objects.
```scala
-scala> query.logicalPlan
-res1: catalyst.plans.logical.LogicalPlan =
-Project {key#0,value#1}
- Project {key#0,value#1}
+scala> query.queryExecution.analyzed
+res4: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan =
+Project [key#10,value#11]
+ Project [key#10,value#11]
MetastoreRelation default, src, None
-scala> query.logicalPlan transform {
+scala> query.queryExecution.analyzed transform {
| case Project(projectList, child) if projectList == child.output => child
| }
-res2: catalyst.plans.logical.LogicalPlan =
-Project {key#0,value#1}
+res5: res17: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan =
+Project [key#10,value#11]
MetastoreRelation default, src, None
```
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org