You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by linbojin <gi...@git.apache.org> on 2016/08/15 06:58:41 UTC
[GitHub] spark pull request #14645: [MINOR] [DOC] Correct code snippet results in qui...
GitHub user linbojin opened a pull request:
https://github.com/apache/spark/pull/14645
[MINOR] [DOC] Correct code snippet results in quick start documentation
## What changes were proposed in this pull request?
As README.md file is updated over time. Some code snippet outputs are not correct based on new README.md file. For example:
```
scala> textFile.count()
res0: Long = 126
```
should be
```
scala> textFile.count()
res0: Long = 99
```
This pr is to correct these outputs so that new spark learners have a correct reference.
Also, fixed a samll bug, inside current documentation, the outputs of linesWithSpark.count() without and with cache are different (one is 15 and the other is 19)
```
scala> val linesWithSpark = textFile.filter(line => line.contains("Spark"))
linesWithSpark: org.apache.spark.rdd.RDD[String] = MapPartitionsRDD[2] at filter at <console>:27
scala> textFile.filter(line => line.contains("Spark")).count() // How many lines contain "Spark"?
res3: Long = 15
...
scala> linesWithSpark.cache()
res7: linesWithSpark.type = MapPartitionsRDD[2] at filter at <console>:27
scala> linesWithSpark.count()
res8: Long = 19
```
## How was this patch tested?
manual test: run `$ SKIP_API=1 jekyll serve --watch`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/linbojin/spark quick-start-documentation
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14645.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14645
----
commit f093e3a44a6447f619edd987bf30ee838899c578
Author: linbojin <li...@gmail.com>
Date: 2016-08-15T06:26:39Z
correct result numbers inside quick start docs based on new README.md file
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14645: [MINOR] [DOC] Correct code snippet results in qui...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/14645
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14645: [MINOR] [DOC] Correct code snippet results in qui...
Posted by linbojin <gi...@git.apache.org>.
Github user linbojin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14645#discussion_r74733368
--- Diff: docs/quick-start.md ---
@@ -40,7 +40,7 @@ RDDs have _[actions](programming-guide.html#actions)_, which return values, and
{% highlight scala %}
scala> textFile.count() // Number of items in this RDD
-res0: Long = 126
+res0: Long = 99
--- End diff --
Ok, i will just comment these. But for the bug mentioned above (with and without cache, the results are not matched), do i need to fix?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14645: [MINOR] [DOC] Correct code snippet results in quick star...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14645
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14645: [MINOR] [DOC] Correct code snippet results in quick star...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14645
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63780/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14645: [MINOR] [DOC] Correct code snippet results in qui...
Posted by linbojin <gi...@git.apache.org>.
Github user linbojin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14645#discussion_r74737989
--- Diff: docs/quick-start.md ---
@@ -40,7 +40,7 @@ RDDs have _[actions](programming-guide.html#actions)_, which return values, and
{% highlight scala %}
scala> textFile.count() // Number of items in this RDD
-res0: Long = 126
+res0: Long = 99
--- End diff --
OK, i changed into 15.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14645: [MINOR] [DOC] Correct code snippet results in quick star...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14645
**[Test build #63780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63780/consoleFull)** for PR 14645 at commit [`c9419aa`](https://github.com/apache/spark/commit/c9419aaaca104860110c072f9e257f7d86df020c).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14645: [MINOR] [DOC] Correct code snippet results in quick star...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14645
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14645: [MINOR] [DOC] Correct code snippet results in qui...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14645#discussion_r74737211
--- Diff: docs/quick-start.md ---
@@ -40,7 +40,7 @@ RDDs have _[actions](programming-guide.html#actions)_, which return values, and
{% highlight scala %}
scala> textFile.count() // Number of items in this RDD
-res0: Long = 126
+res0: Long = 99
--- End diff --
Oh I get it, it was already in the docs as 19 later. You're not saying you had a reproduction of a bug. OK yes of course it should be consistent at 19 (or whatever it is now).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14645: [MINOR] [DOC] Correct code snippet results in qui...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14645#discussion_r74735625
--- Diff: docs/quick-start.md ---
@@ -40,7 +40,7 @@ RDDs have _[actions](programming-guide.html#actions)_, which return values, and
{% highlight scala %}
scala> textFile.count() // Number of items in this RDD
-res0: Long = 126
+res0: Long = 99
--- End diff --
The result should be the same either way. I can't reproduce that and don't see it in the docs either? It's 15 before and you change it to 19.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14645: [MINOR] [DOC] Correct code snippet results in quick star...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14645
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14645: [MINOR] [DOC] Correct code snippet results in quick star...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14645
Merged to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14645: [MINOR] [DOC] Correct code snippet results in quick star...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14645
**[Test build #63780 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63780/consoleFull)** for PR 14645 at commit [`c9419aa`](https://github.com/apache/spark/commit/c9419aaaca104860110c072f9e257f7d86df020c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14645: [MINOR] [DOC] Correct code snippet results in qui...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14645#discussion_r74730995
--- Diff: docs/quick-start.md ---
@@ -40,7 +40,7 @@ RDDs have _[actions](programming-guide.html#actions)_, which return values, and
{% highlight scala %}
scala> textFile.count() // Number of items in this RDD
-res0: Long = 126
+res0: Long = 99
--- End diff --
Because this will change if `README.md` happens to change, you may wish to just comment that the number may be different, and so we don't have to update it again
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14645: [MINOR] [DOC] Correct code snippet results in qui...
Posted by linbojin <gi...@git.apache.org>.
Github user linbojin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14645#discussion_r74737117
--- Diff: docs/quick-start.md ---
@@ -40,7 +40,7 @@ RDDs have _[actions](programming-guide.html#actions)_, which return values, and
{% highlight scala %}
scala> textFile.count() // Number of items in this RDD
-res0: Long = 126
+res0: Long = 99
--- End diff --
From http://spark.apache.org/docs/latest/quick-start.html
without cache
and with cache
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14645: [MINOR] [DOC] Correct code snippet results in qui...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14645#discussion_r74738593
--- Diff: docs/quick-start.md ---
@@ -40,7 +40,7 @@ RDDs have _[actions](programming-guide.html#actions)_, which return values, and
{% highlight scala %}
scala> textFile.count() // Number of items in this RDD
-res0: Long = 126
+res0: Long = 99
--- End diff --
(15 is the result you currently get when running this right?) Yes the comment is good, and you're welcome to update the results to match what you get in master now. If that's all in order I'll merge
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org