You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by gengliangwang <gi...@git.apache.org> on 2018/05/15 03:02:47 UTC
[GitHub] spark pull request #21326: [SPARK-24275][SQL] Revise doc comments in InputPa...
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/21326
[SPARK-24275][SQL] Revise doc comments in InputPartition
## What changes were proposed in this pull request?
In #21145, DataReaderFactory is renamed to InputPartition.
This PR is to revise wording in the comments to make it more clear.
## How was this patch tested?
None
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gengliangwang/spark revise_reader_comments
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21326.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21326
----
commit 76a116a93bd76b1f9fddec4f14515cf3a116a54f
Author: Gengliang Wang <ge...@...>
Date: 2018-05-15T02:47:03Z
revise the comments in read path
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by gengliangwang <gi...@git.apache.org>.
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/21326
@rdblue @cloud-fan
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21326: [SPARK-24275][SQL] Revise doc comments in InputPa...
Posted by rdblue <gi...@git.apache.org>.
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21326#discussion_r189730557
--- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/DataSourceReader.java ---
@@ -59,21 +59,21 @@
* Returns the actual schema of this data source reader, which may be different from the physical
* schema of the underlying storage, as column pruning or other optimizations may happen.
*
- * If this method fails (by throwing an exception), the action would fail and no Spark job was
+ * If this method fails (by throwing an exception), the action will fail and no Spark job will be
* submitted.
*/
StructType readSchema();
/**
- * Returns a list of read tasks. Each task is responsible for creating a data reader to
- * output data for one RDD partition. That means the number of tasks returned here is same as
- * the number of RDD partitions this scan outputs.
+ * Returns a list of {@link InputPartition}s. Each {@link InputPartition} is responsible for
--- End diff --
Instead of `{@link InputPartition}s` use `{@link InputPartition partitions}` to add the plural. That avoids awkward formatting and links that are missing 's'.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21326: [SPARK-24275][SQL] Revise doc comments in InputPa...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21326#discussion_r188354387
--- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/InputPartition.java ---
@@ -42,9 +43,9 @@
*
* Note that if a host name cannot be recognized by Spark, it will be ignored as it was not in
* the returned locations. By default this method returns empty string array, which means this
- * task has no location preference.
+ * data reader has no location preference.
--- End diff --
input partition
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90649/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21326
thanks, merging to master!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21326
**[Test build #90620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90620/testReport)** for PR 21326 at commit [`76a116a`](https://github.com/apache/spark/commit/76a116a93bd76b1f9fddec4f14515cf3a116a54f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90620/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3238/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21326: [SPARK-24275][SQL] Revise doc comments in InputPa...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21326
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21326
lgtm except one comment
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21326
**[Test build #90650 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90650/testReport)** for PR 21326 at commit [`f571750`](https://github.com/apache/spark/commit/f571750b26a7da936e48ba5e40528e6a16c43744).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3217/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21326
**[Test build #90649 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90649/testReport)** for PR 21326 at commit [`a5bfc1b`](https://github.com/apache/spark/commit/a5bfc1ba0f35dc052023304c79fcebcbff196530).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90650/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3239/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21326: [SPARK-24275][SQL] Revise doc comments in InputPa...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21326#discussion_r188354590
--- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/InputPartition.java ---
@@ -42,9 +43,9 @@
*
* Note that if a host name cannot be recognized by Spark, it will be ignored as it was not in
* the returned locations. By default this method returns empty string array, which means this
- * task has no location preference.
+ * data reader has no location preference.
--- End diff --
or `this input partition's reader`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by rdblue <gi...@git.apache.org>.
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21326
Belated +1.
In the future, I think it is better for readable docs to pluralize linked terms as I suggested in a comment.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21326
**[Test build #90650 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90650/testReport)** for PR 21326 at commit [`f571750`](https://github.com/apache/spark/commit/f571750b26a7da936e48ba5e40528e6a16c43744).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21326
**[Test build #90620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90620/testReport)** for PR 21326 at commit [`76a116a`](https://github.com/apache/spark/commit/76a116a93bd76b1f9fddec4f14515cf3a116a54f).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21326
**[Test build #90649 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90649/testReport)** for PR 21326 at commit [`a5bfc1b`](https://github.com/apache/spark/commit/a5bfc1ba0f35dc052023304c79fcebcbff196530).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21326: [SPARK-24275][SQL] Revise doc comments in InputPartition
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21326
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org