You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/12 21:00:26 UTC
[GitHub] [spark] DavidToneian opened a new pull request #27553:
[MINOR][DOCS] Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
DavidToneian opened a new pull request #27553: [MINOR][DOCS] Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553
This commit is published into the public domain.
### What changes were proposed in this pull request?
This PR fixes the documentation of `DataFrameReader`, `DataFrameWriter`, `DataStreamReader`, and `DataStreamWriter`, where attributes of other classes were misrepresented as functions. Additionally, creation of hyperlinks across modules was fixed in these instances.
### Why are the changes needed?
The old state produced documentation that suggested invalid usage of PySpark objects (accessing attributes as though they were callable.)
### Does this PR introduce any user-facing change?
No, except for improved documentation.
### How was this patch tested?
No test added; documentation build runs through.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585500632
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118328/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-586061374
Merged to master, branch-3.0 and branch-2.4.
Thanks, @DavidToneian for the contribution.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585500621
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#discussion_r378601062
##########
File path: python/pyspark/sql/readwriter.py
##########
@@ -616,7 +616,7 @@ def jdbc(self, url, table, column=None, lowerBound=None, upperBound=None, numPar
class DataFrameWriter(OptionUtils):
"""
Interface used to write a :class:`DataFrame` to external storage systems
- (e.g. file systems, key-value stores, etc). Use :func:`DataFrame.write`
+ (e.g. file systems, key-value stores, etc). Use :attr:`DataFrame.write`
Review comment:
Why is this `attr`? Technically this should be `meth`. Can you double check the generated documentation is correct? You can check via `cd python/docs && make clean html`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585500621
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585491616
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#discussion_r378601289
##########
File path: python/pyspark/sql/streaming.py
##########
@@ -276,9 +276,9 @@ def resetTerminated(self):
class DataStreamReader(OptionUtils):
"""
- Interface used to load a streaming :class:`DataFrame` from external storage systems
- (e.g. file systems, key-value stores, etc). Use :func:`spark.readStream`
- to access this.
+ Interface used to load a streaming :class:`DataFrame <pyspark.sql.DataFrame>` from external
+ storage systems (e.g. file systems, key-value stores, etc).
+ Use :attr:`SparkSession.readStream <pyspark.sql.SparkSession.readStream>` to access this.
Review comment:
ditto - technically this should be `meth`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27553:
[MINOR][DOCS] Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27553: [MINOR][DOCS] Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585417175
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585500299
**[Test build #118328 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118328/testReport)** for PR 27553 at commit [`6d0f6ed`](https://github.com/apache/spark/commit/6d0f6edbcadc83092eb0e9e7000de1f133fe8fa1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] DavidToneian commented on a change in pull request #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
DavidToneian commented on a change in pull request #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#discussion_r379093032
##########
File path: python/pyspark/sql/streaming.py
##########
@@ -276,9 +276,9 @@ def resetTerminated(self):
class DataStreamReader(OptionUtils):
"""
- Interface used to load a streaming :class:`DataFrame` from external storage systems
- (e.g. file systems, key-value stores, etc). Use :func:`spark.readStream`
- to access this.
+ Interface used to load a streaming :class:`DataFrame <pyspark.sql.DataFrame>` from external
Review comment:
Writing `` :class:`DataFrame` `` results in documentation that does not link to the class, and makes Sphinx output the following warning when running Sphinx in nit-picky mode (`-n` on the command line or `nitpicky = True` in the Sphinx config):
> spark/python/pyspark/sql/streaming.py:docstring of pyspark.sql.streaming.DataStreamReader:1: WARNING: py:class reference target not found: DataFrame
One could also just write `` :class:`pyspark.sql.DataFrame` `` to fix the warning and documentation linkage, but then the docs are rendered with the full `pyspark.sql.DataFrame` name. I figured rendering only `DataFrame` by writing `` :class:`DataFrame <pyspark.sql.DataFrame>` `` was more fitting with the rest of the docs, but I can of course change this is you'd prefer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on issue #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
maropu commented on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585490728
ok to test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585491255
**[Test build #118328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118328/testReport)** for PR 27553 at commit [`6d0f6ed`](https://github.com/apache/spark/commit/6d0f6edbcadc83092eb0e9e7000de1f133fe8fa1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#discussion_r379217246
##########
File path: python/pyspark/sql/streaming.py
##########
@@ -276,9 +276,9 @@ def resetTerminated(self):
class DataStreamReader(OptionUtils):
"""
- Interface used to load a streaming :class:`DataFrame` from external storage systems
- (e.g. file systems, key-value stores, etc). Use :func:`spark.readStream`
- to access this.
+ Interface used to load a streaming :class:`DataFrame <pyspark.sql.DataFrame>` from external
Review comment:
okie
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585417726
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585491620
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23085/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27553: [MINOR][DOCS]
Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27553: [MINOR][DOCS] Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585417175
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#discussion_r378601174
##########
File path: python/pyspark/sql/streaming.py
##########
@@ -276,9 +276,9 @@ def resetTerminated(self):
class DataStreamReader(OptionUtils):
"""
- Interface used to load a streaming :class:`DataFrame` from external storage systems
- (e.g. file systems, key-value stores, etc). Use :func:`spark.readStream`
- to access this.
+ Interface used to load a streaming :class:`DataFrame <pyspark.sql.DataFrame>` from external
Review comment:
Why `<pyspark.sql.DataFrame>` is needed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] DavidToneian commented on a change in pull request #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
DavidToneian commented on a change in pull request #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#discussion_r379089002
##########
File path: python/pyspark/sql/readwriter.py
##########
@@ -616,7 +616,7 @@ def jdbc(self, url, table, column=None, lowerBound=None, upperBound=None, numPar
class DataFrameWriter(OptionUtils):
"""
Interface used to write a :class:`DataFrame` to external storage systems
- (e.g. file systems, key-value stores, etc). Use :func:`DataFrame.write`
+ (e.g. file systems, key-value stores, etc). Use :attr:`DataFrame.write`
Review comment:
`DataFrame.write` is defined as a function decorated with `@property`, and as such behaves as an attribute, rather than a function. Indeed, one needs to write e.g. `df.write.saveAsTable('foo')` rather than `df.write().saveAsTable('foo')`; the latter will not work.
Changing `func` to `attr` makes Sphinx render `DataFrame.write` correctly, instead of misleadingly rendering `DataFrame.write()` in the documentation.
I've double-checked the resulting documentation, looks fine to me.
Edit: Thanks for reviewing! :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] DavidToneian commented on a change in pull request #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
DavidToneian commented on a change in pull request #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#discussion_r379093517
##########
File path: python/pyspark/sql/streaming.py
##########
@@ -276,9 +276,9 @@ def resetTerminated(self):
class DataStreamReader(OptionUtils):
"""
- Interface used to load a streaming :class:`DataFrame` from external storage systems
- (e.g. file systems, key-value stores, etc). Use :func:`spark.readStream`
- to access this.
+ Interface used to load a streaming :class:`DataFrame <pyspark.sql.DataFrame>` from external
+ storage systems (e.g. file systems, key-value stores, etc).
+ Use :attr:`SparkSession.readStream <pyspark.sql.SparkSession.readStream>` to access this.
Review comment:
See above -- `meth` would render like `func`, and wrongly suggest that one can call `readStream()`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on issue #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
maropu commented on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585490766
cc: @HyukjinKwon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27553: [MINOR][DOCS]
Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27553: [MINOR][DOCS] Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585417726
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585491620
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23085/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] DavidToneian commented on a change in pull request #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
DavidToneian commented on a change in pull request #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#discussion_r379089002
##########
File path: python/pyspark/sql/readwriter.py
##########
@@ -616,7 +616,7 @@ def jdbc(self, url, table, column=None, lowerBound=None, upperBound=None, numPar
class DataFrameWriter(OptionUtils):
"""
Interface used to write a :class:`DataFrame` to external storage systems
- (e.g. file systems, key-value stores, etc). Use :func:`DataFrame.write`
+ (e.g. file systems, key-value stores, etc). Use :attr:`DataFrame.write`
Review comment:
`DataFrame.write` is defined as a function decorated with `@property`, and as such behaves as an attribute, rather than a function. Indeed, one needs to write e.g. `df.write.saveAsTable('foo')` rather than `df.write().saveAsTable('foo')`; the latter will not work.
Changing `func` to `attr` makes Sphinx render `DataFrame.write` correctly, instead of misleadingly rendering `DataFrame.write()` in the documentation.
I've double-checked the resulting documentation, looks fine to me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#discussion_r379217221
##########
File path: python/pyspark/sql/readwriter.py
##########
@@ -616,7 +616,7 @@ def jdbc(self, url, table, column=None, lowerBound=None, upperBound=None, numPar
class DataFrameWriter(OptionUtils):
"""
Interface used to write a :class:`DataFrame` to external storage systems
- (e.g. file systems, key-value stores, etc). Use :func:`DataFrame.write`
+ (e.g. file systems, key-value stores, etc). Use :attr:`DataFrame.write`
Review comment:
Okay.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585491616
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27553:
[PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585500632
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118328/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27553: [PYSPARK][DOCS]
[MINOR]Changed `:func:` to `:attr:` Sphinx roles,
fixed links in documentation of `Data{Frame, Stream}{Reader, Writer}`.
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27553: [PYSPARK][DOCS] [MINOR]Changed `:func:` to `:attr:` Sphinx roles, fixed links in documentation of `Data{Frame,Stream}{Reader,Writer}`.
URL: https://github.com/apache/spark/pull/27553#issuecomment-585491255
**[Test build #118328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118328/testReport)** for PR 27553 at commit [`6d0f6ed`](https://github.com/apache/spark/commit/6d0f6edbcadc83092eb0e9e7000de1f133fe8fa1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org