You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/26 10:45:53 UTC

[GitHub] [spark] HyukjinKwon opened a new pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

HyukjinKwon opened a new pull request #30149:
URL: https://github.com/apache/spark/pull/30149


   ### What changes were proposed in this pull request?
   
   This PR proposes to migrate to NumPy documentation style (from reST style) in PySpark docstrings.
   This PR also adds one migration example of `SparkContext`.
   
   - **Before:**
       ...
       ![Screen Shot 2020-10-26 at 7 02 05 PM](https://user-images.githubusercontent.com/6477701/97161090-a8ea0200-17c0-11eb-8204-0e70d18fc571.png)
       ...
       ![Screen Shot 2020-10-26 at 7 02 09 PM](https://user-images.githubusercontent.com/6477701/97161100-aab3c580-17c0-11eb-92ad-f5ad4441ce16.png)
       ...
   
   - **After:**
   
       ...
       ![Screen Shot 2020-10-26 at 7 24 08 PM](https://user-images.githubusercontent.com/6477701/97161219-d636b000-17c0-11eb-80ab-d17a570ecb4b.png)
       ...
   
   ### Why are the changes needed?
   
   There are many reasons for switching to NumPy documentation style.
   
   1. Arguably reST style doesn't fit well when the docstring grows large because it provides (arguably) less structures and syntax.
   
   2. NumPy documentation style provides a better human readable docstring format. For example, notebook users often just do `help(...)` by `pydoc`.
   
   3. NumPy documentation style is pretty commonly used in data science libraries, for example, pandas, numpy, Dask, Koalas, 
   matplotlib, ... Using NumPy documentation style can give users a consistent documentation style.
   
   ### Does this PR introduce _any_ user-facing change?
   
   The dependency itself doesn't change anything user-facing.
   The documentation change in `SparkContext` does, as shown above.
   
   ### How was this patch tested?
   
   Manually tested via running `cd python` and `make clean html`.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716485162






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716538486






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716508213






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716487067


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/130280/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716538777


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34881/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716538787


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716508213






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon edited a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716490037


   Just for a bit of more contexts, I plan to completely migrate to numpydoc style, and then add a plenty of examples and scenarios - our current examples are pretty poorly documented and not enough vs, for example, [pandas.groupby](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.groupby.html#pandas.DataFrame.groupby) vs [pyspark.groupBy](https://spark.apache.org/docs/latest/api/python/pyspark.sql.html?highlight=groupby#pyspark.sql.DataFrame.groupBy).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zero323 commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
zero323 commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-724361103


   __Note__:
   
   Seems like ellipsis at the end
   
   ```python
       >>> sc2 = SparkContext('local', 'test2') # doctest: +IGNORE_EXCEPTION_DETAIL
       Traceback (most recent call last):
           ...
       ValueError:...
   ```
   
   breaks syntax highlighting. Adding a whitespace
   
   ```python
       >>> sc2 = SparkContext('local', 'test2') # doctest: +IGNORE_EXCEPTION_DETAIL
       Traceback (most recent call last):
           ...
       ValueError: ...
   ```
   resolves the issue.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716574140






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716498349


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34879/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716508197


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34879/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716538486






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716485235






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716486943


   **[Test build #130280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130280/testReport)** for PR 30149 at commit [`8e67437`](https://github.com/apache/spark/commit/8e674375308b7daae86b68dbb3099b8dc852d92c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716574140






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716487029






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716538787






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #30149:
URL: https://github.com/apache/spark/pull/30149#discussion_r511868759



##########
File path: python/docs/source/_templates/autosummary/class.rst
##########
@@ -0,0 +1,38 @@
+..  Licensed to the Apache Software Foundation (ASF) under one

Review comment:
       I referred to pandas' https://raw.githubusercontent.com/pandas-dev/pandas/master/doc/_templates/autosummary/class.rst




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716467538


   **[Test build #130275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130275/testReport)** for PR 30149 at commit [`62c7b18`](https://github.com/apache/spark/commit/62c7b18f9ad8779bf08a656857ca77af0f1bbeb9).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716983027


   Thank you guys. Merged to master.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] BryanCutler commented on a change in pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
BryanCutler commented on a change in pull request #30149:
URL: https://github.com/apache/spark/pull/30149#discussion_r512337126



##########
File path: python/pyspark/context.py
##########
@@ -83,40 +130,6 @@ class SparkContext(object):
     def __init__(self, master=None, appName=None, sparkHome=None, pyFiles=None,
                  environment=None, batchSize=0, serializer=PickleSerializer(), conf=None,
                  gateway=None, jsc=None, profiler_cls=BasicProfiler):
-        """

Review comment:
       So the constructor param descriptions get moved to the class doc? 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716538464


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34883/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716485162






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon edited a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716490037


   Just for a bit of more contexts, I plan to completely migrate to numpydoc style, and then add a plenty of examples and scenarios - our current examples are pretty poorly documented and not enough vs, for example, [pandas.groupby](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.groupby.html#pandas.DataFrame.groupby) vs [pyspark.groupBy](https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrame.groupBy).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716542326


   It does require to rewrite a lot of docs. I am thinking that in most cases there would not be too much diff except when rewriting parameters. 
   
   By doing that, it can show docstrings with a better human readable text format.
   
   Interactive shall users (or notebook users) can now see the better docs via running `help(...)` 
    which shows a text docstring, e.g., `help(pyspark.sql.DataFrame)`. It covers this case as well.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716572355


   **[Test build #130282 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130282/testReport)** for PR 30149 at commit [`89801ca`](https://github.com/apache/spark/commit/89801caa82298b0f4f7be5a620b2ae0ae58b8089).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #30149:
URL: https://github.com/apache/spark/pull/30149#discussion_r511894618



##########
File path: python/pyspark/context.py
##########
@@ -83,40 +130,6 @@ class SparkContext(object):
     def __init__(self, master=None, appName=None, sparkHome=None, pyFiles=None,
                  environment=None, batchSize=0, serializer=PickleSerializer(), conf=None,
                  gateway=None, jsc=None, profiler_cls=BasicProfiler):
-        """

Review comment:
       I removed `__init__` because the doc is already pretty intuitive about that:
   ![Screen Shot 2020-10-26 at 8 34 03 PM](https://user-images.githubusercontent.com/6477701/97167743-aee4e080-17ca-11eb-9730-e0fc89b41e42.png)
   
    




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716503022


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34875/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716490470


   **[Test build #130282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130282/testReport)** for PR 30149 at commit [`89801ca`](https://github.com/apache/spark/commit/89801caa82298b0f4f7be5a620b2ae0ae58b8089).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716479631


   **[Test build #130279 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130279/testReport)** for PR 30149 at commit [`2feb2db`](https://github.com/apache/spark/commit/2feb2dba48e1a7ef3b0d5b00e577e8f1a7d2b480).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716503043






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-724370163


   Thanks @zero323!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716522333


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34881/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon closed pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #30149:
URL: https://github.com/apache/spark/pull/30149


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716490035


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34875/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716526674


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34883/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716503043






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716490037


   Just for a bit of more contexts, I plan to completely migrate to numpydoc style, and then add a plenty of examples and scenarios - our current examples are pretty poorly documented and not enough vs, for example, [pandas.groupby](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.groupby.html#pandas.DataFrame.groupby) vs [pyspark.groupby](https://spark.apache.org/docs/latest/api/python/pyspark.sql.html?highlight=groupby#pyspark.sql.DataFrame.groupby).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716538795


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/34881/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716490470


   **[Test build #130282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130282/testReport)** for PR 30149 at commit [`89801ca`](https://github.com/apache/spark/commit/89801caa82298b0f4f7be5a620b2ae0ae58b8089).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716466754


   cc @zero323, @holdenk, @srowen, @Fokko, @ueshin, @BryanCutler, @huaxingao, @zhengruifeng FYI


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30149: [SPARK-33243][PYTHON][BUILD] Add numpydoc into documentation dependency

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30149:
URL: https://github.com/apache/spark/pull/30149#issuecomment-716486943






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org