You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by rekhajoshm <gi...@git.apache.org> on 2018/01/21 03:47:48 UTC

[GitHub] spark pull request #20338: [SPARK-11222][BUILD][PYTHON] python code style ch...

GitHub user rekhajoshm opened a pull request:

    https://github.com/apache/spark/pull/20338

    [SPARK-11222][BUILD][PYTHON] python code style checker update

    ## What changes were proposed in this pull request?
    Referencing latest python code style checking from PyPi/pycodestyle
    Removed pending TODO
    For now, excluded the additional style error in tox.ini discovered on existing python due to latest style checker (will fallback on review comment to finalize check exclusion or not)
    Any further code styling needs to be part of pycodestyle, not here.
    
    ## How was this patch tested?
    ./dev/run-tests


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rekhajoshm/spark SPARK-11222

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/20338.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #20338
    
----
commit e3677c9fa9697e0d34f9df52442085a6a481c9e9
Author: Rekha Joshi <re...@...>
Date:   2015-05-05T23:10:08Z

    Merge pull request #1 from apache/master
    
    Pulling functionality from apache spark

commit 106fd8eee8f6a6f7c67cfc64f57c1161f76d8f75
Author: Rekha Joshi <re...@...>
Date:   2015-05-08T21:49:09Z

    Merge pull request #2 from apache/master
    
    pull latest from apache spark

commit 0be142d6becba7c09c6eba0b8ea1efe83d649e8c
Author: Rekha Joshi <re...@...>
Date:   2015-06-22T00:08:08Z

    Merge pull request #3 from apache/master
    
    Pulling functionality from apache spark

commit 6c6ee12fd733e3f9902e10faf92ccb78211245e3
Author: Rekha Joshi <re...@...>
Date:   2015-09-17T01:03:09Z

    Merge pull request #4 from apache/master
    
    Pulling functionality from apache spark

commit b123c601e459d1ad17511fd91dd304032154882a
Author: Rekha Joshi <re...@...>
Date:   2015-11-25T18:50:32Z

    Merge pull request #5 from apache/master
    
    pull request from apache/master

commit c73c32aadd6066e631956923725a48d98a18777e
Author: Rekha Joshi <re...@...>
Date:   2016-03-18T19:13:51Z

    Merge pull request #6 from apache/master
    
    pull latest from apache spark

commit 7dbf7320057978526635bed09dabc8cf8657a28a
Author: Rekha Joshi <re...@...>
Date:   2016-04-05T20:26:40Z

    Merge pull request #8 from apache/master
    
    pull latest from apache spark

commit 5e9d71827f8e2e4d07027281b80e4e073e7fecd1
Author: Rekha Joshi <re...@...>
Date:   2017-05-01T23:00:30Z

    Merge pull request #9 from apache/master
    
    Pull apache spark

commit 63d99b3ce5f222d7126133170a373591f0ac67dd
Author: Rekha Joshi <re...@...>
Date:   2017-09-30T22:26:44Z

    Merge pull request #10 from apache/master
    
    pull latest apache spark

commit a7fc787466b71784ff86f9694f617db0f1042da8
Author: Rekha Joshi <re...@...>
Date:   2018-01-21T00:17:58Z

    Merge pull request #11 from apache/master
    
    Apache spark pull latest

commit 88734413c5e33803b7d5f8c0f81899ed1d3577ff
Author: rjoshi2 <re...@...>
Date:   2018-01-21T03:40:59Z

    [SPARK-11222][BUILD][PYTHON] python code style checker update

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/167/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-23174][BUILD][PYTHON] python code style ch...

Posted by rekhajoshm <gi...@git.apache.org>.
Github user rekhajoshm commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20338#discussion_r162838933
  
    --- Diff: dev/tox.ini ---
    @@ -13,7 +13,7 @@
     # See the License for the specific language governing permissions and
     # limitations under the License.
     
    -[pep8]
    -ignore=E402,E731,E241,W503,E226
    +[pycodestyle]
    +ignore=E402,E731,E241,W503,E226,E722,E741,E305
    --- End diff --
    
    Updated SPARK-11222. Added https://issues.apache.org/jira/browse/SPARK-23174.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86569/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86569/testReport)** for PR 20338 at commit [`34a8590`](https://github.com/apache/spark/commit/34a8590b7fa205be3d2b3f4913e252ab0e96d091).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/166/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/174/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-23174][BUILD][PYTHON] python code style ch...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20338#discussion_r163467168
  
    --- Diff: dev/tox.ini ---
    @@ -13,7 +13,7 @@
     # See the License for the specific language governing permissions and
     # limitations under the License.
     
    -[pep8]
    -ignore=E402,E731,E241,W503,E226
    +[pycodestyle]
    +ignore=E402,E731,E241,W503,E226,E722,E741,E305
    --- End diff --
    
    Yea .. there look actually many instances about it. I manually ran it against this PR. Will maybe take this one and open a PR separately after double checking each rule. Didn't take a look carefully yet.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/80/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-11222][BUILD][PYTHON] python code style ch...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20338#discussion_r162804771
  
    --- Diff: dev/tox.ini ---
    @@ -13,7 +13,7 @@
     # See the License for the specific language governing permissions and
     # limitations under the License.
     
    -[pep8]
    -ignore=E402,E731,E241,W503,E226
    +[pycodestyle]
    +ignore=E402,E731,E241,W503,E226,E722,E741,E305
    --- End diff --
    
    So, do those `E722,E741,E305` include rules to validate the blank line before starting doctests, which is described in the JIRA?
    
    Seems:
    E722: "do not use bare except'"
    E741: "ambiguous variable name 'l'", 
    E305: "expected 2 blank lines after class or function definition"
    
    



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    @rekhajoshm, mind updating variable names and one nit above? Will merge this one as soon as they are fixed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86578/testReport)** for PR 20338 at commit [`34a8590`](https://github.com/apache/spark/commit/34a8590b7fa205be3d2b3f4913e252ab0e96d091).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged to master.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-23174][BUILD][PYTHON] python code style ch...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/20338


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86559/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/184/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86423 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86423/testReport)** for PR 20338 at commit [`8873441`](https://github.com/apache/spark/commit/88734413c5e33803b7d5f8c0f81899ed1d3577ff).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86454/testReport)** for PR 20338 at commit [`c386a9a`](https://github.com/apache/spark/commit/c386a9a9b130e3974dc756b0fa89b7cff93f09ac).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86578 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86578/testReport)** for PR 20338 at commit [`34a8590`](https://github.com/apache/spark/commit/34a8590b7fa205be3d2b3f4913e252ab0e96d091).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-11222][BUILD][PYTHON] python code style ch...

Posted by rekhajoshm <gi...@git.apache.org>.
Github user rekhajoshm commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20338#discussion_r162837153
  
    --- Diff: dev/tox.ini ---
    @@ -13,7 +13,7 @@
     # See the License for the specific language governing permissions and
     # limitations under the License.
     
    -[pep8]
    -ignore=E402,E731,E241,W503,E226
    +[pycodestyle]
    +ignore=E402,E731,E241,W503,E226,E722,E741,E305
    --- End diff --
    
    Here - https://github.com/PyCQA/pycodestyle/issues/723


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-23174][BUILD][PYTHON] python code style ch...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20338#discussion_r163243073
  
    --- Diff: dev/lint-python ---
    @@ -35,11 +35,10 @@ compile_status="${PIPESTATUS[0]}"
     
     # Get pep8 at runtime so that we don't rely on it being installed on the build server.
     #+ See: https://github.com/apache/spark/pull/1744#issuecomment-50982162
    -#+ TODOs:
    -#+  - Download pep8 from PyPI. It's more "official".
    -PEP8_VERSION="1.7.0"
    +# Updated to latest official version for pep8.pep8 is formally renamed to pycodestyle.
    --- End diff --
    
    nit .. : `pep8.pep8` -> `pep8. pep8`


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86423/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Ah, seems this one doesn't resolve SPARK-11222 too. I think that JIRA describes to add `pydocstyle` check in our project too given my reading of @rekhajoshm's issue link. Will double check and take an action for SPARK-11222 soon. 
    
    This JIRA, SPARK-23174, seems describing simply upgrading the pep8 (formally renamed to `pycodestyle`).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/83/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86454/testReport)** for PR 20338 at commit [`c386a9a`](https://github.com/apache/spark/commit/c386a9a9b130e3974dc756b0fa89b7cff93f09ac).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86451 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86451/testReport)** for PR 20338 at commit [`8718891`](https://github.com/apache/spark/commit/871889113057f819337bd24379cf1f07516c3298).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-11222][BUILD][PYTHON] python code style ch...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20338#discussion_r162804815
  
    --- Diff: dev/lint-python ---
    @@ -35,11 +35,9 @@ compile_status="${PIPESTATUS[0]}"
     
     # Get pep8 at runtime so that we don't rely on it being installed on the build server.
     #+ See: https://github.com/apache/spark/pull/1744#issuecomment-50982162
    -#+ TODOs:
    -#+  - Download pep8 from PyPI. It's more "official".
    -PEP8_VERSION="1.7.0"
    +PEP8_VERSION="2.3.1"
     PEP8_SCRIPT_PATH="$SPARK_ROOT_DIR/dev/pep8-$PEP8_VERSION.py"
    -PEP8_SCRIPT_REMOTE_PATH="https://raw.githubusercontent.com/jcrocholl/pep8/$PEP8_VERSION/pep8.py"
    +PEP8_SCRIPT_REMOTE_PATH="https://raw.githubusercontent.com/PyCQA/pycodestyle/$PEP8_VERSION/pycodestyle.py"
    --- End diff --
    
    Shall we leave a note that `pep8` is formally renamed to `pycodestyle` to reduce confusion, and in the PR description too?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-11222][BUILD][PYTHON] python code style ch...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20338#discussion_r162836359
  
    --- Diff: dev/tox.ini ---
    @@ -13,7 +13,7 @@
     # See the License for the specific language governing permissions and
     # limitations under the License.
     
    -[pep8]
    -ignore=E402,E731,E241,W503,E226
    +[pycodestyle]
    +ignore=E402,E731,E241,W503,E226,E722,E741,E305
    --- End diff --
    
    Ah, I asked this to double check if this PR fixes the issue linked JIRA. I think we could make another JIRA if this PR doesn't target to fix the linked JIRA. For the best, it'd be nicer if you'd leave a issue link you raised under pep8, if I understood you correctly.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86560/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86560/testReport)** for PR 20338 at commit [`34a8590`](https://github.com/apache/spark/commit/34a8590b7fa205be3d2b3f4913e252ab0e96d091).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86560 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86560/testReport)** for PR 20338 at commit [`34a8590`](https://github.com/apache/spark/commit/34a8590b7fa205be3d2b3f4913e252ab0e96d091).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86559/testReport)** for PR 20338 at commit [`d10fbb4`](https://github.com/apache/spark/commit/d10fbb49cc6b062bfa7d463f0bae098c77b9a890).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    This doesn't actually run the Python lint in Jenkins build though because there's no change on the files with `.py`. Mind double checking and fixing:
    
    ```diff
    diff --git a/dev/run-tests.py b/dev/run-tests.py
    index 7e6f7ff0603..6fac324972e 100755
    --- a/dev/run-tests.py
    +++ b/dev/run-tests.py
    @@ -576,7 +576,10 @@ def main():
                                     for f in changed_files):
             # run_java_style_checks()
             pass
    -    if not changed_files or any(f.endswith(".py") for f in changed_files):
    +    if not changed_files or any(f.endswith("lint-python")
    +                                or f.endswith("tox.ini")
    +                                or f.endswith(".py")
    +                                for f in changed_files):
             run_python_style_checks()
         if not changed_files or any(f.endswith(".R") for f in changed_files):
             run_sparkr_style_checks()
    ```
    
    too?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-23174][BUILD][PYTHON] python code style ch...

Posted by ueshin <gi...@git.apache.org>.
Github user ueshin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20338#discussion_r162852255
  
    --- Diff: dev/lint-python ---
    @@ -35,11 +35,10 @@ compile_status="${PIPESTATUS[0]}"
     
     # Get pep8 at runtime so that we don't rely on it being installed on the build server.
     #+ See: https://github.com/apache/spark/pull/1744#issuecomment-50982162
    -#+ TODOs:
    -#+  - Download pep8 from PyPI. It's more "official".
    -PEP8_VERSION="1.7.0"
    +# Updated to latest official version for pep8.pep8 is formally renamed to pycodestyle.
    +PEP8_VERSION="2.3.1"
    --- End diff --
    
    Should we also use `pycodestyle` instead of `pep8` for variable names or script names?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86569 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86569/testReport)** for PR 20338 at commit [`34a8590`](https://github.com/apache/spark/commit/34a8590b7fa205be3d2b3f4913e252ab0e96d091).
     * This patch **fails due to an unknown error code, -9**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-11222][BUILD][PYTHON] python code style ch...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20338#discussion_r162836431
  
    --- Diff: dev/run-tests.py ---
    @@ -238,7 +238,6 @@ def build_spark_documentation():
             sys.exit(int(os.environ.get("CURRENT_BLOCK", 255)))
         else:
             run_cmd([jekyll_bin, "build"])
    -
    --- End diff --
    
    I know I am insane but let's revert back unrelated changes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86578/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86454/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86559 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86559/testReport)** for PR 20338 at commit [`d10fbb4`](https://github.com/apache/spark/commit/d10fbb49cc6b062bfa7d463f0bae098c77b9a890).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by rekhajoshm <gi...@git.apache.org>.
Github user rekhajoshm commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    @HyukjinKwon @ueshin was bit hesitant to change variable/references as pep8 is very clear within python circles as python style checker.however after pondering, have done the needful. please check.thanks
    
    @srowen for SPARK-11222, I had raised a issue under pycodestyle as it does not belong within Spark.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #20338: [SPARK-23174][BUILD][PYTHON] python code style ch...

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20338#discussion_r163465854
  
    --- Diff: dev/tox.ini ---
    @@ -13,7 +13,7 @@
     # See the License for the specific language governing permissions and
     # limitations under the License.
     
    -[pep8]
    -ignore=E402,E731,E241,W503,E226
    +[pycodestyle]
    +ignore=E402,E731,E241,W503,E226,E722,E741,E305
    --- End diff --
    
    should we fix the code that violates
    
    E722: "do not use bare except'"
    E741: "ambiguous variable name 'l'",
    E305: "expected 2 blank lines after class or function definition"
    ?
    
    Or that's another PR/JIRA?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/62/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86423/testReport)** for PR 20338 at commit [`8873441`](https://github.com/apache/spark/commit/88734413c5e33803b7d5f8c0f81899ed1d3577ff).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86451/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    **[Test build #86451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86451/testReport)** for PR 20338 at commit [`8718891`](https://github.com/apache/spark/commit/871889113057f819337bd24379cf1f07516c3298).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    (If this also resolves SPARK-11222 you can also include that in the title.)


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #20338: [SPARK-11222][BUILD][PYTHON] python code style checker u...

Posted by rekhajoshm <gi...@git.apache.org>.
Github user rekhajoshm commented on the issue:

    https://github.com/apache/spark/pull/20338
  
    Thanks @HyukjinKwon for your review.updated, please verify.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org