You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by cclauss <gi...@git.apache.org> on 2018/08/25 06:05:55 UTC
[GitHub] spark pull request #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle ...
GitHub user cclauss opened a pull request:
https://github.com/apache/spark/pull/22231
[MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4.0
See https://pycodestyle.readthedocs.io/en/latest/developer.html#changes for changes made in this release.
## What changes were proposed in this pull request?
Upgrade pycodestyle to v2.4.0
## How was this patch tested?
__pycodestyle__
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cclauss/spark patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22231.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22231
----
commit 00f34cafbe4b3526b48ac794677b9569e58ae01d
Author: cclauss <cc...@...>
Date: 2018-08-25T06:04:29Z
[MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4.0
See https://pycodestyle.readthedocs.io/en/latest/developer.html#changes for changes made in this release.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycode...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22231
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95250/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #96020 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96020/testReport)** for PR 22231 at commit [`719bb2b`](https://github.com/apache/spark/commit/719bb2b090dc12894aa9363c633dc8bc20bad45e).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #95245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95245/testReport)** for PR 22231 at commit [`00f34ca`](https://github.com/apache/spark/commit/00f34cafbe4b3526b48ac794677b9569e58ae01d).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by holdenk <gi...@git.apache.org>.
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/22231
This looks fine to me as well pending Sean's comment.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22231
Hm, I don't think the `# noqa` worked. That's the same failure. I thought it passed locally, hm. Well, I think that line should remove the noqa and remove the back-ticks. They aren't actually important.
Yes just fix the one last instance in `sql/functions.py`. The docstring needs an r in front of it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #96042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96042/testReport)** for PR 22231 at commit [`aba7ff9`](https://github.com/apache/spark/commit/aba7ff9499d1cce9c654d5e591baf16bb3c00fc4).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #96054 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96054/testReport)** for PR 22231 at commit [`4683649`](https://github.com/apache/spark/commit/4683649e4c49fbaa4049e551ae4045440d6eacb8).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22231
ok to test
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by cclauss <gi...@git.apache.org>.
Github user cclauss commented on the issue:
https://github.com/apache/spark/pull/22231
OK... __# noqa__ works within the flake8 wrapper around PyFlakes but does not work when PyFlakes is called outside of flake8. Also added r"comment" as you suggested.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by cclauss <gi...@git.apache.org>.
Github user cclauss commented on the issue:
https://github.com/apache/spark/pull/22231
Removed the escaped backticks.
@HyukjinKwon Undefined names (F821) is a flake8 error, not a PyCodeStyle error. That means that PyCodeStyle on its own (this PR) is unable to detect the problem with `__version__`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22231
Oh, that line isn't the error, although the warning says it is! It's really line 134, which actually has escaped back-ticks. I think you can honestly remove all of these back-ticks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22231
Merged to master/2.4
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #95245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95245/testReport)** for PR 22231 at commit [`00f34ca`](https://github.com/apache/spark/commit/00f34cafbe4b3526b48ac794677b9569e58ae01d).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22231
shall we fix them here? Looks not many instances
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [25238][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22231
Mind fixing the PR title to `[SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle to v2.4.0` so that it links the PR to JIRA?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by cclauss <gi...@git.apache.org>.
Github user cclauss commented on the issue:
https://github.com/apache/spark/pull/22231
@srowen Could you please provide some test cases in Python 2 and Python 3 to prove that the changes that you propose will be properly interpreted without side effects in both?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #96020 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96020/testReport)** for PR 22231 at commit [`719bb2b`](https://github.com/apache/spark/commit/719bb2b090dc12894aa9363c633dc8bc20bad45e).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #95250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95250/testReport)** for PR 22231 at commit [`b9b6a15`](https://github.com/apache/spark/commit/b9b6a15be6f1b813875debea22e975940d45fb82).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22231
Yeah I noticed that. I think we should leave it, and, if somehow RC1 passes, we'll mark this as fixed for a later release.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96042/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #96042 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96042/testReport)** for PR 22231 at commit [`aba7ff9`](https://github.com/apache/spark/commit/aba7ff9499d1cce9c654d5e591baf16bb3c00fc4).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95245/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [25238][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by cclauss <gi...@git.apache.org>.
Github user cclauss commented on the issue:
https://github.com/apache/spark/pull/22231
https://issues.apache.org/jira/browse/SPARK-25238
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #96065 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96065/testReport)** for PR 22231 at commit [`5ed4f0e`](https://github.com/apache/spark/commit/5ed4f0eb1b669a3c2effd0b0fe796e19db20b16f).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #95250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95250/testReport)** for PR 22231 at commit [`b9b6a15`](https://github.com/apache/spark/commit/b9b6a15be6f1b813875debea22e975940d45fb82).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by holdenk <gi...@git.apache.org>.
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/22231
I'm a little slammed this week until Friday, I'll try and cycle back then.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycode...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22231#discussion_r217428438
--- Diff: dev/run-tests-jenkins.py ---
@@ -116,7 +116,7 @@ def run_tests(tests_timeout):
failure_note_by_errcode = {
# error to denote run-tests script failures:
- 1: 'executing the `dev/run-tests` script', # noqa: W605
+ 1: 'executing the `dev/run-tests` script',
--- End diff --
I don't think we need to exclude this file... just remove the back-ticks here. They're just part of text, not functional.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96082/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #96082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96082/testReport)** for PR 22231 at commit [`67f21eb`](https://github.com/apache/spark/commit/67f21ebebf87f590f178883213f197e5c5a30d74).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22231
> OK... # noqa works within the flake8 wrapper around PyFlakes but does not work when PyFlakes is called outside of flake8. Also added r"comment" as you suggested.
Is this a bug in flake8? `# noqa` should work since I fixed one by `# noqa` - https://github.com/apache/spark/blob/5cdb8a23df6f269d6be0bf3536e9af9e29c4a05f/python/setup.py#L37
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #96082 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96082/testReport)** for PR 22231 at commit [`67f21eb`](https://github.com/apache/spark/commit/67f21ebebf87f590f178883213f197e5c5a30d74).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96020/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96054/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #96065 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96065/testReport)** for PR 22231 at commit [`5ed4f0e`](https://github.com/apache/spark/commit/5ed4f0eb1b669a3c2effd0b0fe796e19db20b16f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22231
Eh, @cclauss, mind filing a JIRA please?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22231
http://pycodestyle.pycqa.org/en/latest/intro.html
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by cclauss <gi...@git.apache.org>.
Github user cclauss commented on the issue:
https://github.com/apache/spark/pull/22231
OK... Removed backticks and we are once again testing ./dev/run-tests-jenkins.py
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22231
Interesting, errors like:
```
./dev/run-tests.py:172:7: W605 invalid escape sequence '\d'
```
on lines like:
```
match = re.search('(\d+)\.(\d+)\.(\d+)', raw_version_str)
```
How does the above actually work? I'd think `\d` just produces `d` in the pattern as backslash is also a Python escape, and this isn't a raw string. Does Python not interpret `\d` in this case because `d` isn't escaped in Python? https://docs.python.org/3/library/re.html
Might be worth heeding the warning and making all of these instances raw strings, to avoid the warning and follow the advice in the link above.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycode...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22231#discussion_r217626074
--- Diff: dev/run-tests-jenkins.py ---
@@ -131,7 +131,7 @@ def run_tests(tests_timeout):
ERROR_CODES["BLOCK_PYSPARK_UNIT_TESTS"]: 'PySpark unit tests',
ERROR_CODES["BLOCK_PYSPARK_PIP_TESTS"]: 'PySpark pip packaging tests',
ERROR_CODES["BLOCK_SPARKR_UNIT_TESTS"]: 'SparkR unit tests',
- ERROR_CODES["BLOCK_TIMEOUT"]: 'from timeout after a configured wait of \`%s\`' % (
+ ERROR_CODES["BLOCK_TIMEOUT"]: 'from timeout after a configured wait of %s' % (
--- End diff --
Can you just backquote here too likewise? Does this fail?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22231
I think we can simply skip, for example, `W504 line break after binary operator` per PEP8:
> In Python code, it is permissible to break before or after a binary operator, as long as the convention is consistent locally.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22231
**[Test build #96054 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96054/testReport)** for PR 22231 at commit [`4683649`](https://github.com/apache/spark/commit/4683649e4c49fbaa4049e551ae4045440d6eacb8).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by cclauss <gi...@git.apache.org>.
Github user cclauss commented on the issue:
https://github.com/apache/spark/pull/22231
@srowen Following the merge of #22400 there are just three W605s at the bottom of https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96020/console Should we __# noqa__ those lines or do you have a slicker solution/ Thanks massively for your persistence.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96065/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by cclauss <gi...@git.apache.org>.
Github user cclauss commented on the issue:
https://github.com/apache/spark/pull/22231
@holdenk Can you please do me a big favor and create a separate PR that makes sure that we are doing at least __flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics__ on the whole codebase on Python 2 and Python 3? When I create PRs on this project, I make so many mistakes that it takes pages of conversation and weeks of testing to get them merged. I just want to make sure that the project never backslides on F821s because it has taken [a lot of effort](https://github.com/apache/spark/pulls?q=is%3Apr+author%3Acclauss+is%3Aclosed) to get us clean. Thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22231
I can really answer my own question, I'm just being lazy. This seems to work in both Python 2 and 3:
```
>>> re.search('(\d+)\.(\d+)\.(\d+)', "1.2.3").group(1)
'1'
>>> re.search(r'(\d+)\.(\d+)\.(\d+)', "1.2.3").group(1)
'1'
>>> re.search('(\\d+)\.(\\d+)\.(\\d+)', "1.2.3").group(1)
'1'
```
I claim it's weird that the first one works at all, given that the other two do. That's the essence of the warning. Using the second form seems more logical.
It's not a small change to fix all of them. Although it's straightforward to make the strings into raw strings, there are potential landmines like `re.search('(\d+)\.(\d+)\.(\\d+)', "1.2.3").group(3)`. That's a bad regex as it mixes this confusing Python syntax behavior both ways; making in a raw string breaks the behavior :(
It'd be a worthy project to try to carefully evaluate and clean them up as I think most will be straightforward.
And the rest of the warnings looks like "line break after binary operator" which can be suppressed, and a few lines that are too long.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by cclauss <gi...@git.apache.org>.
Github user cclauss commented on the issue:
https://github.com/apache/spark/pull/22231
Let's wait until #22400 is merged before proceeding with this PR.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by rekhajoshm <gi...@git.apache.org>.
Github user rekhajoshm commented on the issue:
https://github.com/apache/spark/pull/22231
thanks for the fyi @HyukjinKwon , ack.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycode...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22231#discussion_r217719711
--- Diff: dev/run-tests-jenkins.py ---
@@ -131,7 +131,7 @@ def run_tests(tests_timeout):
ERROR_CODES["BLOCK_PYSPARK_UNIT_TESTS"]: 'PySpark unit tests',
ERROR_CODES["BLOCK_PYSPARK_PIP_TESTS"]: 'PySpark pip packaging tests',
ERROR_CODES["BLOCK_SPARKR_UNIT_TESTS"]: 'SparkR unit tests',
- ERROR_CODES["BLOCK_TIMEOUT"]: 'from timeout after a configured wait of \`%s\`' % (
+ ERROR_CODES["BLOCK_TIMEOUT"]: 'from timeout after a configured wait of %s' % (
--- End diff --
Yeah maybe but I don't see the value in these anyway. It seems to be trying to quote text, but the quote isn't even meaningful. It's not code.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22231
Note that, RC1 was cut before merging this PR, which means, this patch is not available in 2.4.0. I hit some problems running the release scripts and spent quite a lot of time to fix them, so the final vote is several days behind the RC1 tag creation.
@srowen please advice if we should
1. fail the RC1 to include this patch
2. do nothing and release it with 2.4.1
3. revert it from 2.4 since it's an upgrade.
Thanks!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22231
What do you think @cclauss ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22231
cc @rekhajoshm FYI
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22231
It's weird, `# noqa` should work pep8 (`pycodestyle`) too.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22231
@holdenk what do you think about the change I proposed above? I didn't know enough to know whether this was the right fix
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22231: [MINOR][PYTHON] lint-python: Upgrade pycodestyle to v2.4...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22231
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org