You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/05/14 13:32:36 UTC
[GitHub] [beam] kamilwu opened a new pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
kamilwu opened a new pull request #11707:
URL: https://github.com/apache/beam/pull/11707
List of changes:
* added `py38`, `py38-cython` and `py38-cloud` tox suites (all suites pass)
* made `preCommitPy38` a part of pythonPreCommit. I think it's acceptable to extend pythonPreCommit execution time and make sure the codebase supports Python 3.8 until we prioritize python versions.
* `py37-docs` became a `py38-docs`
* moved `formatter` task to py38
I didn't change py37-lint to py38-lint, because this requires updating dependencies (for example, our current version of pycodestyle doesn't support Python 3.8)
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-629456098
re:
> I didn't change py37-lint to py38-lint, because this requires updating dependencies (for example, our current version of pycodestyle doesn't support Python 3.8
Upgrading dependency should not be a problem as long as there is a version that works. We leave lint under py37 for now, and later configure to run it against highest "high-pri" version, once we have that concept.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634635541
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu removed a comment on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu removed a comment on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634714585
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn merged pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
tvalentyn merged pull request #11707:
URL: https://github.com/apache/beam/pull/11707
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634714585
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu removed a comment on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu removed a comment on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634701208
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-628683817
R: @lazylynx @tvalentyn
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634701208
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634513015
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634614486
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-628685339
It seems that creating a virtualenv using Python 3.8 fails on Jenkins workers. Created a jira issue: https://issues.apache.org/jira/browse/BEAM-9994
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634596264
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-631500053
> We need to address BEAM-9994 before we can merge this.
I think we get it. Please see the description of the issue.
However, I'm a bit worried about the time needed to prepare venv for Python 3.8. It took 16 min to complete [this](https://builds.apache.org/job/beam_PreCommit_PythonFormatter_Phrase/27/) formatter task, while normally it takes a minute or two. Log shows that the most time consuming is building wheel for grpcio-tools.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lazylynx commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
lazylynx commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-631604104
LGTM, Thanks!
I checked pydoc generated with `py38-docs` to make sure and it's OK.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634649590
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu removed a comment on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu removed a comment on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634649590
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634783915
Yes - feel free to self-merge once Python precommits pass. Just checking that they still work after Jenkins changes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-631385686
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-631622571
Thanks! Let's merge this once BEAM-9994 is closed. I will add comments there.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634542088
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-631472237
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634774229
@tvalentyn [BEAM-9994](https://issues.apache.org/jira/browse/BEAM-9994) is closed. Can we move forward?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-631067325
The code change LGTM, thank you. We need to address BEAM-9994 before we can merge this. Would you have time to investigate & recommend a solution for that issue, @kamilwu ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634664271
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634614624
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634783372
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-631385430
Run PythonFormatter PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-631393434
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu removed a comment on pull request #11707: [BEAM-9810] Add a Tox (precommit) suite for Python 3.8
Posted by GitBox <gi...@apache.org>.
kamilwu removed a comment on pull request #11707:
URL: https://github.com/apache/beam/pull/11707#issuecomment-634614624
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org