You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/05/27 03:30:30 UTC
[GitHub] [beam] epicfaace opened a new pull request #11819: [BEAM-8371] Remove support for Python 2
epicfaace opened a new pull request #11819:
URL: https://github.com/apache/beam/pull/11819
This PR does two things:
- Raises a `RuntimeError` if Python 2 is used (just like we currently raise a `RuntimeError` if anything other than Python 2.7 / 3 is used)
- Remove tests and build processes that are specific to Python 2. I just deleted most of these tests, but I moved some of them, such as `chicagoTaxiExample`, from Python 2 to Python 3.5 because they didn't seem to be covered in the existing Python 3 tests.
I'm not sure when we want to move ahead with removing support for Python 2 (presumably we will do so this year), but wanted to make this PR so we would be ready to do so.
Can you run the Jenkins tests on this PR to see where we are on this (and if there is anything in addition that needs to be changed)?
R: @pabloem @tvalentyn
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] epicfaace commented on a change in pull request #11819: [BEAM-8371] Remove support for Python 2
Posted by GitBox <gi...@apache.org>.
epicfaace commented on a change in pull request #11819:
URL: https://github.com/apache/beam/pull/11819#discussion_r430705916
##########
File path: sdks/python/findSupportedPython.groovy
##########
@@ -18,7 +18,7 @@
/* This (groovy-maven-plugin) script finds the supported python interpreter and pip
* binary in the path. As there is no strict naming convention exists amongst OSes
- * for Python & pip (some call it python2.7, others name it python-2.7),
+ * for Python & pip (some call it python3.5, others name it python-3.5),
Review comment:
This file only seems to be using Python 2, so I wasn't sure if we still needed it.
##########
File path: sdks/python/findSupportedPython.groovy
##########
@@ -18,7 +18,7 @@
/* This (groovy-maven-plugin) script finds the supported python interpreter and pip
* binary in the path. As there is no strict naming convention exists amongst OSes
- * for Python & pip (some call it python2.7, others name it python-2.7),
+ * for Python & pip (some call it python3.5, others name it python-3.5),
Review comment:
This file only seems to be using Python 2, so I wasn't sure if we still needed it. Can we safely delete it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #11819: [BEAM-8371] Remove support for Python 2
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on pull request #11819:
URL: https://github.com/apache/beam/pull/11819#issuecomment-634282371
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] epicfaace commented on pull request #11819: [BEAM-8371] Remove support for Python 2
Posted by GitBox <gi...@apache.org>.
epicfaace commented on pull request #11819:
URL: https://github.com/apache/beam/pull/11819#issuecomment-634347950
@tvalentyn Got it. I didn't think that the actual build files for python 2 would change much until the time we drop Py2 support, so that's why I thought this PR would be useful for when we actually drop Py2 support.
What parts of this PR would be fine to do for now? Do you think we could move some tests (such as the runner tests) to py3 for now, as @pabloem mentioned?
Also, do you have an idea of when we might want to drop Py2 support? We did commit to the fact that we would drop Py2 support in 2020 -- are you thinking sometime in the fall?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #11819: [BEAM-8371] Remove support for Python 2
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on pull request #11819:
URL: https://github.com/apache/beam/pull/11819#issuecomment-636127486
Closing for now sounds good, note that PR is not lost as we can reopen, but it will signal committers that it is not requiring immediate attention.
We do run several test suites on Python 3; test suites should be configured using high priority and low priority versions (see the discussion: https://lists.apache.org/thread.html/re621331e10896ac65f487c1a83cc4a91152e2fd6d7e363c115b1857f%40%3Cdev.beam.apache.org%3E).
If some suite does not have coverage on Py3, it would be useful to add such coverage.
Re: When Beam will drop Py2 coverage - this is under discussion, see latest updates on threads referenced in [BEAM-8371].
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11819: [BEAM-8371] Remove support for Python 2
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11819:
URL: https://github.com/apache/beam/pull/11819#issuecomment-635669412
Should we close this pull request for now?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11819: [BEAM-8371] Remove support for Python 2
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11819:
URL: https://github.com/apache/beam/pull/11819#issuecomment-634281064
thanks @epicfaace - I suspect it may be a little early to remove all of our Python 2 tests, as we still have some users, and we don't want to leave them hanging.
I think it's good to move the runner tests to Python 3, as you have here.
I'll let @tvalentyn make the call regarding which tests should be moved to py3, which should be removed and which should remain.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] epicfaace closed pull request #11819: [BEAM-8371] Remove support for Python 2
Posted by GitBox <gi...@apache.org>.
epicfaace closed pull request #11819:
URL: https://github.com/apache/beam/pull/11819
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org