You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by GitBox <gi...@apache.org> on 2020/03/17 00:46:22 UTC
[GitHub] [beam] davidyan74 opened a new pull request #11142: [BEAM-9510]
Fixing version incompatibilities in base_image_requirements.txt
davidyan74 opened a new pull request #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142
Tried this out and it doesn't have any compatibility error messages any more for python3.
R: @tvalentyn
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] tvalentyn commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-600167192
Test logs LGTM: https://builds.apache.org/job/beam_PostCommit_Py_ValCont_PR/123/timestamps/?time=HH:mm:ss&timeZone=GMT-7&appendLog&locale=en_US
FYI @amaliujia - containers on the latest release branch may also be broken and require changes to fix incompatible dependencies. Do you think you can check this? Otherwise it may come up during release validation. You can make a similar change to the release branch, then have a look at the logs of ValidatesContainer suite against the release branch, and make sure pip does not complain about incompatible dependencies.
@Hannah-Jiang and @davidyan74 also can help if you have questions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] tvalentyn commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-599826365
Run Python Dataflow ValidatesContainer
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] tvalentyn commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-599896374
Thanks, @davidyan74! Added a small change to reflect that `typing` should only be installed on Python 2 (this was recently changed in Beam master).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] davidyan74 commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
davidyan74 commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-600268879
Fiiled BEAM-9530.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] tvalentyn commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-599825064
Run Dataflow Python ValidatesContainer
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] tvalentyn commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-599896422
Run Python Dataflow ValidatesContainer
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] tvalentyn commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-599906766
While Jenkins does not show test result on the PR, the tests are actually running: https://builds.apache.org/job/beam_PostCommit_Py_ValCont_PR/122/.
Let's check the console logs that there are no conflicts before the merge.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] tvalentyn commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-599900258
LGTM
A few things out of scope of this PR:
1) This PR fixes the incompatibility in dependencies, but the list of dependencies is in base_image_requirements.txt is still not exhaustive. So, container builds may not be repeatable over time, and there is a risk that we will introduce incompatible dependencies down the road and won't notice it.
2) We can consider switching the base image to `python:slim-stretch` instead of `python:stretch` to reduce the size of the images: https://github.com/apache/beam/blob/master/sdks/python/container/Dockerfile#L20
cc: @Hannah-Jiang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] davidyan74 commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
davidyan74 commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-600268032
Thanks @tvalentyn. We might want to add `pip check` after pip install in our tests to make sure there is no incompatibility going forward. `pip install` does not return an error exit code for broken dependencies for historical reasons.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] tvalentyn commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-599922203
Run Python Dataflow ValidatesContainer
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] davidyan74 commented on issue #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
davidyan74 commented on issue #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142#issuecomment-599822693
R: @aaltay
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] tvalentyn merged pull request #11142: [BEAM-9510] Fixing
version incompatibilities in base_image_requirements.txt
Posted by GitBox <gi...@apache.org>.
tvalentyn merged pull request #11142: [BEAM-9510] Fixing version incompatibilities in base_image_requirements.txt
URL: https://github.com/apache/beam/pull/11142
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services