You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by GitBox <gi...@apache.org> on 2020/03/23 17:20:02 UTC
[GitHub] [beam] piotr-szuberski opened a new pull request #11194:
[BEAM-9507] Fix python dependency check task
piotr-szuberski opened a new pull request #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194
Use environment from buildPython task instead of creating one from scratch
@kamilwu
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu removed a comment on issue #11194: [BEAM-9507] Fix
python dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu removed a comment on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603239865
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11194: [BEAM-9507] Fix python
dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603277637
Dependency Check job has passed: https://builds.apache.org/job/beam_Dependency_Check/259/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11194: [BEAM-9507] Fix python
dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603115281
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11194: [BEAM-9507] Fix python
dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603246255
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] piotr-szuberski commented on issue #11194: [BEAM-9507] Fix
python dependency check task
Posted by GitBox <gi...@apache.org>.
piotr-szuberski commented on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603138587
I see. It wasn't clear to me why we cannot execute the dependency check. Now it's clear :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on a change in pull request #11194:
[BEAM-9507] Fix python dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#discussion_r397019242
##########
File path: sdks/python/build.gradle
##########
@@ -89,6 +89,7 @@ task depSnapshot {
task dependencyUpdates {
dependsOn ':dependencyUpdates'
+ dependsOn buildPython
Review comment:
I think this tasks does not install all dependencies we need.
We'd like to install all extras (there are currently five of them: docs, test, gcp, interactive, aws. You can see them in `setup.py`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11194: [BEAM-9507] Fix python
dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603204468
What about using `setupVirtualenv` as a dependency for your gradle task? You wouldn't have to set up a venv manually. Also, those tricky `rm -rf` in the script could have been deleted.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu merged pull request #11194: [BEAM-9507] Fix python
dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu merged pull request #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on a change in pull request #11194:
[BEAM-9507] Fix python dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#discussion_r397019242
##########
File path: sdks/python/build.gradle
##########
@@ -89,6 +89,7 @@ task depSnapshot {
task dependencyUpdates {
dependsOn ':dependencyUpdates'
+ dependsOn buildPython
Review comment:
I think this tasks does not install all dependencies we need.
We'd like to install all extras (there are currently five of them: docs, test, gcp, interactive, aws. You can see them in `setup.py`)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11194: [BEAM-9507] Fix python
dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603137245
Also, I'd rather not execute `beam_Dependency_Check` job until we're sure it works as expected. The job sends emails to dev@ and it'd be good to avoid sending any unnecessary emails.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11194: [BEAM-9507] Fix python
dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603246318
Run Dependency Check
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] piotr-szuberski commented on issue #11194: [BEAM-9507] Fix
python dependency check task
Posted by GitBox <gi...@apache.org>.
piotr-szuberski commented on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603235074
> What about using `setupVirtualenv` as a dependency for your gradle task? You wouldn't have to set up a venv manually. Also, those tricky `rm -rf` in the script could have been deleted.
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11194: [BEAM-9507] Fix python
dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603277716
LGTM, thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11194: [BEAM-9507] Fix python
dependency check task
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11194: [BEAM-9507] Fix python dependency check task
URL: https://github.com/apache/beam/pull/11194#issuecomment-603239865
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services