You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/07/14 22:37:34 UTC
[GitHub] [beam] saavannanavati opened a new pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
saavannanavati opened a new pull request #12257:
URL: https://github.com/apache/beam/pull/12257
This PR enables the generation of Python coverage reports during pre-commit in the `py38-cloud` Tox environment.
- The resultant reports are uploaded to `codecov.io`, as can be seen [here](https://codecov.io/gh/saavannanavati/beam/tree/7e9d684698adbe2654fb59a734553c4f1682c356/sdks/python/apache_beam).
- To work on the live testing infrastructure, an environment variable `CODECOV_TOKEN` must be defined on Jenkins with a value that can be found [here](https://codecov.io/gh/apache/beam/).
The full design document related to this PR can be found [here](https://docs.google.com/document/d/1K6zEJnDNvk7TkqDEPwoIa7NTj5KxtUIdq-TGewUgpmo/edit).
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
--- | --- | --- | --- | --- | --- | ---
Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
XLang | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] saavannanavati commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
saavannanavati commented on a change in pull request #12257:
URL: https://github.com/apache/beam/pull/12257#discussion_r466614068
##########
File path: sdks/python/test-suites/tox/py38/build.gradle
##########
@@ -32,4 +32,4 @@ check.dependsOn formatter
apply from: "../common.gradle"
// TODO(BEAM-8954): Remove this once tox uses isolated builds.
-testPy38Cython.mustRunAfter testPython38, testPy38Cloud
+testPy38Cython.mustRunAfter testPython38, python38CloudCoverage
Review comment:
Thanks! Fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] saavannanavati commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
saavannanavati commented on a change in pull request #12257:
URL: https://github.com/apache/beam/pull/12257#discussion_r454738417
##########
File path: sdks/python/setup.py
##########
@@ -193,6 +193,8 @@ def get_version():
'requests_mock>=1.7,<2.0',
'tenacity>=5.0.2,<6.0',
'pytest>=4.4.0,<5.0',
+ # More recent versions of pytest-cov do not support pytest 4.4.0
+ 'pytest-cov==2.9.0',
Review comment:
Alternatively, this could be a dependency only for `py38-cloud`. That would make more sense since no other Tox environment requires coverage. Should I move it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] saavannanavati commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
saavannanavati commented on a change in pull request #12257:
URL: https://github.com/apache/beam/pull/12257#discussion_r454738417
##########
File path: sdks/python/setup.py
##########
@@ -193,6 +193,8 @@ def get_version():
'requests_mock>=1.7,<2.0',
'tenacity>=5.0.2,<6.0',
'pytest>=4.4.0,<5.0',
+ # More recent versions of pytest-cov do not support pytest 4.4.0
+ 'pytest-cov==2.9.0',
Review comment:
Alternatively, this could be a dependency only for `py38-cloud`. That makes more sense to me now that I think about it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-672995840
run seed job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-665218601
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] saavannanavati commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
saavannanavati commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-670161660
run python precommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on a change in pull request #12257:
URL: https://github.com/apache/beam/pull/12257#discussion_r459626048
##########
File path: sdks/python/test-suites/tox/py38/build.gradle
##########
@@ -29,7 +29,10 @@ pythonVersion = '3.8'
toxTask "formatter", "py3-yapf-check"
check.dependsOn formatter
+toxTask "testPy38CloudCoverage", "py38-cloudcoverage"
+test.dependsOn "testPy38CloudCoverage"
+
apply from: "../common.gradle"
// TODO(BEAM-8954): Remove this once tox uses isolated builds.
-testPy38Cython.mustRunAfter testPython38, testPy38Cloud
+testPy38Cython.mustRunAfter testPython38, testPy38Cloud, testPy38CloudCoverage
Review comment:
No need to run testPy38Cloud
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on a change in pull request #12257:
URL: https://github.com/apache/beam/pull/12257#discussion_r466612715
##########
File path: sdks/python/test-suites/tox/py38/build.gradle
##########
@@ -32,4 +32,4 @@ check.dependsOn formatter
apply from: "../common.gradle"
// TODO(BEAM-8954): Remove this once tox uses isolated builds.
-testPy38Cython.mustRunAfter testPython38, testPy38Cloud
+testPy38Cython.mustRunAfter testPython38, python38CloudCoverage
Review comment:
I think you meant `testPy38CloudCoverage`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] saavannanavati commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
saavannanavati commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-660516111
Looks like it takes ~ 5 min longer on Jenkins
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-670612461
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] saavannanavati commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
saavannanavati commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-664695003
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] saavannanavati commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
saavannanavati commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-658447736
R: @udim
R: @robertwb
CC: @tysonjh
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on a change in pull request #12257:
URL: https://github.com/apache/beam/pull/12257#discussion_r468943368
##########
File path: sdks/python/tox.ini
##########
@@ -172,6 +172,17 @@ extras = test,gcp,interactive,aws
commands =
{toxinidir}/scripts/run_pytest.sh {envname} "{posargs}"
+[testenv:py38-cloudcoverage]
+# More recent versions of pytest-cov do not support pytest 4.4.0
+deps =
+ codecov
+ pytest-cov==2.9.0
+passenv = GIT_* BUILD_* ghprb* CHANGE_ID BRANCH_NAME JENKINS_* CODECOV_*
+extras = test,gcp,interactive,aws
+commands =
+ {toxinidir}/scripts/run_pytest.sh {envname} "{posargs}" --cov=apache_beam
+ codecov -t CODECOV_TOKEN
Review comment:
CODECOV_TOKEN is the default value, so no need to specify.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-672999018
run python precommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on a change in pull request #12257:
URL: https://github.com/apache/beam/pull/12257#discussion_r468946744
##########
File path: sdks/python/tox.ini
##########
@@ -172,6 +172,17 @@ extras = test,gcp,interactive,aws
commands =
{toxinidir}/scripts/run_pytest.sh {envname} "{posargs}"
+[testenv:py38-cloudcoverage]
+# More recent versions of pytest-cov do not support pytest 4.4.0
+deps =
+ codecov
+ pytest-cov==2.9.0
+passenv = GIT_* BUILD_* ghprb* CHANGE_ID BRANCH_NAME JENKINS_* CODECOV_*
+extras = test,gcp,interactive,aws
+commands =
+ {toxinidir}/scripts/run_pytest.sh {envname} "{posargs}" --cov=apache_beam
+ codecov -t CODECOV_TOKEN
Review comment:
What I mean is that by default it will look at the CODECOV_TOKEN environment variable. The current value being passed is not a UUID.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on a change in pull request #12257:
URL: https://github.com/apache/beam/pull/12257#discussion_r466608830
##########
File path: sdks/python/test-suites/tox/common.gradle
##########
@@ -27,6 +27,13 @@ test.dependsOn "testPy${pythonVersionSuffix}Cloud"
toxTask "testPy${pythonVersionSuffix}Cython", "py${pythonVersionSuffix}-cython"
test.dependsOn "testPy${pythonVersionSuffix}Cython"
+toxTask "testPy38CloudCoverage", "py38-cloudcoverage"
+test.dependsOn "testPy38CloudCoverage"
+
project.task("preCommitPy${pythonVersionSuffix}") {
- dependsOn = ["testPy${pythonVersionSuffix}Cloud", "testPy${pythonVersionSuffix}Cython"]
+ if (pythonVersionSuffix.equals('38')) {
Review comment:
Cool!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] saavannanavati commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
saavannanavati commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-658449426
An estimate for the extra time this will add to `py38-cloud`. These numbers are from testing on my machine.
- Without coverage, it runs in ~ `9m 40s`
- With coverage, it runs in ~ `23m 30s`
So it should be about an additional 14 min, worst case. Assuming Jenkins is faster, it will be less.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim merged pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim merged pull request #12257:
URL: https://github.com/apache/beam/pull/12257
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] saavannanavati commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
saavannanavati commented on a change in pull request #12257:
URL: https://github.com/apache/beam/pull/12257#discussion_r454738417
##########
File path: sdks/python/setup.py
##########
@@ -193,6 +193,8 @@ def get_version():
'requests_mock>=1.7,<2.0',
'tenacity>=5.0.2,<6.0',
'pytest>=4.4.0,<5.0',
+ # More recent versions of pytest-cov do not support pytest 4.4.0
+ 'pytest-cov==2.9.0',
Review comment:
Alternatively, this could be a dependency just for `py38-cloud`. That would make more sense since no other Tox environment requires coverage. Should I move it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-670131768
run python precommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-672414494
We can give this a try by adding this to `CommonJobProperties.groovy`:
```
credentialsBinding {
// other values
string("CODECOV_TOKEN", "beam-codecov-token")
}
```
I've set up `beam-codecov-token` credential on our Jenkins master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] udim commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit
Posted by GitBox <gi...@apache.org>.
udim commented on pull request #12257:
URL: https://github.com/apache/beam/pull/12257#issuecomment-670115023
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org