You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/02/20 06:11:52 UTC
[GitHub] [beam] youngoli opened a new pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
youngoli opened a new pull request #14030:
URL: https://github.com/apache/beam/pull/14030
Pretty straightforward. The only weird part is that I couldn't figure out if it's possible to add a "flag" of sorts to a task. So in this case, I needed to make a separate ULR ValidatesRunner task just to add the --jenkins flag to the script call.
As a side note, it may be time soon to remove the previous integration test code.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [x] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
--- | --- | --- | --- | --- | --- | ---
Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam
.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
XLang | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website | Whitespace | Typescript
--- | --- | --- | --- | --- | --- | ---
Non-portable | [](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) <br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/)
Portable | --- | [](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | --- | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli removed a comment on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli removed a comment on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-785538375
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-786432368
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli merged pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli merged pull request #14030:
URL: https://github.com/apache/beam/pull/14030
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-786941274
Run Go Precommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
lostluck commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-783562630
Run Go PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-782569065
R: @lostluck
(Still gotta see if the PreCommit still passes though)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-788515556
Alright, code is ready for review @lostluck
I already tested the new portable precommit manually several times, it just got lost in the ~5 days worth of testing comments I deleted to declutter this thread. If you'd like to double check, just do "Run Seed Job" followed by "Run GoPortable Precommit" afterwards. (GoPortable needs to be one word, because splitting it up caused problems with the PrecommitBuilder script due to having a space).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-785521945
Run Go PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
lostluck commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-783673345
To separate out to a new pre/post commit we would need to add a new jenkins files here: https://github.com/apache/beam/tree/master/.test-infra/jenkins (search for _Go to find the various Go related files).
Given one is a "set of unit tests" and the XLang integration tests are fully fledged runner tests, we lose signal if we just combine them into a single jenkins target.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-786935712
Run GoPortable Precommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-786936571
Run GoPortable Precommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
lostluck commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-783657109
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli removed a comment on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli removed a comment on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-785521945
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-786933930
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli removed a comment on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli removed a comment on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-786935712
Run GoPortable Precommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
lostluck commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-783563405
Retrying since it seems to be a bit flaky. We may want to split it into a 2nd target as it is more heavy weight. The current pre-commit nicely fits into the "non-portable" slot, while this could be the "portable" slot as it validates against the python portable runner.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-785550881
Run Go Precommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-785546264
Run Go Precommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-787691125
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-786441063
Run Go Precommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-787682953
Run Go PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-785538375
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] youngoli commented on pull request #14030: [BEAM-11415] Adding ValidatesRunner w/ Python ULR tests to Go Precommit
Posted by GitBox <gi...@apache.org>.
youngoli commented on pull request #14030:
URL: https://github.com/apache/beam/pull/14030#issuecomment-787634616
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org