You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/07/17 00:08:24 UTC
[GitHub] [beam] aaltay opened a new pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
aaltay opened a new pull request #12288:
URL: https://github.com/apache/beam/pull/12288
This is a follow up to https://github.com/apache/beam/pull/11744/files. Newly added pipeline option breaks these tests because `contains_inanyorder` requires all items to match the matcher list. Changed this to `has_items` which reverses the relationship and require each matcher to match an item.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
--- | --- | --- | --- | --- | --- | ---
Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
XLang | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660255342
@TobKed
- I was able to re-run the workflow, but they did not remove the failed runs for the github UI. It looked like a mix of failed/pending runs. I re-pushed to restart all tests.
- Could you add notes to the wiki (https://cwiki.apache.org/confluence/display/BEAM/Python+Tips) about troubleshooting/restarting these pipelines?
- Should we also file an issue with github? Could this be a quota issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660384313
> @ibzib - May I ignore the failure and merge this change? I think this change is good (passing other tests). And it would be good to merge this as it address another flakiness.
Yes, I think the failure is fine to ignore.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660283557
Run Portable_Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660283797
Run Python2_PVR_Flink PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660322544
Re-pushed to pick up (https://github.com/apache/beam/pull/12303)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660283482
Restarting tests, since this failures are not related to the specific flake this PR is fixing. And triaged the issues in the previous comment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660319748
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TobKed commented on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
TobKed commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-659893430
"This check failed" is really weird, I never seen this before. Looks like GitHub Actions issue.
I found related issue: https://github.com/actions/virtual-environments/issues/841
@aaltay do you have permission to rerun workflow? If not please push some changes e.g. rebase. It should re-trigger run and hopefully we will get more info.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TobKed commented on pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
TobKed commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-665684372
I created PR with documentation for GItHub Actions CI https://github.com/apache/beam/pull/12405
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-659768145
@TobKed - question related to failed "Build python wheels / Build wheels on macos-latest" (https://github.com/apache/beam/pull/12288/checks?check_run_id=879924787) I do not see any logs other that "This check failed" message. How do we debug this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660332983
Now all the external transform tests in Python2_PVR_Flink are timing out instead. I'm thinking about either increasing the timeouts to these tests, or removing them since they might be redundant (XVR_Flink has its own postcommit already).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660372940
@ibzib - May I ignore the failure and merge this change? I think this change is good (passing other tests). And it would be good to merge this as it address another flakiness.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-661443823
> @aaltay
>
> > * Could you add notes to the wiki (https://cwiki.apache.org/confluence/display/BEAM/Python+Tips) about troubleshooting/restarting these pipelines?
>
> Sure. What is the process to do it?
Thank you.
There is really not much of a process. Anyone can signup and edit the wiki. After you signup/login, you can go to the page I linked, edit and add a small section about these tests. Does it make sense?
>
> > * Should we also file an issue with github? Could this be a quota issue?
>
> I see it is still happens: https://github.com/apache/beam/actions/runs/175737977
> I think it is not related to the quota, at least not from project side. Seems like GitHub have some issues with infrastructure.
> There is issue created: [actions/virtual-environments#841](https://github.com/actions/virtual-environments/issues/841) . I will observe it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660297740
Run Python2_PVR_Flink PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660321940
> @chamikaramj @ihji test_expand_kafka_read and test_external_transform are failing on Flink. Link to failing tests: (https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Phrase/304/) -- same error as before. This looks like an issue, I will find the JIRA issue or file a new one.
I think this is fixed by (https://github.com/apache/beam/pull/12303). (@ibzib to confirm.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660283665
Run Python2_PVR_Flink PreCommit"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay removed a comment on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay removed a comment on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660283665
Run Python2_PVR_Flink PreCommit"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay merged pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay merged pull request #12288:
URL: https://github.com/apache/beam/pull/12288
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tysonjh commented on pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
tysonjh commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-661488284
> @tysonjh - Gradle is failing with "java.io.IOException: No space left on device" errors. Link to failing tests: https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Commit/12004/
This was fixed AFAICT by running some Jenkins jobs to clean up the local storage. Sorry about the delayed response.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-660283172
@tysonjh - Gradle is failing with "java.io.IOException: No space left on device" errors. Link to failing tests: https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Commit/12004/
@chamikaramj @ihji test_expand_kafka_read and test_external_transform are failing on Flink. Link to failing tests: https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Commit/5854/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TobKed commented on pull request #12288: [BEAM-10474] Relax to matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
TobKed commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-659894132
I found information:
> The request: 1087213 was abandoned due to an infrastructure failure. Notification of assignment to a runner was never received.
It is visible in **Annotations** section here:
https://github.com/apache/beam/actions/runs/172037362
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TobKed commented on pull request #12288: [BEAM-10474] Relax matchers to match display data from the specific tests, not from all options.
Posted by GitBox <gi...@apache.org>.
TobKed commented on pull request #12288:
URL: https://github.com/apache/beam/pull/12288#issuecomment-661138174
@aaltay
> - Could you add notes to the wiki (https://cwiki.apache.org/confluence/display/BEAM/Python+Tips) about troubleshooting/restarting these pipelines?
Sure. What is the process to do it?
> - Should we also file an issue with github? Could this be a quota issue?
I see it is still happens: https://github.com/apache/beam/actions/runs/175737977
I think it is not related to the quota, at least not from project side. Seems like GitHub have some issues with infrastructure.
There is issue created: https://github.com/actions/virtual-environments/issues/841 . I will observe it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org