You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "claudevdm (via GitHub)" <gi...@apache.org> on 2025/03/04 15:07:04 UTC
[PR] Fix test [beam]
claudevdm opened a new pull request, #34162:
URL: https://github.com/apache/beam/pull/34162
I intentionally made this test fail to be sure they are actually running, but forgot to revert the failure force.
This should fix Python PostCommit, also triggered a run.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #34162:
URL: https://github.com/apache/beam/pull/34162#issuecomment-2701583577
Will merge once checks complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "claudevdm (via GitHub)" <gi...@apache.org>.
claudevdm commented on PR #34162:
URL: https://github.com/apache/beam/pull/34162#issuecomment-2703816929
Looks like the tests are all green @damccorm .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "claudevdm (via GitHub)" <gi...@apache.org>.
claudevdm commented on PR #34162:
URL: https://github.com/apache/beam/pull/34162#issuecomment-2699605046
Postcommits were still failing after changing the runner to highmem22 due to apache_beam.utils.subprocess_server:subprocess_server.py:213 java.io.IOException: Insufficient number of network buffers: required 2, but only 0 available. The total number of network buffers is currently set to 2048 of 32768 bytes each. You can increase this number by setting the configuration keys 'taskmanager.memory.network.fraction', 'taskmanager.memory.network.min', and 'taskmanager.memory.network.max'
Adding flink config fixed this. I ran just the portable test suite here https://github.com/apache/beam/actions/runs/13666811744 and it succeeded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #34162:
URL: https://github.com/apache/beam/pull/34162#discussion_r1981846974
##########
sdks/python/test-suites/portable/common.gradle:
##########
@@ -385,18 +385,19 @@ project.tasks.register("postCommitPy${pythonVersionSuffix}IT") {
doLast {
def tests = [
"apache_beam/io/gcp/bigquery_read_it_test.py",
- "apache_beam/io/external/xlang_jdbcio_it_test.py",
Review Comment:
Ok, SGTM. That is quite memory intensive, I'd be interested in understanding why eventually, but this does not need to be a priority.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn merged PR #34162:
URL: https://github.com/apache/beam/pull/34162
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "claudevdm (via GitHub)" <gi...@apache.org>.
claudevdm commented on PR #34162:
URL: https://github.com/apache/beam/pull/34162#issuecomment-2697964000
R: @Abacn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "claudevdm (via GitHub)" <gi...@apache.org>.
claudevdm commented on code in PR #34162:
URL: https://github.com/apache/beam/pull/34162#discussion_r1981825982
##########
sdks/python/test-suites/portable/common.gradle:
##########
@@ -385,18 +385,19 @@ project.tasks.register("postCommitPy${pythonVersionSuffix}IT") {
doLast {
def tests = [
"apache_beam/io/gcp/bigquery_read_it_test.py",
- "apache_beam/io/external/xlang_jdbcio_it_test.py",
Review Comment:
I chatted to Yi, and decided to move the xlang jdbcio test to the xlang direct suite.
Even with the conf dir and the super high memory runner, one of the four postcommits OOM'd. It ends up spiking to 100+gb memory when using flink runner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34162:
URL: https://github.com/apache/beam/pull/34162#issuecomment-2697968089
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #34162:
URL: https://github.com/apache/beam/pull/34162#issuecomment-2701843150
> Looks like all the tests are passing, except Xlang Gcp Dataflow is stuck with the same symptom as [#33885 (comment)](https://github.com/apache/beam/pull/33885#issuecomment-2654266995)
>
> Last time I did a sync + rebase on master to get it unstuck.
I'll cancel/restart it to see if we can get green
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "claudevdm (via GitHub)" <gi...@apache.org>.
claudevdm commented on PR #34162:
URL: https://github.com/apache/beam/pull/34162#issuecomment-2699605500
Triggered a full postcommit at https://github.com/apache/beam/actions/runs/13667238620.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #34162:
URL: https://github.com/apache/beam/pull/34162#discussion_r1981815970
##########
sdks/python/test-suites/portable/common.gradle:
##########
@@ -385,18 +385,19 @@ project.tasks.register("postCommitPy${pythonVersionSuffix}IT") {
doLast {
def tests = [
"apache_beam/io/gcp/bigquery_read_it_test.py",
- "apache_beam/io/external/xlang_jdbcio_it_test.py",
Review Comment:
Do we still need to exclude this with the flink_conf_dir setting now set?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "claudevdm (via GitHub)" <gi...@apache.org>.
claudevdm commented on PR #34162:
URL: https://github.com/apache/beam/pull/34162#issuecomment-2701835449
Looks like all the tests are passing, except Xlang Gcp Dataflow is stuck with the same symptom as https://github.com/apache/beam/pull/33885#issuecomment-2654266995
Last time I did a sync + rebase on master to get it unstuck.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix test [beam]
Posted by "claudevdm (via GitHub)" <gi...@apache.org>.
claudevdm commented on PR #34162:
URL: https://github.com/apache/beam/pull/34162#issuecomment-2701847335
> > Looks like all the tests are passing, except Xlang Gcp Dataflow is stuck with the same symptom as [#33885 (comment)](https://github.com/apache/beam/pull/33885#issuecomment-2654266995)
> > Last time I did a sync + rebase on master to get it unstuck.
>
> I'll cancel/restart it to see if we can get green
Thanks. Not sure if it will resolve the issue though, last time if I recall you did a restart and it was still stuck, untill I rebased + force pushed. Will keep an eye on the progress to see if it gets stuck again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org