You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/11/05 03:36:32 UTC
[I] The PostCommit Python Arm job is flaky [beam]
github-actions[bot] opened a new issue, #30760:
URL: https://github.com/apache/beam/issues/30760
The PostCommit Python Arm is failing over 50% of the time
Please visit https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Arm.yml?query=is%3Afailure+branch%3Amaster to see the logs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2079165366
Great, thanks @volatilemolotov
Looks like we're still flaky - https://github.com/apache/beam/actions/runs/8843342204/job/24283441647 - but that's an improvement and it looks like a test flake instead of infra
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2077505290
@volatilemolotov could you put up a PR to make that change? Definitely seems like it is getting further.
@svetakvsundhar do you know what scope is missing? Given the normal postcommit python isn't failing, it might just be an issue with your service account specifically?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3196345820
@damccorm @mohamedawnallah
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3383320382
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2787360999
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3204084508
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2077561041
Thanks - merged, lets see what the result on master is
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2082757026
I think that's wrong - https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Arm.yml?query=branch%3Amaster+event%3Aschedule
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-5382734302
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3193919128
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3554725396
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3612755051
Yi pointed out that this is due to the arm container not having up to date dependencies. We should be able to just update the requirements files; running https://github.com/apache/beam/actions/runs/19933947366 to do this, and then I'll just cherry-pick the kms updates to the release branch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2075397421
I actually can't find a single green run since this test suite was created (back in September)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3101157805
Service account key updated but workflow still red
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3086131109
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2457512165
This is failing because of Dataflow issues, not because of Beam. Dataflow is requesting arm machines in regions where there are none, failing the job. I reopened an internal bug (id 352725422)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3573446534
The job started to fail a few days ago. The error seems to be a space limit issue on the runner.
```
Unhandled exception. System.IO.IOException: No space left on device : '/home/runner/actions-runner/cached/_diag/Worker_20251124-110158-utc.log'
at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.Diagnostics.TextWriterTraceListener.Flush()
at System.Diagnostics.TraceSource.Flush()
at GitHub.Runner.Common.Tracing.Dispose(Boolean disposing)
at GitHub.Runner.Common.Tracing.Dispose()
at GitHub.Runner.Common.TraceManager.Dispose(Boolean disposing)
at GitHub.Runner.Common.TraceManager.Dispose()
at GitHub.Runner.Common.HostContext.Dispose(Boolean disposing)
at GitHub.Runner.Common.HostContext.Dispose()
at GitHub.Runner.Worker.Program.Main(String[] args)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2075506680
This was working last month - https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Arm.yml?query=is%3Asuccess+branch%3Amaster+event%3Aschedule
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3292541870
https://github.com/apache/beam/pull/36140 should fix this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3499483992
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3573358166
Looks like most failures were in Python 3.10 workflow again.
I see the following quota error in the log:
```
apitools.base.py.exceptions.HttpForbiddenError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/leader_board_it_dataset17636003405f47ce/tables?alt=json>: response: <***'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 20 Nov 2025 01:02:59 GMT', 'server': 'ESF', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '403', 'content-length': '534', '-content-encoding': 'gzip'***>, content <***
2025-11-20T01:03:06.0930567Z "error": ***
2025-11-20T01:03:06.0930822Z "code": 403,
2025-11-20T01:03:06.0931590Z "message": "Exceeded rate limits: too many table update operations for this table. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas",
2025-11-20T01:03:06.0932301Z "errors": [
2025-11-20T01:03:06.0932607Z ***
2025-11-20T01:03:06.0933254Z "message": "Exceeded rate limits: too many table update operations for this table. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas",
2025-11-20T01:03:06.0933972Z "domain": "usageLimits",
2025-11-20T01:03:06.0934285Z "reason": "rateLimitExceeded"
2025-11-20T01:03:06.0934590Z ***
2025-11-20T01:03:06.0934808Z ],
2025-11-20T01:03:06.0935055Z "status": "PERMISSION_DENIED"
2025-11-20T01:03:06.0935343Z ***
2025-11-20T01:03:06.0935573Z ***
2025-11-20T01:03:06.0935784Z >
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2610961376
The following tests appear to be consistently failing:
- apache_beam/ml/inference/sklearn_inference_it_test.py::SklearnInference::test_sklearn_mnist_classification
- apache_beam/ml/inference/sklearn_inference_it_test.py::SklearnInference::test_sklearn_mnist_classification_large_model
- apache_beam/ml/inference/sklearn_inference_it_test.py::SklearnInference::test_sklearn_regression
Errors are smilar:
`ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject`
@tvalentyn @damccorm any ideas?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2077553101
Sure, here it is
https://github.com/apache/beam/pull/31102
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2075471497
You may be right, thanks for correction, @ahmedabu98
```
2024-04-24T12:03:53.0963029Z Please verify that you have permissions to write to the parent directory..
2024-04-24T12:03:53.0964903Z The configuration directory may not be writable. To learn more, see https://cloud.google.com/sdk/docs/configurations#creating_a_configuration
2024-04-24T12:03:53.0968080Z ERROR: (gcloud.auth.docker-helper) Could not create directory [/var/lib/kubelet/pods/573a1844-124b-4e12-bb0f-0325d0f3c3aa/volumes/kubernetes.io~empty-dir/gcloud]: Permission denied.
2024-04-24T12:03:53.0969612Z
2024-04-24T12:03:53.0970063Z Please verify that you have permissions to write to the parent directory.
2024-04-24T12:03:53.3953756Z #29 pushing layers 1.4s done
2024-04-24T12:03:53.3956208Z #29 ERROR: failed to push us.gcr.io/apache-beam-testing/github-actions/beam_python3.8_sdk:2.57.0-SNAPSHOT: error getting credentials - err: exit status 1, out: ``
2024-04-24T12:03:53.8953735Z ------
```
cc: @damccorm - do you remember if this suite never worked or the above error is an artifact of GHA migration?
We can reclassify this as part part of ARM backlog work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "akashorabek (via GitHub)" <gi...@apache.org>.
akashorabek commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2618579486
I tried retraining the model, but the error` ValueError: numpy.dtype size changed still appears.` Most likely, there is an incompatibility in `scikit-learn` with the new `NumPy `version specifically for `Python 3.9` and `3.10`.
I found a [similar issue here](https://github.com/openai/whisper/discussions/2334), and it seems that downgrading `NumPy` was the only solution that worked. I tried it locally, and it fixed the problem.
Can we revert to an older NumPy version for Python 3.9 and 3.10? I see @Abacn already created [draft PR](https://github.com/apache/beam/pull/33774) for this.
@damccorm @liferoad
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2075510792
Ahh my apologies, I was looking at it through a `is:failure` filter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-4036210787
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3311430974
FIxed by https://github.com/apache/beam/pull/36140
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3292270064
Culprit: https://github.com/apache/beam/pull/36027
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2075508010
Looks like it went flaky then permared around then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "svetakvsundhar (via GitHub)" <gi...@apache.org>.
svetakvsundhar commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2077650329
> @svetakvsundhar do you know what scope is missing? Given the normal postcommit python isn't failing, it might just be an issue with your service account specifically?
+1, it could be a service account specific issue. I'd want to see a couple of more runs of this to see if it's actually an issue. If so, a thought might be to add `["https://www.googleapis.com/auth/cloud-platform"]` as a scope manually in the test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2078975189
https://github.com/apache/beam/actions/runs/8840477636
it works now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3617390267
`#30 ERROR: mount callback failed on /tmp/containerd-mount3865708938: mount callback failed on /tmp/containerd-mount1734725722: failed to write compressed diff: failed to create diff tar stream: failed to copy: /tmp/containerd-mount1734725722/usr/local/gcloud/google-cloud-sdk/lib/googlecloudsdk/generated_clients/apis/alloydb/v1alpha/__pycache__/alloydb_v1alpha_messages.cpython-313.pyc: write /var/lib/buildkit/runc-overlayfs/content/ingest/2cfc7caa86c3a03a0325679044fa0df14a503106ea68c68c2d09d12cd1e2b5bf/data: no space left on device`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-4809669888
The workflow is green again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-4805621613
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-4039677030
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-5549098924
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-5831748532
The workflow is currently failing consistently on two integration tests because the CI service account is missing a couple of GCP IAM permissions:
- VertexAIModelMonitoringV2IntegrationTest fails because the account does not have aiplatform.models.upload
- RecommendationAIIT.test_predict passes, but its teardown errors because the account cannot run automlrecommendations.catalogItems.list
Log:
google.api_core.exceptions.PermissionDenied: 403
Permission aiplatform.models.upload denied on resource
//aiplatform.googleapis.com/projects/apache-beam-testing/locations/us-central1
cc: @damccorm @Abacn @shunping
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-5833659349
ack, fixing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-4533750108
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3111939891
Fixed by #35658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "mohamedawnallah (via GitHub)" <gi...@apache.org>.
mohamedawnallah commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3196459498
> Failing after [#35577](https://github.com/apache/beam/pull/35577)
> @damccorm @mohamedawnallah
I believe the error that milvus extra dependency not added properly. I submitted a patch in the PR to properly add milvus as extra dependency (the PR mainly for fixing docker issues for milvus) https://github.com/apache/beam/pull/35816#issuecomment-3193646680. Please let me know I can isolate this into a seperate PR or we can go ahead and merge this
@damccorm, @liferoad
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3195595052
Failing after #35577
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3282712171
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2367952573
Fixed by https://github.com/apache/beam/pull/32530
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2456161744
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2611376939
where are the instructions to retrain the model? The model training should be part of tests to avoid maintaining this in the future if the training is cheap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2619222687
I think we should either disable this test for Py39 and Py310 or create a Python venv with the numpy 1.xx to run it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3414101914
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-5790413827
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2071162255
@tvalentyn do we have a good owner for this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2082747886
Permared now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "volatilemolotov (via GitHub)" <gi...@apache.org>.
volatilemolotov commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2077353830
So by removing
https://github.com/apache/beam/blob/master/.github/workflows/beam_PostCommit_Python_Arm.yml#L113
I get the test to move along but its still failing on my fork due to some permission with the Healthcare api.
Oauth scope is wrong or something:
https://github.com/volatilemolotov/beam/actions/runs/8820257015/job/24213449686#step:13:13113
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2366910879
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2610964660
Python Postcommits are broken for the same reason: https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python.yml
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-2611039811
I'm guessing this is from https://github.com/apache/beam/pull/33658/files where we updated the numpy version to 2.x. See https://stackoverflow.com/questions/40845304/runtimewarning-numpy-dtype-size-changed-may-indicate-binary-incompatibility
It is coming from unpickling the model here - https://github.com/apache/beam/blob/fa0152b4f44df8b39a4fd9c8a609aa257afd5bd7/sdks/python/apache_beam/ml/inference/sklearn_inference_it_test.py#L62 - we probably just need to retrain/upload the model
cc/ @liferoad @Amar3tto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3087440531
Service account key is expired, need to update
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3608645750
`File "/usr/local/lib/python3.13/site-packages/apache_beam/transforms/util.py", line 517, in get_secret_bytes
2025-12-03T19:59:08.3879496Z raise RuntimeError(
2025-12-03T19:59:08.3879833Z f'Failed to retrieve or create secret bytes for secret '
2025-12-03T19:59:08.3880273Z f'***self._secret_version_name*** with exception ***e***')
2025-12-03T19:59:08.3881786Z RuntimeError: Failed to retrieve or create secret bytes for secret HsmGeneratedSecret_1203_185821_py313 with exception Failed to generate DEK with exception cannot import name 'kms' from 'google.cloud' (unknown location) [while running 'Start 1/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Group by encrypted key/ParDo(_EncryptMessage)-ptransform-46']
2025-12-03T19:59:08.3883012Z
2025-12-03T19:59:08.3883282Z Worker ID: beamapp-runner-1203190336-12031103-afyz-harness-k5rp,
2025-12-03T19:59:08.3883577Z
2025-12-03T19:59:08.3883732Z Root cause: Traceback (most recent call last):
2025-12-03T19:59:08.3884320Z File "/usr/local/lib/python3.13/site-packages/apache_beam/transforms/util.py", line 539, in generate_dek
2025-12-03T19:59:08.3884868Z from google.cloud import kms
2025-12-03T19:59:08.3885287Z ImportError: cannot import name 'kms' from 'google.cloud' (unknown location)
2025-12-03T19:59:08.3885609Z
2025-12-03T19:59:08.3885819Z During handling of the above exception, another exception occurred:
2025-12-03T19:59:08.3886116Z
2025-12-03T19:59:08.3886248Z Traceback (most recent call last):
2025-12-03T19:59:08.3886810Z File "/usr/local/lib/python3.13/site-packages/apache_beam/transforms/util.py", line 494, in get_secret_bytes
2025-12-03T19:59:08.3889606Z new_key = self.generate_dek()
2025-12-03T19:59:08.3890174Z File "/usr/local/lib/python3.13/site-packages/apache_beam/transforms/util.py", line 567, in generate_dek
2025-12-03T19:59:08.3890858Z raise RuntimeError(f'Failed to generate DEK with exception ***e***')
2025-12-03T19:59:08.3891518Z RuntimeError: Failed to generate DEK with exception cannot import name 'kms' from 'google.cloud' (unknown location)
2025-12-03T19:59:08.3891958Z `
Not sure about the root of this error
CC @damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3424787932
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-3368641610
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #30760: The PostCommit Python Arm job is flaky
URL: https://github.com/apache/beam/issues/30760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-4614928860
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python Arm job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30760:
URL: https://github.com/apache/beam/issues/30760#issuecomment-5393893508
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org