You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "chamikaramj (via GitHub)" <gi...@apache.org> on 2024/12/20 23:40:05 UTC
[I] [Bug]: beam_CleanupGCPResources is failing due to "The operation was canceled" [beam]
chamikaramj opened a new issue, #33435:
URL: https://github.com/apache/beam/issues/33435
### What happened?
https://github.com/apache/beam/actions/workflows/beam_CleanUpGCPResources.yml
For example,
https://github.com/apache/beam/actions/runs/12363928955/job/34506222289
```
> Task :beam-test-tools:removeStaleK8sWorkload
namespace "beam-performancetests-singlestoreio-12352184229" deleted
Error: The operation was canceled.
```
### Issue Priority
Priority: 1 (data loss / total loss of function)
### Issue Components
- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Infrastructure
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2610200082
Re-open this issue to track the task that makes clean up script working for such workload with non-empty finalizer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2634218569
@Abacn is this bug a release blocker for 2.63.0? It doesn't appear to be but i'd like to confirm before moving it off the milestone
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #33435: [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled"
URL: https://github.com/apache/beam/issues/33435
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #33435: [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled"
URL: https://github.com/apache/beam/issues/33435
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2557891271
The error log doesn't seem to be very useful unfortunately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2564030015
the command I used was `gcloud container clusters get-credentials io-datastores --zone us-central1-a --project apache-beam-testing`. It is given by GCP console. One can run in cloud shell or locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2563963394
From where would you run `kubectl` ? Probably good to document exact commands you used to resolve the error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2560328778
@Abacn assigning to you since you probably have the most context regarding this workflow. Feel free to forward if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2634295922
yeah it doesn't. The milestone was due to the issue get reopened
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2563954459
for future reference, it is due to a namespace (`beam-performancetests-singlestoreio-12352184229`) with finalizer stuck at teardown
To fix, follow https://cloud.google.com/knowledge/kb/deleted-namespace-remains-in-terminating-status-000004867
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: beam_CleanupGCPResources is failing due to "The operation was canceled" [beam]
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2557890482
cc: @Abacn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org