You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "chamikaramj (via GitHub)" <gi...@apache.org> on 2024/12/20 23:40:05 UTC

[I] [Bug]: beam_CleanupGCPResources is failing due to "The operation was canceled" [beam]

chamikaramj opened a new issue, #33435:
URL: https://github.com/apache/beam/issues/33435

   ### What happened?
   
   https://github.com/apache/beam/actions/workflows/beam_CleanUpGCPResources.yml
   
   For example,
   https://github.com/apache/beam/actions/runs/12363928955/job/34506222289
   
   ```
   > Task :beam-test-tools:removeStaleK8sWorkload
   namespace "beam-performancetests-singlestoreio-12352184229" deleted
   Error: The operation was canceled.
   ```
   
   ### Issue Priority
   
   Priority: 1 (data loss / total loss of function)
   
   ### Issue Components
   
   - [ ] Component: Python SDK
   - [ ] Component: Java SDK
   - [ ] Component: Go SDK
   - [ ] Component: Typescript SDK
   - [ ] Component: IO connector
   - [ ] Component: Beam YAML
   - [ ] Component: Beam examples
   - [ ] Component: Beam playground
   - [ ] Component: Beam katas
   - [ ] Component: Website
   - [ ] Component: Infrastructure
   - [ ] Component: Spark Runner
   - [ ] Component: Flink Runner
   - [ ] Component: Samza Runner
   - [ ] Component: Twister2 Runner
   - [ ] Component: Hazelcast Jet Runner
   - [ ] Component: Google Cloud Dataflow Runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2610200082

   Re-open this issue to track the task that makes clean up script working for such workload with non-empty finalizer


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]

Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2634218569

   @Abacn is this bug a release blocker for 2.63.0? It doesn't appear to be but i'd like to confirm before moving it off the milestone


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #33435: [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled"
URL: https://github.com/apache/beam/issues/33435


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]

Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #33435: [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled"
URL: https://github.com/apache/beam/issues/33435


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]

Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2557891271

   The error log doesn't seem to be very useful unfortunately.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2564030015

   the command I used was `gcloud container clusters get-credentials io-datastores --zone us-central1-a --project apache-beam-testing`. It is given by GCP console. One can run in cloud shell or locally.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]

Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2563963394

   From where would you run `kubectl` ? Probably good to document exact commands you used to resolve the error.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]

Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2560328778

   @Abacn assigning to you since you probably have the most context regarding this workflow. Feel free to forward if needed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2634295922

   yeah it doesn't. The milestone was due to the issue get reopened


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing with log "The operation was canceled" [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2563954459

   for future reference, it is due to a namespace (`beam-performancetests-singlestoreio-12352184229`) with finalizer stuck at teardown
   
   To fix, follow https://cloud.google.com/knowledge/kb/deleted-namespace-remains-in-terminating-status-000004867


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: beam_CleanupGCPResources is failing due to "The operation was canceled" [beam]

Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on issue #33435:
URL: https://github.com/apache/beam/issues/33435#issuecomment-2557890482

   cc: @Abacn 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org