You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "HansMarcus01 (via GitHub)" <gi...@apache.org> on 2026/08/20 03:53:37 UTC
[PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
HansMarcus01 opened a new pull request, #39826:
URL: https://github.com/apache/beam/pull/39826
# Overview
This pull request improves the lifecycle management of Pub/Sub resources in our integration tests to actively reduce GCP consumption. It replaces manual teardown methods with the robust TestPubsubContext manager and expands the automated garbage collector's scope to catch previously untracked test prefixes.
## Key Changes
* Integration Test Refactoring (`pubsub_integration_test.py`, `streaming_wordcount_it_test.py`, `streaming_wordcount_debugging_it_test.py`):
- * Replaced manual and error-prone teardown logic (`test_utils.cleanup_subscriptions` and `test_utils.cleanup_topics`) with the TestPubsubContext monitor.
- * Explicitly registered temporary test topics and subscriptions (including ordering topics) to ensure guaranteed, graceful teardown on test completion.
* **Stale Resource Cleaner Expansion** (`.test-infra/tools/stale_cleaner.py`):
- * Appended missing test prefixes to the cleanup tracking list (e.g.,` wc_topic_*`,` psit_*`, `leader_board_*`).
- * Ensures any orphaned resources that slip through the cracks during local or CI test failures are properly detected and deleted by the automated stale cleaner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem commented on PR #39826:
URL: https://github.com/apache/beam/pull/39826#issuecomment-5375133751
lgtm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem commented on code in PR #39826:
URL: https://github.com/apache/beam/pull/39826#discussion_r3833550658
##########
.test-infra/tools/stale_cleaner.py:
##########
@@ -1,441 +0,0 @@
-#!/usr/bin/env python
Review Comment:
accidentally deleted stale cleaner. we dont want this either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39826:
URL: https://github.com/apache/beam/pull/39826#discussion_r3837557327
##########
sdks/python/apache_beam/testing/README.md:
##########
@@ -0,0 +1,109 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Audit Log and Justification: Implementation of TestPubsubContext
+#### Date: 2026-08-20
+
+This document serves as a living log of resource leaks detected in our GCP environment due to failures or premature interruptions in CI/CD pipelines (such as Jenkins or GitHub Actions). The purpose of this log is to centralize evidence of orphaned components and provide technical and economic justification for implementing the TestPubsubContext lifecycle manager across all integration tests. The engineering team is encouraged to document any new leaks detected in future test suites in this file to maintain strict control over infrastructure consumption.
+
+## Detection Methodology (Audit Script)
+
+To identify optimization opportunities and prevent the accumulation of phantom resources in GCP, we developed an automated audit script (`auditar_backlog_wordcount.py`). This script connects to the `apache-beam-testing` project and actively filters for orphaned subscriptions based on the prefixes used by our test suites.
+```
+python
+
+from google.cloud import pubsub_v1
+from datetime import datetime, timezone, timedelta
+
+def audit_wordcount_subscriptions(project_id):
+ subscriber = pubsub_v1.SubscriberClient()
+ project_path = f"projects/{project_id}"
+
+ print(f"Searching for orphan subscriptions with 'resource_sub' prefixes in: {project_id}...\n")
+ print(f"{'Orphan Subscription Detected':<70} | {'Status'}")
+ print("-" * 90)
+ total_leaks = 0
+
+ # List subscriptions in the GCP project
+ for sub in subscriber.list_subscriptions(project=project_path):
+ sub_name = sub.name.split("/")[-1]
+
+ # Filter by those created by wordcount_it_test ('name of the resource')
+ if sub_name.startswith("resource_sub") or "resource_subscription" in sub_name:
+ total_leaks += 1
+ print(f"{sub_name:<70} | ACTIVE (ORPHAN)")
+
+ print("-" * 90)
+ print(f"Diagnosis: Detected {total_leaks} active orphan 'resource_sub' subscriptions in GCP.")
+
+if __name__ == "__main__":
+ audit_wordcount_subscriptions("apache-beam-testing")
+
+```
+
+## Evidence: Leaks in Pub/Sub Integration Tests (`psit_`)
+
+During the execution of the main integration test suite, the standard cleanup mechanism proved insufficient when tests failed or were abruptly aborted.
+
+### **Critical findings:**
Review Comment:
This dairy like log doesn't belong to Beam repo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem commented on PR #39826:
URL: https://github.com/apache/beam/pull/39826#issuecomment-5445699926
Hi Yi!
thanks for the feedback. I figured it might be useful to keep a record of
the investigation done to make this change. Do you now think that's
reasonable to keep?
On Sat, Aug 22, 2026 at 7:47 PM Yi Hu ***@***.***> wrote:
> ***@***.**** commented on this pull request.
> ------------------------------
>
> In sdks/python/apache_beam/testing/README.md
> <https://github.com/apache/beam/pull/39826#discussion_r3837557327>:
>
> > + total_leaks += 1
> + print(f"{sub_name:<70} | ACTIVE (ORPHAN)")
> +
> + print("-" * 90)
> + print(f"Diagnosis: Detected {total_leaks} active orphan 'resource_sub' subscriptions in GCP.")
> +
> +if __name__ == "__main__":
> + audit_wordcount_subscriptions("apache-beam-testing")
> +
> +```
> +
> +## Evidence: Leaks in Pub/Sub Integration Tests (`psit_`)
> +
> +During the execution of the main integration test suite, the standard cleanup mechanism proved insufficient when tests failed or were abruptly aborted.
> +
> +### **Critical findings:**
>
> This dairy like log doesn't belong to Beam repo
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/pull/39826?email_source=notifications&email_token=AAJ5Z3HAIOSBNEMWFPUWMTD5LJLMHA5CNFSNUABKM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UKJSXM2LFO4XTKMBQGE2TEOBUGQ2KM4TFMFZW63VMON2GC5DFL5RWQYLOM5S2KZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#pullrequestreview-5001528444>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAJ5Z3FYVDY2SK7D3K3WQZT5LJLMHAVCNFSNUABEKJSXA33TNF2G64TZHM2TAOJQGQZDINJ3JFZXG5LFHM2TCOJZGQ4TOOJYGSQXMAQ>
> .
> Triage notifications, keep track of coding agent tasks and review pull
> requests on the go with GitHub Mobile for iOS
> <https://github.com/notifications/mobile/ios/AAJ5Z3EI2OWMWPVDASME6WD5LJLMHA5CNFSNUABKM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UKJSXM2LFO4XTKMBQGE2TEOBUGQ2KM4TFMFZW63VMON2GC5DFL5RWQYLOM5S2KZLWMVXHJKTGN5XXIZLSL5UW64Y>
> and Android
> <https://github.com/notifications/mobile/android/AAJ5Z3CJJS4DSULDB4CWV3D5LJLMHA5CNFSNUABKM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UKJSXM2LFO4XTKMBQGE2TEOBUGQ2KM4TFMFZW63VMON2GC5DFL5RWQYLOM5S2KZLWMVXHJLTGN5XXIZLSL5QW4ZDSN5UWI>.
> Download it today!
> You are receiving this because you modified the open/close state.Message
> ID: ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39826:
URL: https://github.com/apache/beam/pull/39826#issuecomment-5351482685
Assigning reviewers:
R: @claudevdm for label python.
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem commented on PR #39826:
URL: https://github.com/apache/beam/pull/39826#issuecomment-5446149412
good point. I'll ask Hansel to remove.
On Thu, Aug 27, 2026 at 3:06 PM Yi Hu ***@***.***> wrote:
> *Abacn* left a comment (apache/beam#39826)
> <https://github.com/apache/beam/pull/39826#issuecomment-5445778129>
>
> Hi Yi! thanks for the feedback. I figured it might be useful to keep a
> record of the investigation done to make this change. Do you now think
> that's reasonable to keep?
>
> I'm not sure, at least not in Beam Python SDK, which is packaged and
> released to users. For generic infra doc
> https://github.com/apache/beam/blob/master/.test-infra/tools/README.md
> may be a good entry point
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/pull/39826?email_source=notifications&email_token=AAJ5Z3EZCCLYUN4K7TI2IUT5MCWH3A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNBUGU3TOOBRGI42M4TFMFZW63VMON2GC5DFL5RWQYLOM5S2KZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-5445778129>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAJ5Z3HU3J52GCTHEJ5VVYD5MCWH3AVCNFSNUABEKJSXA33TNF2G64TZHM2TAOJQGQZDINJ3JFZXG5LFHM2TCOJZGQ4TOOJYGSQXMAQ>
> .
> Triage notifications, keep track of coding agent tasks and review pull
> requests on the go with GitHub Mobile for iOS
> <https://github.com/notifications/mobile/ios/AAJ5Z3HXKI7PIQBH3OJN6V35MCWH3A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNBUGU3TOOBRGI42M4TFMFZW63VMON2GC5DFL5RWQYLOM5S2KZLWMVXHJKTGN5XXIZLSL5UW64Y>
> and Android
> <https://github.com/notifications/mobile/android/AAJ5Z3EG3M4QDBW4LZHOEZD5MCWH3A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNBUGU3TOOBRGI42M4TFMFZW63VMON2GC5DFL5RWQYLOM5S2KZLWMVXHJLTGN5XXIZLSL5QW4ZDSN5UWI>.
> Download it today!
> You are receiving this because you modified the open/close state.Message
> ID: ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem commented on code in PR #39826:
URL: https://github.com/apache/beam/pull/39826#discussion_r3831880783
##########
.test-infra/tools/stale_cleaner.py:
##########
@@ -421,6 +422,12 @@ def clean_pubsub_subscriptions():
# Restrict subscription cleanup to the NYC taxi prefix only.
prefixes = [
"taxirides-realtime_beam_",
+ "pubsub_io_performance",
+ "psit_sub_input",
+ "psit_sub_output",
+ "psit_sub_ordering",
+ "wc_subscription_input",
+ "wc_subscription_output",
Review Comment:
this change will have consequences for our production environment. let us make it in a different PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #39826:
URL: https://github.com/apache/beam/pull/39826#issuecomment-5365550098
## [Codecov](https://app.codecov.io/gh/apache/beam/pull/39826?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 57.19%. Comparing base ([`9451007`](https://app.codecov.io/gh/apache/beam/commit/94510075efe1ea52c9f2a3dcc42ab8fc0a54b9e2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`98a4019`](https://app.codecov.io/gh/apache/beam/commit/98a4019ef9d1a3b627acbd277d0c693108c676ac?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 4 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #39826 +/- ##
============================================
+ Coverage 57.10% 57.19% +0.08%
Complexity 3643 3643
============================================
Files 1191 1193 +2
Lines 192542 192979 +437
Branches 3816 3816
============================================
+ Hits 109957 110378 +421
- Misses 79027 79043 +16
Partials 3558 3558
```
| [Flag](https://app.codecov.io/gh/apache/beam/pull/39826/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [python](https://app.codecov.io/gh/apache/beam/pull/39826/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `79.93% <ø> (+0.08%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/beam/pull/39826?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem merged PR #39826:
URL: https://github.com/apache/beam/pull/39826
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #39826:
URL: https://github.com/apache/beam/pull/39826#issuecomment-5383861924
It breaks Python PostCommit: https://github.com/apache/beam/runs/97127657945
> self.pubsub_monitor_register_topic(self.input_topic.name)
E AttributeError: 'PubSubIntegrationTest' object has no attribute 'pubsub_monitor_register_topic'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GSoC-273] Feat: Integrate TestPubsubContext to prevent Pub/Sub resource leaks and expand stale cleaner scope [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #39826:
URL: https://github.com/apache/beam/pull/39826#issuecomment-5445778129
> Hi Yi! thanks for the feedback. I figured it might be useful to keep a record of the investigation done to make this change. Do you now think that's reasonable to keep?
I'm not sure, at least not in Beam Python SDK, which is packaged and released to users. For generic infra doc https://github.com/apache/beam/blob/master/.test-infra/tools/README.md may be a good entry point
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org