You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "derrickaw (via GitHub)" <gi...@apache.org> on 2025/12/17 18:28:58 UTC
[PR] move pipelineGraph import [beam]
derrickaw opened a new pull request, #37134:
URL: https://github.com/apache/beam/pull/37134
1. Current code has circular import - causing failures with https://github.com/apache/beam/actions/runs/20287544814
2. Move import to _get_pipeline_graph to allow module to fully load before importing.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] move pipelineGraph import [beam]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on PR #37134:
URL: https://github.com/apache/beam/pull/37134#issuecomment-3666632849
## Summary of Changes
Hello @derrickaw, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request addresses a critical circular import dependency in the interactive runner's recording manager. By deferring the import of the `PipelineGraph` class until it is actually needed within a specific method, the change prevents module loading issues and ensures the stability of the build process.
### Highlights
* **Circular Import Fix**: Resolved a circular import issue involving `PipelineGraph` within `recording_manager.py` that was causing build failures in GitHub Actions.
* **Deferred Import**: The `PipelineGraph` import has been moved from the module's top level into the `_get_pipeline_graph` function, ensuring the module fully loads before the import is attempted.
<details>
<summary><b>Using Gemini Code Assist</b></summary>
<br>
The full guide for Gemini Code Assist can be found on our [documentation page](https://developers.google.com/gemini-code-assist/docs/review-github-code), here are some quick tips.
<b>Invoking Gemini</b>
You can request assistance from Gemini at any point by creating a comment using either `/gemini <command>` or `@gemini-code-assist <command>`. Below is a summary of the supported commands on the current page.
Feature | Command | Description
--- | --- | ---
Code Review | `/gemini review` | Performs a code review for the current pull request in its current state.
Pull Request Summary | `/gemini summary` | Provides a summary of the current pull request in its current state.
Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help | `/gemini help` | Displays a list of available commands.
<b>Customization</b>
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a `.gemini/` folder in the base of the repository. Detailed instructions can be found [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github).
<b>Limitations & Feedback</b>
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up [here](https://google.qualtrics.com/jfe/form/SV_2cyuGuTWsEw84yG).
<b>You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the [Gemini Code Assist IDE Extension](https://cloud.google.com/products/gemini/code-assist).</b>
</details>
[^1]: Review the [Privacy Notices](https://policies.google.com/privacy), [Generative AI Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy), [Terms of Service](https://policies.google.com/terms), and learn how to configure Gemini Code Assist in GitHub [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Gemini can make mistakes, so double check it and [use code with caution](https://support.google.com/legal/answer/13505487).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix circular PipelineGraph import [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on PR #37134:
URL: https://github.com/apache/beam/pull/37134#issuecomment-3667992044
I took one example from the error and tried on playground, but could not see the error in the UI. Maybe the website and the test here are going through different code paths?
```
[2025-12-18 00:49:52,176] ERROR [root._verify_examples:220] Example: /runner/_work/beam/beam/./sdks/python/apache_beam/examples/snippets/transforms/aggregation/groupby_two_exprs.py has execution error: error: exit status 1
output: /usr/local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.19) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
warnings.warn(message, FutureWarning)
Traceback (most recent call last):
File "/opt/playground/backend/executable_files/e8c3718c-16af-446a-96c7-56d5c77975be/e8c3718c-16af-446a-96c7-56d5c77975be.py", line 58, in <module>
groupby_two_exprs()
File "/opt/playground/backend/executable_files/e8c3718c-16af-446a-96c7-56d5c77975be/e8c3718c-16af-446a-96c7-56d5c77975be.py", line 49, in groupby_two_exprs
from apache_beam.runners.interactive.display import pipeline_graph
File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/interactive/display/pipeline_graph.py", line 37, in <module>
from apache_beam.runners.interactive import interactive_environment as ie
File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/interactive/interactive_environment.py", line 41, in <module>
from apache_beam.runners.interactive.recording_manager import RecordingManager
File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/interactive/recording_manager.py", line 44, in <module>
from apache_beam.runners.interactive.display.pipeline_graph import PipelineGraph
ImportError: cannot import name 'PipelineGraph' from partially initialized module 'apache_beam.runners.interactive.display.pipeline_graph' (most likely due to a circular import) (/usr/local/lib/python3.10/site-packages/apache_beam/runners/interactive/display/pipeline_graph.py)
```
https://play.beam.apache.org/?path=SDK_PYTHON_GroupByTwoExprs&sdk=python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] move pipelineGraph import [beam]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #37134:
URL: https://github.com/apache/beam/pull/37134#issuecomment-3666685023
## [Codecov](https://app.codecov.io/gh/apache/beam/pull/37134?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 36.31%. Comparing base ([`d7cd81a`](https://app.codecov.io/gh/apache/beam/commit/d7cd81a1649bc665581951d2330c4b8acd19ed72?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`9907c25`](https://app.codecov.io/gh/apache/beam/commit/9907c25ecbc5925cbde75bc4440cf258315a5217?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/beam/pull/37134?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache\_beam/runners/interactive/recording\_manager.py](https://app.codecov.io/gh/apache/beam/pull/37134?src=pr&el=tree&filepath=sdks%2Fpython%2Fapache_beam%2Frunners%2Finteractive%2Frecording_manager.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/beam/pull/37134?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #37134 +/- ##
=============================================
- Coverage 55.13% 36.31% -18.82%
Complexity 1676 1676
=============================================
Files 1067 1067
Lines 167119 167119
Branches 1208 1208
=============================================
- Hits 92136 60692 -31444
- Misses 72803 104247 +31444
Partials 2180 2180
```
| [Flag](https://app.codecov.io/gh/apache/beam/pull/37134/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [python](https://app.codecov.io/gh/apache/beam/pull/37134/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `40.61% <0.00%> (-40.16%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/beam/pull/37134?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix circular PipelineGraph import [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping merged PR #37134:
URL: https://github.com/apache/beam/pull/37134
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix circular PipelineGraph import [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on PR #37134:
URL: https://github.com/apache/beam/pull/37134#issuecomment-3667978860
> Let's monitor the test run tonight: https://github.com/apache/beam/actions/workflows/beam_Playground_CI_Nightly.yml
Looks like this workflow is always running against a released version.
https://github.com/apache/beam/actions/runs/20321904216/workflow#L86
So it will be there until we have a new release...
cc'ed @damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix circular PipelineGraph import [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on PR #37134:
URL: https://github.com/apache/beam/pull/37134#issuecomment-3671481937
@derrickaw: looks like there is another circular import problem of `RecordingManager`, introduced in #36853.
```
Traceback (most recent call last):
File "./sdks/python/playpen/import_loop", line 3, in <module>
from apache_beam.runners.interactive.recording_manager import RecordingManager
File "./sdks/python/apache_beam/runners/interactive/recording_manager.py", line 38, in <module>
from apache_beam.runners.interactive import background_caching_job as bcj
File "./sdks/python/apache_beam/runners/interactive/background_caching_job.py", line 46, in <module>
from apache_beam.runners.interactive import interactive_environment as ie
File "./sdks/python/apache_beam/runners/interactive/interactive_environment.py", line 41, in <module>
from apache_beam.runners.interactive.recording_manager import RecordingManager
ImportError: cannot import name 'RecordingManager' from partially initialized module 'apache_beam.runners.interactive.recording_manager' (most likely due to a circular import) (./sdks/python/apache_beam/runners/interactive/recording_manager.py)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix circular PipelineGraph import [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on PR #37134:
URL: https://github.com/apache/beam/pull/37134#issuecomment-3668029293
Looks like we are using an old python backend image for the playground, so the error is currently not shown on the playground website.
<img width="1134" height="328" alt="image" src="https://github.com/user-attachments/assets/ce4e367f-d30b-4954-b77c-01e2b996bb11" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] move pipelineGraph import [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37134:
URL: https://github.com/apache/beam/pull/37134#issuecomment-3667072128
Assigning reviewers:
R: @tvalentyn for label python.
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] move pipelineGraph import [beam]
Posted by "derrickaw (via GitHub)" <gi...@apache.org>.
derrickaw commented on PR #37134:
URL: https://github.com/apache/beam/pull/37134#issuecomment-3666952204
Run Python_Transforms PreCommit 3.13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix circular PipelineGraph import [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on PR #37134:
URL: https://github.com/apache/beam/pull/37134#issuecomment-3667199133
Let's monitor the test run tonight:
https://github.com/apache/beam/actions/workflows/beam_Playground_CI_Nightly.yml
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org