You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "yquaziii (via GitHub)" <gi...@apache.org> on 2025/07/30 18:52:55 UTC

[PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

yquaziii opened a new pull request, #35744:
URL: https://github.com/apache/beam/pull/35744

   **Please** add a meaningful description for your change here
   >> Added warning if the stage_location and temp_location is invalid it will try to create default bucket.
   Fixes #35584
   
   Tested : 
   <img width="2254" height="486" alt="AxsXncGrX2sDKhB" src="https://github.com/user-attachments/assets/54fd464a-1448-4779-904c-89663fb4659c" />
   
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/actions/workflows/build_wheels.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/actions/workflows/python_tests.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/actions/workflows/java_tests.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/actions/workflows/go_tests.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

Posted by "stankiewicz (via GitHub)" <gi...@apache.org>.
stankiewicz commented on PR #35744:
URL: https://github.com/apache/beam/pull/35744#issuecomment-3333583973

   there are still format issues, please check log (https://github.com/apache/beam/actions/runs/16631409020/job/47061515465?pr=35744) and https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-LintandFormattingChecks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

Posted by "derrickaw (via GitHub)" <gi...@apache.org>.
derrickaw commented on PR #35744:
URL: https://github.com/apache/beam/pull/35744#issuecomment-3179488497

   waiting on author


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35744:
URL: https://github.com/apache/beam/pull/35744#discussion_r2243913001


##########
sdks/python/apache_beam/options/pipeline_options.py:
##########
@@ -1172,6 +1172,10 @@ def _handle_temp_and_staging_locations(self, validator):
       return []
     # Both staging and temp locations are bad, try to use default bucket.
     else:
+      _LOGGER.warning(

Review Comment:
   It is valid when error is "ERR_MISSING_GCS_PATH". We should only issue warning when either is "ERR_INVALID_GCS_PATH"



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #35744:
URL: https://github.com/apache/beam/pull/35744#discussion_r2243605428


##########
sdks/python/apache_beam/options/pipeline_options.py:
##########
@@ -1169,6 +1169,10 @@ def _handle_temp_and_staging_locations(self, validator):
     elif not staging_errors and not temp_errors:
       self._warn_if_soft_delete_policy_enabled('temp_location')
       self._warn_if_soft_delete_policy_enabled('staging_location')
+      _LOGGER.warning(
+        "Both temp_location and staging_location are invalid. "
+        "Expected format is 'gs://<bucket>/<object>'. "
+        "Attempting to create a default GCS bucket.")

Review Comment:
   ![critical](https://www.gstatic.com/codereviewagent/critical.svg)
   
   This warning is placed in the wrong logical block. The `elif not staging_errors and not temp_errors:` condition is true when both `staging_location` and `temp_location` are valid, but the warning message states that they are invalid. This will result in a confusing and incorrect warning being logged for valid configurations. This warning should be moved to the `else` block (starting at line 1178), which handles the case where both locations are invalid.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

Posted by "yquaziii (via GitHub)" <gi...@apache.org>.
yquaziii commented on code in PR #35744:
URL: https://github.com/apache/beam/pull/35744#discussion_r2243629349


##########
sdks/python/apache_beam/options/pipeline_options.py:
##########
@@ -1169,6 +1169,10 @@ def _handle_temp_and_staging_locations(self, validator):
     elif not staging_errors and not temp_errors:
       self._warn_if_soft_delete_policy_enabled('temp_location')
       self._warn_if_soft_delete_policy_enabled('staging_location')
+      _LOGGER.warning(
+        "Both temp_location and staging_location are invalid. "
+        "Expected format is 'gs://<bucket>/<object>'. "
+        "Attempting to create a default GCS bucket.")

Review Comment:
   ack, added to else block.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35744:
URL: https://github.com/apache/beam/pull/35744#issuecomment-3596384248

   This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #35744: Fix: Handle invalid bucket path warning in Beam Python SDK
URL: https://github.com/apache/beam/pull/35744


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35744:
URL: https://github.com/apache/beam/pull/35744#discussion_r2243913001


##########
sdks/python/apache_beam/options/pipeline_options.py:
##########
@@ -1172,6 +1172,10 @@ def _handle_temp_and_staging_locations(self, validator):
       return []
     # Both staging and temp locations are bad, try to use default bucket.
     else:
+      _LOGGER.warning(

Review Comment:
   It is valid when error is "ERR_MISSING_GCS_PATH". We should only issue warning when either is "ERR_MISSING_GCS_PATH"



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35744:
URL: https://github.com/apache/beam/pull/35744#issuecomment-3137667014

   Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix: Handle invalid bucket path warning in Beam Python SDK [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35744:
URL: https://github.com/apache/beam/pull/35744#issuecomment-3570628743

   This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org