You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "olehborysevych (via GitHub)" <gi...@apache.org> on 2023/04/14 09:20:52 UTC
[GitHub] [beam] olehborysevych opened a new pull request, #26279: Tob example never run tag
olehborysevych opened a new pull request, #26279:
URL: https://github.com/apache/beam/pull/26279
Adding new example tag to upload prepared output and logs
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] olehborysevych commented on a diff in pull request #26279: Add new example never_run tag for precompiled objects
Posted by "olehborysevych (via GitHub)" <gi...@apache.org>.
olehborysevych commented on code in PR #26279:
URL: https://github.com/apache/beam/pull/26279#discussion_r1170147008
##########
playground/infrastructure/verify.py:
##########
@@ -66,7 +72,46 @@ async def _run_and_verify(self, examples: List[Example]):
examples: beam examples that should be run
"""
+
async def _populate_fields(example: Example):
+ """
+ Populate fields of the example reading them from the backend or from the repository.
+ Args:
+ example: beam example that should be verified
+ """
+ if example.tag.never_run:
+ logging.info("populating example fields from provided files %s", example.filepath)
+ _populate_from_repo(example)
+ else:
+ await _populate_from_runner(example)
+
+ def _populate_from_repo(example: Example):
+ """
+ Populate fields of the example reading them from the repository.
+ Args:
+ example: beam example that should be verified
+ """
+ path = Path(example.filepath)
+ example_folder = path.parent
+
+ log_file_path = example_folder / self.LOGS_FILENAME
+ # Check if the file exists and read its content
+ if log_file_path.exists():
Review Comment:
> I'm trying to understand the motivation, could you give me an example of where we would want to use `never_run`?
This is to be able to skip CI and upload logs and output for examples that cannot be run in production. We would like to upload cross-language examples with this tag as we agreed with Kerry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #26279: Add new example never_run tag for precompiled objects
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26279:
URL: https://github.com/apache/beam/pull/26279#issuecomment-1508530893
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] olehborysevych commented on pull request #26279: Add new example never_run tag for precompiled objects
Posted by "olehborysevych (via GitHub)" <gi...@apache.org>.
olehborysevych commented on PR #26279:
URL: https://github.com/apache/beam/pull/26279#issuecomment-1508529399
R: @damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #26279: Add new example never_run tag for precompiled objects
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #26279:
URL: https://github.com/apache/beam/pull/26279#discussion_r1166961818
##########
playground/infrastructure/verify.py:
##########
@@ -66,7 +72,46 @@ async def _run_and_verify(self, examples: List[Example]):
examples: beam examples that should be run
"""
+
async def _populate_fields(example: Example):
+ """
+ Populate fields of the example reading them from the backend or from the repository.
+ Args:
+ example: beam example that should be verified
+ """
+ if example.tag.never_run:
+ logging.info("populating example fields from provided files %s", example.filepath)
+ _populate_from_repo(example)
+ else:
+ await _populate_from_runner(example)
+
+ def _populate_from_repo(example: Example):
+ """
+ Populate fields of the example reading them from the repository.
+ Args:
+ example: beam example that should be verified
+ """
+ path = Path(example.filepath)
+ example_folder = path.parent
+
+ log_file_path = example_folder / self.LOGS_FILENAME
+ # Check if the file exists and read its content
+ if log_file_path.exists():
Review Comment:
If these files don't exist, should we be raising some sort of exception?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] olehborysevych commented on a diff in pull request #26279: Add new example never_run tag for precompiled objects
Posted by "olehborysevych (via GitHub)" <gi...@apache.org>.
olehborysevych commented on code in PR #26279:
URL: https://github.com/apache/beam/pull/26279#discussion_r1170149575
##########
playground/infrastructure/verify.py:
##########
@@ -66,7 +72,46 @@ async def _run_and_verify(self, examples: List[Example]):
examples: beam examples that should be run
"""
+
async def _populate_fields(example: Example):
+ """
+ Populate fields of the example reading them from the backend or from the repository.
+ Args:
+ example: beam example that should be verified
+ """
+ if example.tag.never_run:
+ logging.info("populating example fields from provided files %s", example.filepath)
+ _populate_from_repo(example)
+ else:
+ await _populate_from_runner(example)
+
+ def _populate_from_repo(example: Example):
+ """
+ Populate fields of the example reading them from the repository.
+ Args:
+ example: beam example that should be verified
+ """
+ path = Path(example.filepath)
+ example_folder = path.parent
+
+ log_file_path = example_folder / self.LOGS_FILENAME
+ # Check if the file exists and read its content
+ if log_file_path.exists():
Review Comment:
> If these files don't exist, should we be raising some sort of exception?
We do not treat this as exception. It's ok if some files are missing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm merged pull request #26279: Add new example never_run tag for precompiled objects
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm merged PR #26279:
URL: https://github.com/apache/beam/pull/26279
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org