You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "robertwb (via GitHub)" <gi...@apache.org> on 2023/11/17 23:17:36 UTC
[PR] [YAML] Clean up some confusing error messages. [beam]
robertwb opened a new pull request, #29481:
URL: https://github.com/apache/beam/pull/29481
* Provide sane defaults rather than NullPtrExceptions for optional BigQuery parameters.
* Better error when cross-language is used with (incompatible) local streaming Python runner.
* Add format and schema to readme PubSub examples.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [YAML] Clean up some confusing error messages. [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29481:
URL: https://github.com/apache/beam/pull/29481#issuecomment-1817279657
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [YAML] Clean up some confusing error messages. [beam]
Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb commented on PR #29481:
URL: https://github.com/apache/beam/pull/29481#issuecomment-1832604736
R: @Polber
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [YAML] Clean up some confusing error messages. [beam]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #29481:
URL: https://github.com/apache/beam/pull/29481#issuecomment-1817254393
## [Codecov](https://app.codecov.io/gh/apache/beam/pull/29481?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: `6 lines` in your changes are missing coverage. Please review.
> Comparison is base [(`67dbdd1`)](https://app.codecov.io/gh/apache/beam/commit/67dbdd196ab86d2198907642bb98d4d8322af425?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 37.85% compared to head [(`9d154da`)](https://app.codecov.io/gh/apache/beam/pull/29481?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 37.85%.
> Report is 5 commits behind head on master.
| [Files](https://app.codecov.io/gh/apache/beam/pull/29481?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...python/apache\_beam/runners/direct/direct\_runner.py](https://app.codecov.io/gh/apache/beam/pull/29481?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZGlyZWN0X3J1bm5lci5weQ==) | 0.00% | [6 Missing :warning: ](https://app.codecov.io/gh/apache/beam/pull/29481?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #29481 +/- ##
==========================================
- Coverage 37.85% 37.85% -0.01%
==========================================
Files 690 690
Lines 101305 101311 +6
==========================================
Hits 38350 38350
- Misses 61362 61368 +6
Partials 1593 1593
```
| [Flag](https://app.codecov.io/gh/apache/beam/pull/29481/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [python](https://app.codecov.io/gh/apache/beam/pull/29481/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `29.00% <0.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/beam/pull/29481?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [YAML] Clean up some confusing error messages. [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29481:
URL: https://github.com/apache/beam/pull/29481#issuecomment-1832606741
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [YAML] Clean up some confusing error messages. [beam]
Posted by "Polber (via GitHub)" <gi...@apache.org>.
Polber commented on code in PR #29481:
URL: https://github.com/apache/beam/pull/29481#discussion_r1416263299
##########
sdks/python/apache_beam/runners/direct/direct_runner.py:
##########
@@ -507,6 +507,17 @@ def run_pipeline(self, pipeline, options):
from apache_beam.runners.direct.transform_evaluator import \
TransformEvaluatorRegistry
from apache_beam.testing.test_stream import TestStream
+ from apache_beam.transforms.external import ExternalTransform
+
+ class VerifyNoCrossLanguageTransforms(PipelineVisitor):
+ """Visitor determining whether a Pipeline uses a TestStream."""
+ def visit_transform(self, applied_ptransform):
+ if isinstance(applied_ptransform.transform, ExternalTransform):
+ raise RuntimeError(
+ "Streaming Python direct runner "
+ "does not support cross-language pipelines.")
Review Comment:
Should there be a FR for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [YAML] Clean up some confusing error messages. [beam]
Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb commented on code in PR #29481:
URL: https://github.com/apache/beam/pull/29481#discussion_r1416285850
##########
sdks/python/apache_beam/runners/direct/direct_runner.py:
##########
@@ -507,6 +507,17 @@ def run_pipeline(self, pipeline, options):
from apache_beam.runners.direct.transform_evaluator import \
TransformEvaluatorRegistry
from apache_beam.testing.test_stream import TestStream
+ from apache_beam.transforms.external import ExternalTransform
+
+ class VerifyNoCrossLanguageTransforms(PipelineVisitor):
+ """Visitor determining whether a Pipeline uses a TestStream."""
+ def visit_transform(self, applied_ptransform):
+ if isinstance(applied_ptransform.transform, ExternalTransform):
+ raise RuntimeError(
+ "Streaming Python direct runner "
+ "does not support cross-language pipelines.")
Review Comment:
https://issues.apache.org/jira/browse/BEAM-7514 is probably the closest we have.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [YAML] Clean up some confusing error messages. [beam]
Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb merged PR #29481:
URL: https://github.com/apache/beam/pull/29481
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org