You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/05/23 19:14:09 UTC
[GitHub] [beam] jrmccluskey opened a new pull request, #17735: [BEAM-14499] Step global, unbounded side input case back to warning
jrmccluskey opened a new pull request, #17735:
URL: https://github.com/apache/beam/pull/17735
Turn the check for a globally windowed, unbounded side input into a warning rather than an error. This allows the TestStream integration tests to pass while passert streaming functions are developed.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on PR #17735:
URL: https://github.com/apache/beam/pull/17735#issuecomment-1135046007
Run Go Flink ValidatesRunner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
damccorm commented on code in PR #17735:
URL: https://github.com/apache/beam/pull/17735#discussion_r879812094
##########
sdks/go/pkg/beam/pardo.go:
##########
@@ -63,7 +65,8 @@ func TryParDo(s Scope, dofn interface{}, col PCollection, opts ...Option) ([]PCo
return nil, fmt.Errorf("main input is global windowed in DoFn %v but side input %v is not, cannot map windows correctly. Consider re-windowing the side input PCollection before use", fn, i)
}
if (sideWfn.Kind == window.GlobalWindows) && !sideNode.Bounded() {
- return nil, fmt.Errorf("side input %v is global windowed in DoFn %v but is unbounded, DoFn will block until end of Global Window. Consider windowing your unbounded side input PCollection before use", i, fn)
+ // TODO(BEAM-14501): Replace this warning with an error return when proper streaming test functions have been added.
+ log.Warnf(context.Background(), "side input %v is global windowed in DoFn %v but is unbounded, DoFn will block until end of Global Window. Consider windowing your unbounded side input PCollection before use", i, fn)
Review Comment:
If we're planning on upgrading this to an error, I would say that as part of the warning. That saves us the work of needing to do that sort of rollout work in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17735:
URL: https://github.com/apache/beam/pull/17735#issuecomment-1135045162
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17735:
URL: https://github.com/apache/beam/pull/17735#issuecomment-1135045160
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #17735:
URL: https://github.com/apache/beam/pull/17735#issuecomment-1135056960
# [Codecov](https://codecov.io/gh/apache/beam/pull/17735?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#17735](https://codecov.io/gh/apache/beam/pull/17735?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ec9bd58) into [master](https://codecov.io/gh/apache/beam/commit/1dfab628d03e161cf003dad01f55b9d6674aa8e2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1dfab62) will **decrease** coverage by `0.00%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #17735 +/- ##
==========================================
- Coverage 73.98% 73.98% -0.01%
==========================================
Files 696 696
Lines 91851 91852 +1
==========================================
- Hits 67958 67955 -3
- Misses 22644 22647 +3
- Partials 1249 1250 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| go | `50.44% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/17735?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/pardo.go](https://codecov.io/gh/apache/beam/pull/17735/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9wYXJkby5nbw==) | `47.41% <0.00%> (-3.03%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/17735?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/17735?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1dfab62...ec9bd58](https://codecov.io/gh/apache/beam/pull/17735?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on PR #17735:
URL: https://github.com/apache/beam/pull/17735#issuecomment-1135056238
Run Go Flink ValidatesRunner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on a diff in pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on code in PR #17735:
URL: https://github.com/apache/beam/pull/17735#discussion_r879814739
##########
sdks/go/pkg/beam/pardo.go:
##########
@@ -63,7 +65,8 @@ func TryParDo(s Scope, dofn interface{}, col PCollection, opts ...Option) ([]PCo
return nil, fmt.Errorf("main input is global windowed in DoFn %v but side input %v is not, cannot map windows correctly. Consider re-windowing the side input PCollection before use", fn, i)
}
if (sideWfn.Kind == window.GlobalWindows) && !sideNode.Bounded() {
- return nil, fmt.Errorf("side input %v is global windowed in DoFn %v but is unbounded, DoFn will block until end of Global Window. Consider windowing your unbounded side input PCollection before use", i, fn)
+ // TODO(BEAM-14501): Replace this warning with an error return when proper streaming test functions have been added.
+ log.Warnf(context.Background(), "side input %v is global windowed in DoFn %v but is unbounded, DoFn will block until end of Global Window. Consider windowing your unbounded side input PCollection before use", i, fn)
Review Comment:
That's a good note. Added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on PR #17735:
URL: https://github.com/apache/beam/pull/17735#issuecomment-1135068244
R: @lostluck @damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #17735:
URL: https://github.com/apache/beam/pull/17735#issuecomment-1135069375
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck merged pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
lostluck merged PR #17735:
URL: https://github.com/apache/beam/pull/17735
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17735: [BEAM-14499] Step global, unbounded side input case back to warning
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17735:
URL: https://github.com/apache/beam/pull/17735#issuecomment-1135045172
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org