You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "riteshghorse (via GitHub)" <gi...@apache.org> on 2023/10/11 17:41:17 UTC

[PR] fail windowInto when there are no inputs to it [beam]

riteshghorse opened a new pull request, #28946:
URL: https://github.com/apache/beam/pull/28946

   **Please** add a meaningful description for your change here
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/workflows/Go%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Python] Fail WindowInto when there are no inputs to it [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28946:
URL: https://github.com/apache/beam/pull/28946#issuecomment-1759603161

   Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fail windowInto when there are no inputs to it [beam]

Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #28946:
URL: https://github.com/apache/beam/pull/28946#issuecomment-1759600557

   R: @tvalentyn 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Python] Fail WindowInto when there are no inputs to it [beam]

Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb merged PR #28946:
URL: https://github.com/apache/beam/pull/28946


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Python] Fail WindowInto when there are no inputs to it [beam]

Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb commented on code in PR #28946:
URL: https://github.com/apache/beam/pull/28946#discussion_r1362363985


##########
sdks/python/apache_beam/runners/common.py:
##########
@@ -1928,6 +1928,10 @@ def validate_transform(transform_id):
             "Incompatible input coder %s and output coder %s for transform %s" %
             (transform_id, input_coder, output_coder))
 
+    if transform_proto.spec.urn == common_urns.primitives.ASSIGN_WINDOWS.urn:

Review Comment:
   Let's validate `common_urns.primitives.PAR_DO.urn` as well. 



##########
sdks/python/apache_beam/runners/common.py:
##########
@@ -1928,6 +1928,10 @@ def validate_transform(transform_id):
             "Incompatible input coder %s and output coder %s for transform %s" %
             (transform_id, input_coder, output_coder))
 
+    if transform_proto.spec.urn == common_urns.primitives.ASSIGN_WINDOWS.urn:
+      if not transform_proto.inputs:
+        raise ValueError("Unexpected number of inputs: %s" % transform_proto)

Review Comment:
   Maybe say "Missing inputs?"



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Python] Fail WindowInto when there are no inputs to it [beam]

Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on code in PR #28946:
URL: https://github.com/apache/beam/pull/28946#discussion_r1357225166


##########
sdks/python/apache_beam/pipeline.py:
##########
@@ -1353,18 +1353,28 @@ def transform_to_runner_api(
       environment_id = context.get_environment_id_for_resource_hints(
           self.resource_hints)
 
+    inputs = {

Review Comment:
   > p | beam.WindowInto(...) should also be an error right there (checked in WindowInto's expand method).
   
   +1 to Robert's comment, moving this validation inside `WindowInto` methods  would be better.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fail windowInto when there are no inputs to it [beam]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #28946:
URL: https://github.com/apache/beam/pull/28946#issuecomment-1758205365

   ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/28946?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   > Merging [#28946](https://app.codecov.io/gh/apache/beam/pull/28946?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (eb13b0b) into [master](https://app.codecov.io/gh/apache/beam/commit/2bfcb9fda37be8d5b9550a1b118ed87d03fdf12c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (2bfcb9f) will **decrease** coverage by `0.02%`.
   > Report is 1 commits behind head on master.
   > The diff coverage is `100.00%`.
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #28946      +/-   ##
   ==========================================
   - Coverage   72.16%   72.15%   -0.02%     
   ==========================================
     Files         686      686              
     Lines      101560   101593      +33     
   ==========================================
   + Hits        73294    73302       +8     
   - Misses      26687    26712      +25     
     Partials     1579     1579              
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/beam/pull/28946/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [python](https://app.codecov.io/gh/apache/beam/pull/28946/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `82.58% <100.00%> (-0.03%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files](https://app.codecov.io/gh/apache/beam/pull/28946?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/pipeline.py](https://app.codecov.io/gh/apache/beam/pull/28946?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.78% <100.00%> (+0.03%)` | :arrow_up: |
   
   ... and [7 files with indirect coverage changes](https://app.codecov.io/gh/apache/beam/pull/28946/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   
   :mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fail windowInto when there are no inputs to it [beam]

Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #28946:
URL: https://github.com/apache/beam/pull/28946#issuecomment-1758276923

   Run Python 3.8 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Python] Fail WindowInto when there are no inputs to it [beam]

Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb commented on PR #28946:
URL: https://github.com/apache/beam/pull/28946#issuecomment-1759926310

   FWIW, I think this should be fixed in the service rather than adding a special case to the SDK (Python only, which won't be out for 6 weeks or so).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Python] Fail WindowInto when there are no inputs to it [beam]

Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb commented on PR #28946:
URL: https://github.com/apache/beam/pull/28946#issuecomment-1760126482

   A good place to validate this might be https://github.com/apache/beam/blob/release-2.51.0/sdks/python/apache_beam/runners/common.py#L1895
   
   We could have a mapping of known URNs -> expected input/output cardinality (if any), and flag violations. (We could ensure the inputs/outputs actually exist.)
   
   `p | beam.WindowInto(...)` should also be an error right there (checked in WindowInto's expand method). 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org