You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/01/29 23:21:25 UTC

[GitHub] [beam] yifanmai opened a new pull request #13851: [BEAM-11644] Add ValidatesRunner test for translations.pack_combiners

yifanmai opened a new pull request #13851:
URL: https://github.com/apache/beam/pull/13851


   Add an integration test using TestPipeline with packable combiners. Combiner packing will be triggered if and only if the TestPipeline runner enables it.
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
    - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
   
   Post-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
   --- | --- | --- | --- | --- | --- | ---
   Go | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
   Java | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://ci-beam
 .apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [![Build Status](https://ci-beam.a
 pache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
   Python | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[![Build Status](https://ci-beam
 .apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
   XLang | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
   
   Pre-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   --- |Java | Python | Go | Website | Whitespace | Typescript
   --- | --- | --- | --- | --- | --- | ---
   Non-portable | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) <br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/be
 am_PreCommit_Go_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/)
   Portable | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | --- | --- | ---
   
   See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
   
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] chamikaramj commented on a change in pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #13851:
URL: https://github.com/apache/beam/pull/13851#discussion_r567172005



##########
File path: sdks/python/apache_beam/runners/portability/fn_api_runner/translations_test.py
##########
@@ -227,6 +232,53 @@ def assert_is_topologically_sorted(transform_id, visited_pcolls):
     assert_is_topologically_sorted(
         optimized_pipeline_proto.root_transform_ids[0], set())
 
+  @attr('ValidatesRunner')
+  def test_run_packable_combine_per_key(self):
+    class MultipleCombines(beam.PTransform):
+      def expand(self, pcoll):
+        assert_that(

Review comment:
       For example, Dataflow will not trigger it unless experiment "pre_optimize" is set, right ?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] codecov[bot] edited a comment on pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770109030


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=h1) Report
   > Merging [#13851](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=desc) (97cc14a) into [master](https://codecov.io/gh/apache/beam/commit/73731ec4f3f2d185e89aa3e378d321c2154ecf53?el=desc) (73731ec) will **increase** coverage by `0.08%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13851/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13851      +/-   ##
   ==========================================
   + Coverage   82.84%   82.92%   +0.08%     
   ==========================================
     Files         466      466              
     Lines       57596    58268     +672     
   ==========================================
   + Hits        47713    48321     +608     
   - Misses       9883     9947      +64     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.55% <0.00%> (-0.27%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `79.79% <0.00%> (-0.21%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.70% <0.00%> (-0.13%)` | :arrow_down: |
   | [setup.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/coders/coder\_impl.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL2NvZGVyX2ltcGwucHk=) | `95.38% <0.00%> (+0.13%)` | :arrow_up: |
   | [...ks/python/apache\_beam/coders/coders\_test\_common.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL2NvZGVyc190ZXN0X2NvbW1vbi5weQ==) | `98.77% <0.00%> (+0.26%)` | :arrow_up: |
   | [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.73% <0.00%> (+0.45%)` | :arrow_up: |
   | [sdks/python/apache\_beam/runners/direct/executor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZXhlY3V0b3IucHk=) | `96.82% <0.00%> (+0.52%)` | :arrow_up: |
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `91.31% <0.00%> (+1.79%)` | :arrow_up: |
   | [...python/apache\_beam/runners/direct/direct\_runner.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZGlyZWN0X3J1bm5lci5weQ==) | `96.29% <0.00%> (+2.44%)` | :arrow_up: |
   | ... and [3 more](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=footer). Last update [73731ec...97cc14a](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] chamikaramj commented on pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-771865293






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] codecov[bot] edited a comment on pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770109030


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=h1) Report
   > Merging [#13851](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=desc) (97cc14a) into [master](https://codecov.io/gh/apache/beam/commit/73731ec4f3f2d185e89aa3e378d321c2154ecf53?el=desc) (73731ec) will **decrease** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13851/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13851      +/-   ##
   ==========================================
   - Coverage   82.84%   82.83%   -0.01%     
   ==========================================
     Files         466      466              
     Lines       57596    57596              
   ==========================================
   - Hits        47713    47712       -1     
   - Misses       9883     9884       +1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `86.45% <0.00%> (-1.05%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.22% <0.00%> (-0.64%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.55% <0.00%> (-0.27%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.70% <0.00%> (-0.13%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `91.31% <0.00%> (+1.79%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=footer). Last update [73731ec...97cc14a](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] codecov[bot] edited a comment on pull request #13851: [BEAM-11644] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770109030


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=h1) Report
   > Merging [#13851](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=desc) (a9b999a) into [master](https://codecov.io/gh/apache/beam/commit/73731ec4f3f2d185e89aa3e378d321c2154ecf53?el=desc) (73731ec) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13851/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13851      +/-   ##
   ==========================================
   - Coverage   82.84%   82.83%   -0.02%     
   ==========================================
     Files         466      466              
     Lines       57596    57596              
   ==========================================
   - Hits        47713    47707       -6     
   - Misses       9883     9889       +6     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.85% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.56% <0.00%> (-0.42%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.55% <0.00%> (-0.27%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.57% <0.00%> (-0.26%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.35% <0.00%> (+0.35%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=footer). Last update [73731ec...a9b999a](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] chamikaramj commented on a change in pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #13851:
URL: https://github.com/apache/beam/pull/13851#discussion_r567171701



##########
File path: sdks/python/apache_beam/runners/portability/fn_api_runner/translations_test.py
##########
@@ -227,6 +232,53 @@ def assert_is_topologically_sorted(transform_id, visited_pcolls):
     assert_is_topologically_sorted(
         optimized_pipeline_proto.root_transform_ids[0], set())
 
+  @attr('ValidatesRunner')
+  def test_run_packable_combine_per_key(self):
+    class MultipleCombines(beam.PTransform):
+      def expand(self, pcoll):
+        assert_that(

Review comment:
       How do we confirm that combiner packing was triggered here ? This would pass even without combiner packing, right ?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] chamikaramj merged pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
chamikaramj merged pull request #13851:
URL: https://github.com/apache/beam/pull/13851


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] codecov[bot] edited a comment on pull request #13851: [BEAM-11644] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770109030


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=h1) Report
   > Merging [#13851](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=desc) (a9b999a) into [master](https://codecov.io/gh/apache/beam/commit/73731ec4f3f2d185e89aa3e378d321c2154ecf53?el=desc) (73731ec) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13851/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13851      +/-   ##
   ==========================================
   - Coverage   82.84%   82.83%   -0.02%     
   ==========================================
     Files         466      466              
     Lines       57596    57596              
   ==========================================
   - Hits        47713    47707       -6     
   - Misses       9883     9889       +6     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.85% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.56% <0.00%> (-0.42%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.55% <0.00%> (-0.27%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.57% <0.00%> (-0.26%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.35% <0.00%> (+0.35%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=footer). Last update [73731ec...a9b999a](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] chamikaramj commented on pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-771865293






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] codecov[bot] edited a comment on pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770109030


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=h1) Report
   > Merging [#13851](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=desc) (97cc14a) into [master](https://codecov.io/gh/apache/beam/commit/73731ec4f3f2d185e89aa3e378d321c2154ecf53?el=desc) (73731ec) will **decrease** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13851/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13851      +/-   ##
   ==========================================
   - Coverage   82.84%   82.83%   -0.01%     
   ==========================================
     Files         466      466              
     Lines       57596    57596              
   ==========================================
   - Hits        47713    47712       -1     
   - Misses       9883     9884       +1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `86.45% <0.00%> (-1.05%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.22% <0.00%> (-0.64%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.55% <0.00%> (-0.27%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.70% <0.00%> (-0.13%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `91.31% <0.00%> (+1.79%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=footer). Last update [73731ec...97cc14a](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] codecov[bot] edited a comment on pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770109030


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=h1) Report
   > Merging [#13851](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=desc) (97cc14a) into [master](https://codecov.io/gh/apache/beam/commit/73731ec4f3f2d185e89aa3e378d321c2154ecf53?el=desc) (73731ec) will **decrease** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13851/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13851      +/-   ##
   ==========================================
   - Coverage   82.84%   82.83%   -0.01%     
   ==========================================
     Files         466      466              
     Lines       57596    57596              
   ==========================================
   - Hits        47713    47712       -1     
   - Misses       9883     9884       +1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `86.45% <0.00%> (-1.05%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.22% <0.00%> (-0.64%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.55% <0.00%> (-0.27%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.70% <0.00%> (-0.13%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `91.31% <0.00%> (+1.79%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=footer). Last update [73731ec...97cc14a](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] codecov[bot] edited a comment on pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770109030






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] codecov[bot] commented on pull request #13851: [BEAM-11644] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770109030


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=h1) Report
   > Merging [#13851](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=desc) (a9b999a) into [master](https://codecov.io/gh/apache/beam/commit/73731ec4f3f2d185e89aa3e378d321c2154ecf53?el=desc) (73731ec) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13851/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13851      +/-   ##
   ==========================================
   - Coverage   82.84%   82.83%   -0.02%     
   ==========================================
     Files         466      466              
     Lines       57596    57596              
   ==========================================
   - Hits        47713    47707       -6     
   - Misses       9883     9889       +6     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.85% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.56% <0.00%> (-0.42%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.55% <0.00%> (-0.27%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.57% <0.00%> (-0.26%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.35% <0.00%> (+0.35%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=footer). Last update [73731ec...a9b999a](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] codecov[bot] edited a comment on pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770109030


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=h1) Report
   > Merging [#13851](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=desc) (a9b999a) into [master](https://codecov.io/gh/apache/beam/commit/73731ec4f3f2d185e89aa3e378d321c2154ecf53?el=desc) (73731ec) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13851/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13851      +/-   ##
   ==========================================
   - Coverage   82.84%   82.83%   -0.02%     
   ==========================================
     Files         466      466              
     Lines       57596    57596              
   ==========================================
   - Hits        47713    47707       -6     
   - Misses       9883     9889       +6     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.85% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.56% <0.00%> (-0.42%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.55% <0.00%> (-0.27%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.57% <0.00%> (-0.26%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.35% <0.00%> (+0.35%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=footer). Last update [73731ec...97cc14a](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] chamikaramj merged pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
chamikaramj merged pull request #13851:
URL: https://github.com/apache/beam/pull/13851


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] yifanmai commented on a change in pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
yifanmai commented on a change in pull request #13851:
URL: https://github.com/apache/beam/pull/13851#discussion_r567175544



##########
File path: sdks/python/apache_beam/runners/portability/fn_api_runner/translations_test.py
##########
@@ -227,6 +232,53 @@ def assert_is_topologically_sorted(transform_id, visited_pcolls):
     assert_is_topologically_sorted(
         optimized_pipeline_proto.root_transform_ids[0], set())
 
+  @attr('ValidatesRunner')
+  def test_run_packable_combine_per_key(self):
+    class MultipleCombines(beam.PTransform):
+      def expand(self, pcoll):
+        assert_that(

Review comment:
       Yes this would test without combiner packing. This test serves a different purpose; it tests that packable combines can be run successfully with correct outputs, whether or not the packing actually happens. I updated the comment to clarify this.
   
   The test that combiner packing was triggered has to be runner specific because not every runner supports combiner packing yet:
   
   - DataflowRunner has a test [here](https://github.com/apache/beam/blob/a44768f074ad0c55d5ca19c8ec6a47887b7e8538/sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py#L852-L883) but only checks for it under an experiment flag.
   - FnApiRunner has a test [here] but it is disabled because combiner packing doesn't actually work for PortableRunner and its subclasses yet.
   
   Speaking of experiment flags, ValidatesRunner has the downside of only testing the case with no experiment flags. So it won't do anything useful until combiner packing is enabled by default by each runner.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] codecov[bot] edited a comment on pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770109030


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=h1) Report
   > Merging [#13851](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=desc) (97cc14a) into [master](https://codecov.io/gh/apache/beam/commit/73731ec4f3f2d185e89aa3e378d321c2154ecf53?el=desc) (73731ec) will **increase** coverage by `0.08%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13851/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13851      +/-   ##
   ==========================================
   + Coverage   82.84%   82.92%   +0.08%     
   ==========================================
     Files         466      466              
     Lines       57596    58268     +672     
   ==========================================
   + Hits        47713    48321     +608     
   - Misses       9883     9947      +64     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.55% <0.00%> (-0.27%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `79.79% <0.00%> (-0.21%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.70% <0.00%> (-0.13%)` | :arrow_down: |
   | [setup.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/coders/coder\_impl.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL2NvZGVyX2ltcGwucHk=) | `95.38% <0.00%> (+0.13%)` | :arrow_up: |
   | [...ks/python/apache\_beam/coders/coders\_test\_common.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL2NvZGVyc190ZXN0X2NvbW1vbi5weQ==) | `98.77% <0.00%> (+0.26%)` | :arrow_up: |
   | [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.73% <0.00%> (+0.45%)` | :arrow_up: |
   | [sdks/python/apache\_beam/runners/direct/executor.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZXhlY3V0b3IucHk=) | `96.82% <0.00%> (+0.52%)` | :arrow_up: |
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `91.31% <0.00%> (+1.79%)` | :arrow_up: |
   | [...python/apache\_beam/runners/direct/direct\_runner.py](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZGlyZWN0X3J1bm5lci5weQ==) | `96.29% <0.00%> (+2.44%)` | :arrow_up: |
   | ... and [3 more](https://codecov.io/gh/apache/beam/pull/13851/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=footer). Last update [73731ec...97cc14a](https://codecov.io/gh/apache/beam/pull/13851?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] yifanmai commented on pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
yifanmai commented on pull request #13851:
URL: https://github.com/apache/beam/pull/13851#issuecomment-770132169


   cc @robertwb 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] yifanmai commented on a change in pull request #13851: [BEAM-11715] Add ValidatesRunner test for translations.pack_combiners

Posted by GitBox <gi...@apache.org>.
yifanmai commented on a change in pull request #13851:
URL: https://github.com/apache/beam/pull/13851#discussion_r567175544



##########
File path: sdks/python/apache_beam/runners/portability/fn_api_runner/translations_test.py
##########
@@ -227,6 +232,53 @@ def assert_is_topologically_sorted(transform_id, visited_pcolls):
     assert_is_topologically_sorted(
         optimized_pipeline_proto.root_transform_ids[0], set())
 
+  @attr('ValidatesRunner')
+  def test_run_packable_combine_per_key(self):
+    class MultipleCombines(beam.PTransform):
+      def expand(self, pcoll):
+        assert_that(

Review comment:
       Yes this would test without combiner packing. This test serves a different purpose; it tests that packable combines can be run successfully with correct outputs, whether or not the packing actually happens. I updated the comment to clarify this.
   
   The test that combiner packing was triggered has to be runner specific because not every runner supports combiner packing yet:
   
   - DataflowRunner has a test [here](https://github.com/apache/beam/blob/a44768f074ad0c55d5ca19c8ec6a47887b7e8538/sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py#L852-L883) but only checks for it under an experiment flag.
   - FnApiRunner has a test [here](https://github.com/apache/beam/blob/a44768f074ad0c55d5ca19c8ec6a47887b7e8538/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner_test.py#L1037-L1040) but it is disabled because combiner packing doesn't actually work for PortableRunner and its subclasses yet.
   
   Speaking of experiment flags, ValidatesRunner has the downside of only testing the case with no experiment flags. So it won't do anything useful until combiner packing is enabled by default by each runner.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org