You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/01/29 23:56:42 UTC

[GitHub] [beam] TheNeuralBit opened a new pull request #13854: [BEAM-11324] Impose consistent ordering on partitionings used in PartitioningSession

TheNeuralBit opened a new pull request #13854:
URL: https://github.com/apache/beam/pull/13854


   Without this change, it's possible that an incorrectly specified expression can create a flaky test. Since we only keep the result from the last executed partitioning, it's possible to get a false positive if, say, Singleton partitioning was used last. This change ensures we always use the most challenging partitioning for the final result. 
   
   Pre-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   --- |Java | Python | Go | Website | Whitespace | Typescript
   --- | --- | --- | --- | --- | --- | ---
   Non-portable | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) <br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/be
 am_PreCommit_Go_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/)
   Portable | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | --- | --- | ---
   
   See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
   
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit merged pull request #13854: [BEAM-11324] Impose consistent ordering on partitionings used in PartitioningSession

Posted by GitBox <gi...@apache.org>.
TheNeuralBit merged pull request #13854:
URL: https://github.com/apache/beam/pull/13854


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #13854: [BEAM-11324] Impose consistent ordering on partitionings used in PartitioningSession

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13854:
URL: https://github.com/apache/beam/pull/13854#issuecomment-770122821


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=h1) Report
   > Merging [#13854](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=desc) (b13c3c3) into [master](https://codecov.io/gh/apache/beam/commit/623a9c09bf3dd1a483b0631eb8ece2125c5c79ec?el=desc) (623a9c0) will **decrease** coverage by `0.00%`.
   > The diff coverage is `90.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13854/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13854      +/-   ##
   ==========================================
   - Coverage   82.83%   82.83%   -0.01%     
   ==========================================
     Files         466      466              
     Lines       57596    57605       +9     
   ==========================================
   + Hits        47711    47717       +6     
   - Misses       9885     9888       +3     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/dataframe/expressions.py](https://codecov.io/gh/apache/beam/pull/13854/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2V4cHJlc3Npb25zLnB5) | `91.02% <90.00%> (-0.14%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13854/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.44% <0.00%> (-0.26%)` | :arrow_down: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=footer). Last update [96280f7...b13c3c3](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #13854: [BEAM-11324] Impose consistent ordering on partitionings used in PartitioningSession

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #13854:
URL: https://github.com/apache/beam/pull/13854#issuecomment-770122821


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=h1) Report
   > Merging [#13854](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=desc) (b13c3c3) into [master](https://codecov.io/gh/apache/beam/commit/623a9c09bf3dd1a483b0631eb8ece2125c5c79ec?el=desc) (623a9c0) will **decrease** coverage by `0.00%`.
   > The diff coverage is `90.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13854/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13854      +/-   ##
   ==========================================
   - Coverage   82.83%   82.83%   -0.01%     
   ==========================================
     Files         466      466              
     Lines       57596    57605       +9     
   ==========================================
   + Hits        47711    47717       +6     
   - Misses       9885     9888       +3     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/dataframe/expressions.py](https://codecov.io/gh/apache/beam/pull/13854/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2V4cHJlc3Npb25zLnB5) | `91.02% <90.00%> (-0.14%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13854/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.44% <0.00%> (-0.26%)` | :arrow_down: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=footer). Last update [96280f7...b13c3c3](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] commented on pull request #13854: [BEAM-11324] Impose consistent ordering on partitionings used in PartitioningSession

Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #13854:
URL: https://github.com/apache/beam/pull/13854#issuecomment-770122821


   # [Codecov](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=h1) Report
   > Merging [#13854](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=desc) (b13c3c3) into [master](https://codecov.io/gh/apache/beam/commit/623a9c09bf3dd1a483b0631eb8ece2125c5c79ec?el=desc) (623a9c0) will **decrease** coverage by `0.00%`.
   > The diff coverage is `90.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/13854/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #13854      +/-   ##
   ==========================================
   - Coverage   82.83%   82.83%   -0.01%     
   ==========================================
     Files         466      466              
     Lines       57596    57605       +9     
   ==========================================
   + Hits        47711    47717       +6     
   - Misses       9885     9888       +3     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/dataframe/expressions.py](https://codecov.io/gh/apache/beam/pull/13854/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2V4cHJlc3Npb25zLnB5) | `91.02% <90.00%> (-0.14%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13854/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.44% <0.00%> (-0.26%)` | :arrow_down: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=footer). Last update [96280f7...b13c3c3](https://codecov.io/gh/apache/beam/pull/13854?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on pull request #13854: [BEAM-11324] Impose consistent ordering on partitionings used in PartitioningSession

Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #13854:
URL: https://github.com/apache/beam/pull/13854#issuecomment-770111139


   R: @robertwb 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org