You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/03/03 03:29:21 UTC

[GitHub] [beam] TheNeuralBit opened a new pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

TheNeuralBit opened a new pull request #14135:
URL: https://github.com/apache/beam/pull/14135


   This modifies the ordering defined by is_subpartitioning_of from:
   ```
   Nothing() < Index([i]) < Index([i, j]) < ... < Index() < Singleton()
   ```
   to:
   ```
   Singleton() < Index([i]) < Index([i, j]) < ... < Index() < Nothing()
   ```
   
   Which is logically more correct, since `Singleton()` is equivalent to `Index([])`. The rest of the PR deals with the fallout from this change, which largely entails updated `preserves_partition_by` specs for computed expressions (e0aba2d), and updating the logic for fusing stages (29f41c0). It also includes a patch to have `Index.check` make a strong assertion about partioning, as discussed in BEAM-11324.
   
   Post-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
   --- | --- | --- | --- | --- | --- | ---
   Go | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
   Java | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://ci-beam
 .apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [![Build Status](https://ci-beam.a
 pache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
   Python | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[![Build Status](https://ci-beam
 .apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
   XLang | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
   
   Pre-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   --- |Java | Python | Go | Website | Whitespace | Typescript
   --- | --- | --- | --- | --- | --- | ---
   Non-portable | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) <br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/be
 am_PreCommit_Go_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/)
   Portable | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | --- | --- | ---
   
   See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
   
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (5ba8319) into [master](https://codecov.io/gh/apache/beam/commit/08a9d54b60cd4743afeb61e697fa6016279b1ac6?el=desc) (08a9d54) will **increase** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.17%   +0.02%     
   ==========================================
     Files         469      469              
     Lines       58972    58989      +17     
   ==========================================
   + Hits        49037    49066      +29     
   + Misses       9935     9923      -12     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...\_beam/io/gcp/internal/clients/bigquery/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvX19pbml0X18ucHk=) | | |
   | [...rcs/sdks/python/apache\_beam/typehints/typehints.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3R5cGVoaW50cy5weQ==) | | |
   | [...nchmarks/chicago\_taxi/tfdv\_analyze\_and\_validate.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL2NoaWNhZ29fdGF4aS90ZmR2X2FuYWx5emVfYW5kX3ZhbGlkYXRlLnB5) | | |
   | [...ks/python/apache\_beam/io/gcp/pubsub\_it\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3B1YnN1Yl9pdF9waXBlbGluZS5weQ==) | | |
   | [...uild/srcs/sdks/python/apache\_beam/io/tfrecordio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGZyZWNvcmRpby5weQ==) | | |
   | [...eam/transforms/py\_dataflow\_distribution\_counter.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9weV9kYXRhZmxvd19kaXN0cmlidXRpb25fY291bnRlci5weQ==) | | |
   | [...am/examples/snippets/transforms/aggregation/min.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9taW4ucHk=) | | |
   | [...m/portability/api/beam\_interactive\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMl9ncnBjLnB5) | | |
   | [...\_beam/testing/benchmarks/nexmark/queries/query6.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTYucHk=) | | |
   | [...d/srcs/sdks/python/apache\_beam/internal/pickler.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvcGlja2xlci5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [0ad65c6...5ba8319](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (5ba8319) into [master](https://codecov.io/gh/apache/beam/commit/08a9d54b60cd4743afeb61e697fa6016279b1ac6?el=desc) (08a9d54) will **increase** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.17%   +0.02%     
   ==========================================
     Files         469      469              
     Lines       58972    58989      +17     
   ==========================================
   + Hits        49037    49066      +29     
   + Misses       9935     9923      -12     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ild/srcs/sdks/python/apache\_beam/transforms/sql.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9zcWwucHk=) | | |
   | [...s/sdks/python/apache\_beam/testing/test\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3BpcGVsaW5lLnB5) | | |
   | [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | | |
   | [...m/runners/portability/flink\_uber\_jar\_job\_server.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbGlua191YmVyX2phcl9qb2Jfc2VydmVyLnB5) | | |
   | [...ython/apache\_beam/io/aws/clients/s3/fake\_client.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXdzL2NsaWVudHMvczMvZmFrZV9jbGllbnQucHk=) | | |
   | [...38/build/srcs/sdks/python/apache\_beam/io/fileio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZWlvLnB5) | | |
   | [...m/runners/portability/spark\_uber\_jar\_job\_server.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zcGFya191YmVyX2phcl9qb2Jfc2VydmVyLnB5) | | |
   | [...\_beam/io/gcp/datastore/v1new/adaptive\_throttler.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9hZGFwdGl2ZV90aHJvdHRsZXIucHk=) | | |
   | [...amples/snippets/transforms/aggregation/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9fX2luaXRfXy5weQ==) | | |
   | [...s/python/apache\_beam/examples/snippets/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvX19pbml0X18ucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [0ad65c6...1b3e231](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (5ba8319) into [master](https://codecov.io/gh/apache/beam/commit/08a9d54b60cd4743afeb61e697fa6016279b1ac6?el=desc) (08a9d54) will **increase** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.17%   +0.02%     
   ==========================================
     Files         469      469              
     Lines       58972    58989      +17     
   ==========================================
   + Hits        49037    49066      +29     
   + Misses       9935     9923      -12     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...hon/apache\_beam/examples/wordcount\_with\_metrics.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X3dpdGhfbWV0cmljcy5weQ==) | | |
   | [...pache\_beam/runners/interactive/caching/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoaW5nL19faW5pdF9fLnB5) | | |
   | [.../python/apache\_beam/io/gcp/bigquery\_io\_metadata.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X2lvX21ldGFkYXRhLnB5) | | |
   | [...\_beam/testing/benchmarks/nexmark/queries/query5.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTUucHk=) | | |
   | [...am/examples/complete/juliaset/juliaset/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvanVsaWFzZXQvanVsaWFzZXQvX19pbml0X18ucHk=) | | |
   | [...apache\_beam/runners/dataflow/internal/apiclient.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9hcGljbGllbnQucHk=) | | |
   | [...mples/snippets/transforms/elementwise/partition.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS9wYXJ0aXRpb24ucHk=) | | |
   | [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | | |
   | [...on/apache\_beam/runners/direct/sdf\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3Qvc2RmX2RpcmVjdF9ydW5uZXIucHk=) | | |
   | [...python/apache\_beam/transforms/external\_test\_py3.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbF90ZXN0X3B5My5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [0ad65c6...5ba8319](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on a change in pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

TheNeuralBit commented on a change in pull request #14135:
URL: https://github.com/apache/beam/pull/14135#discussion_r591984316



##########
File path: sdks/python/apache_beam/dataframe/expressions.py
##########
@@ -312,6 +352,11 @@ def __init__(
       requires_partition_by: The required (common) partitioning of the args.
       preserves_partition_by: The level of partitioning preserved.
     """
+    assert preserves_partition_by != partitionings.Nothing(), (
+        "Preserving Nothing() partitioning is not allowed. Any expression "
+        "can trivially preserve at least Singleton() partitioning. If you "
+        "intend to indicate this expression can preserve _any_ partioning, use "
+        "Index() instead.")

Review comment:
       Done




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] aromanenko-dev edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

aromanenko-dev edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-800473198


   Well, my point is that with many small commits it's very difficult to keep a git history of atomic and independent commits (ideally, every commit has to be easy to revert in case if it's needed). Also, we tend to add a Jira prefix to commit message to make it easier for a search in the future. Imho, ideally, we have to have only **one** "well tested" commit per one Jira despite the number of lines and changes (of course there always will be exceptions).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (d8eb470) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14135   +/-   ##
   =======================================
     Coverage   83.04%   83.05%           
   =======================================
     Files         469      469           
     Lines       58473    58482    +9     
   =======================================
   + Hits        48561    48574   +13     
   + Misses       9912     9908    -4     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...y38/build/srcs/sdks/python/apache\_beam/io/vcfio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdmNmaW8ucHk=) | | |
   | [...rcs/sdks/python/apache\_beam/typehints/typecheck.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3R5cGVjaGVjay5weQ==) | | |
   | [...d/srcs/sdks/python/apache\_beam/metrics/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWV0cmljcy9fX2luaXRfXy5weQ==) | | |
   | [...am/testing/benchmarks/nexmark/models/field\_name.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbW9kZWxzL2ZpZWxkX25hbWUucHk=) | | |
   | [...am/examples/snippets/transforms/aggregation/top.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi90b3AucHk=) | | |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | | |
   | [...ython/apache\_beam/io/gcp/experimental/spannerio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2V4cGVyaW1lbnRhbC9zcGFubmVyaW8ucHk=) | | |
   | [...srcs/sdks/python/apache\_beam/dataframe/doctests.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2RvY3Rlc3RzLnB5) | | |
   | [...tes/tox/py38/build/srcs/sdks/python/test\_config.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vdGVzdF9jb25maWcucHk=) | | |
   | [.../srcs/sdks/python/apache\_beam/internal/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvX19pbml0X18ucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...d8eb470](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] aromanenko-dev commented on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

aromanenko-dev commented on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-800267717


   @TheNeuralBit Please, clean up the history before merge (as @robertwb suggested above) to avoid having a bunch of tiny not-atomic commits.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] aromanenko-dev edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

aromanenko-dev edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-800473198


   Well, my point is that with many small commits it's very difficult to keep a git history of atomic and independent commits (ideally, every commit has to be easy to revert in case if it's needed). Also, we tend to add a Jira prefix to commit message to make it easier for a search in the future. Imho, ideally, we have to have only **one** "well tested" commit per one Jira despite the number of lines and changes (of course it always will be exceptions).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (1b3e231) into [master](https://codecov.io/gh/apache/beam/commit/08a9d54b60cd4743afeb61e697fa6016279b1ac6?el=desc) (08a9d54) will **increase** coverage by `0.03%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.18%   +0.03%     
   ==========================================
     Files         469      469              
     Lines       58972    58989      +17     
   ==========================================
   + Hits        49037    49069      +32     
   + Misses       9935     9920      -15     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...srcs/sdks/python/apache\_beam/io/gcp/dicomclient.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RpY29tY2xpZW50LnB5) | | |
   | [...examples/snippets/transforms/aggregation/latest.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9sYXRlc3QucHk=) | | |
   | [...am/examples/snippets/transforms/aggregation/sum.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9zdW0ucHk=) | | |
   | [...apache\_beam/examples/complete/game/leader\_board.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvZ2FtZS9sZWFkZXJfYm9hcmQucHk=) | | |
   | [...ython/apache\_beam/io/gcp/bigquery\_read\_internal.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3JlYWRfaW50ZXJuYWwucHk=) | | |
   | [...beam/portability/api/beam\_artifact\_api\_pb2\_urns.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fYXJ0aWZhY3RfYXBpX3BiMl91cm5zLnB5) | | |
   | [.../interactive/display/interactive\_pipeline\_graph.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L2ludGVyYWN0aXZlX3BpcGVsaW5lX2dyYXBoLnB5) | | |
   | [.../srcs/sdks/python/apache\_beam/io/gcp/bigtableio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3RhYmxlaW8ucHk=) | | |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | | |
   | [...nchmarks/chicago\_taxi/tfdv\_analyze\_and\_validate.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL2NoaWNhZ29fdGF4aS90ZmR2X2FuYWx5emVfYW5kX3ZhbGlkYXRlLnB5) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [0ad65c6...1b3e231](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (c54cc08) into [master](https://codecov.io/gh/apache/beam/commit/08a9d54b60cd4743afeb61e697fa6016279b1ac6?el=desc) (08a9d54) will **increase** coverage by `0.03%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.18%   +0.03%     
   ==========================================
     Files         469      469              
     Lines       58972    58969       -3     
   ==========================================
   + Hits        49037    49054      +17     
   + Misses       9935     9915      -20     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ild/srcs/sdks/python/apache\_beam/io/gcp/dicomio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RpY29taW8ucHk=) | | |
   | [...pache\_beam/dataframe/pandas\_top\_level\_functions.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL3BhbmRhc190b3BfbGV2ZWxfZnVuY3Rpb25zLnB5) | | |
   | [...ld/srcs/sdks/python/apache\_beam/io/aws/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXdzL19faW5pdF9fLnB5) | | |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | | |
   | [...examples/snippets/transforms/aggregation/sample.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9zYW1wbGUucHk=) | | |
   | [...\_beam/testing/benchmarks/nexmark/queries/query8.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTgucHk=) | | |
   | [...cs/sdks/python/apache\_beam/transforms/combiners.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jb21iaW5lcnMucHk=) | | |
   | [...ython/apache\_beam/examples/kafkataxi/kafka\_taxi.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMva2Fma2F0YXhpL2thZmthX3RheGkucHk=) | | |
   | [.../build/srcs/sdks/python/apache\_beam/utils/retry.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcmV0cnkucHk=) | | |
   | [.../examples/snippets/transforms/elementwise/regex.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS9yZWdleC5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [0ad65c6...5ba8319](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (0446559) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14135   +/-   ##
   =======================================
     Coverage   83.04%   83.05%           
   =======================================
     Files         469      469           
     Lines       58473    58499   +26     
   =======================================
   + Hits        48561    48588   +27     
   + Misses       9912     9911    -1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...am/examples/snippets/transforms/aggregation/min.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9taW4ucHk=) | | |
   | [...\_beam/io/gcp/datastore/v1new/adaptive\_throttler.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9hZGFwdGl2ZV90aHJvdHRsZXIucHk=) | | |
   | [...ild/srcs/sdks/python/apache\_beam/io/filesystems.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZXN5c3RlbXMucHk=) | | |
   | [.../sdks/python/apache\_beam/examples/snippets/util.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdXRpbC5weQ==) | | |
   | [...n/apache\_beam/typehints/typed\_pipeline\_test\_py3.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3R5cGVkX3BpcGVsaW5lX3Rlc3RfcHkzLnB5) | | |
   | [...s/sdks/python/apache\_beam/testing/test\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3BpcGVsaW5lLnB5) | | |
   | [...nternal/clients/cloudbuild/cloudbuild\_v1\_client.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9jbGllbnRzL2Nsb3VkYnVpbGQvY2xvdWRidWlsZF92MV9jbGllbnQucHk=) | | |
   | [...python/apache\_beam/examples/complete/distribopt.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvZGlzdHJpYm9wdC5weQ==) | | |
   | [...ers/dataflow/dataflow\_exercise\_metrics\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19leGVyY2lzZV9tZXRyaWNzX3BpcGVsaW5lLnB5) | | |
   | [...s/sdks/python/apache\_beam/io/gcp/bigquery\_tools.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3Rvb2xzLnB5) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...0446559](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (c54cc08) into [master](https://codecov.io/gh/apache/beam/commit/1ad764b33d91bc78af54a53babf73b59aede04ad?el=desc) (1ad764b) will **increase** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.18%   +0.02%     
   ==========================================
     Files         469      469              
     Lines       58954    58969      +15     
   ==========================================
   + Hits        49024    49054      +30     
   + Misses       9930     9915      -15     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../srcs/sdks/python/apache\_beam/io/azure/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXp1cmUvX19pbml0X18ucHk=) | | |
   | [...e\_beam/examples/complete/top\_wikipedia\_sessions.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvdG9wX3dpa2lwZWRpYV9zZXNzaW9ucy5weQ==) | | |
   | [...38/build/srcs/sdks/python/apache\_beam/io/textio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGV4dGlvLnB5) | | |
   | [...pache\_beam/runners/interactive/caching/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoaW5nL19faW5pdF9fLnB5) | | |
   | [...ython/apache\_beam/io/gcp/bigquery\_read\_internal.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3JlYWRfaW50ZXJuYWwucHk=) | | |
   | [...build/srcs/sdks/python/apache\_beam/utils/plugin.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcGx1Z2luLnB5) | | |
   | [...am/examples/complete/juliaset/juliaset/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvanVsaWFzZXQvanVsaWFzZXQvX19pbml0X18ucHk=) | | |
   | [...ers/dataflow/internal/clients/dataflow/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9jbGllbnRzL2RhdGFmbG93L19faW5pdF9fLnB5) | | |
   | [...\_beam/runners/interactive/user\_pipeline\_tracker.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS91c2VyX3BpcGVsaW5lX3RyYWNrZXIucHk=) | | |
   | [...rcs/sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [1ad764b...c54cc08](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on a change in pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

TheNeuralBit commented on a change in pull request #14135:
URL: https://github.com/apache/beam/pull/14135#discussion_r590552721



##########
File path: sdks/python/apache_beam/dataframe/partitionings_test.py
##########
@@ -36,8 +36,8 @@ class PartitioningsTest(unittest.TestCase):
   }).set_index(['shape', 'color', 'size'])
 
   def test_index_is_subpartition(self):
-    ordered_list = [Nothing(), Index([3]), Index([1, 3]), Index(), Singleton()]
-    for loose, strict in zip(ordered_list[:1], ordered_list[1:]):
+    ordered_list = [Singleton(), Index([3]), Index([1, 3]), Index(), Nothing()]
+    for loose, strict in zip(ordered_list[:-1], ordered_list[1:]):

Review comment:
       Note the change to make this `-1`. This test was only verifying `Singleton()` vs `Index([3])` then exiting the loop. The checks between different `Index()` partitionings would have failed, but they weren't being executed.

##########
File path: sdks/python/apache_beam/dataframe/frames.py
##########
@@ -130,7 +130,7 @@ def fillna(self, value, method, axis, **kwargs):
             lambda df,
             value: df.fillna(value, method=method, axis=axis, **kwargs),
             [self._expr, value_expr],
-            preserves_partition_by=partitionings.Singleton(),
+            preserves_partition_by=partitionings.Index(),

Review comment:
       In general, what we previously called `preserves=Singleton()` (or equivalently `preserves=Index()`), i.e. "this expression will preserve any partitioning", now must be indicated by `preserves=Index()`.
   
   Also what we previously called `preserves=Nothing()`, i.e. "this expression cannot preserve any partitioning", is now represented with `preserves=Singleton()`. This is perhaps a little surprising but I think it's easy to reason about, just think of it as "the only thing this expression preserves is Singleton() parititioning, which is trivially preserved".
   
   All the `preserves=` updates are applying this mapping

##########
File path: sdks/python/apache_beam/dataframe/transforms.py
##########
@@ -331,8 +339,7 @@ def expr_to_stages(expr):
       required_partitioning = expr.requires_partition_by()
       for stage in common_stages([expr_to_stages(arg) for arg in expr.args()
                                   if arg not in inputs]):
-        if all(output_is_partitioned_by(arg, stage, required_partitioning)
-               for arg in expr.args() if not is_scalar(arg)):
+        if output_partitioning_in_stage(expr, stage) is not None:

Review comment:
       Instead of duplicating a portion of the logic from `output_is_partitioned_by`, this new implementation moves all of the logic into `output_partitioning_in_stage`. Here we just check for a None output indicating this expression cannot be computed in this stage.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (1b3e231) into [master](https://codecov.io/gh/apache/beam/commit/08a9d54b60cd4743afeb61e697fa6016279b1ac6?el=desc) (08a9d54) will **increase** coverage by `0.03%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.18%   +0.03%     
   ==========================================
     Files         469      469              
     Lines       58972    58989      +17     
   ==========================================
   + Hits        49037    49069      +32     
   + Misses       9935     9920      -15     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../apache\_beam/options/pipeline\_options\_validator.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vb3B0aW9ucy9waXBlbGluZV9vcHRpb25zX3ZhbGlkYXRvci5weQ==) | | |
   | [.../srcs/sdks/python/apache\_beam/io/gcp/bigtableio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3RhYmxlaW8ucHk=) | | |
   | [...ython/apache\_beam/io/gcp/bigquery\_read\_internal.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3JlYWRfaW50ZXJuYWwucHk=) | | |
   | [.../srcs/sdks/python/apache\_beam/examples/sql\_taxi.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc3FsX3RheGkucHk=) | | |
   | [.../runners/portability/fn\_api\_runner/translations.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3RyYW5zbGF0aW9ucy5weQ==) | | |
   | [...on/apache\_beam/portability/api/metrics\_pb2\_urns.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyX3VybnMucHk=) | | |
   | [...beam/testing/load\_tests/load\_test\_metrics\_utils.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9sb2FkX3Rlc3RzL2xvYWRfdGVzdF9tZXRyaWNzX3V0aWxzLnB5) | | |
   | [...sdks/python/apache\_beam/io/watermark\_estimators.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vd2F0ZXJtYXJrX2VzdGltYXRvcnMucHk=) | | |
   | [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | | |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [0ad65c6...1b3e231](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit merged pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

TheNeuralBit merged pull request #14135:
URL: https://github.com/apache/beam/pull/14135


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (29f41c0) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   - Coverage   83.04%   83.02%   -0.02%     
   ==========================================
     Files         469      469              
     Lines       58473    58482       +9     
   ==========================================
   - Hits        48561    48557       -4     
   - Misses       9912     9925      +13     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...s/sdks/python/apache\_beam/testing/test\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3BpcGVsaW5lLnB5) | | |
   | [...pache\_beam/portability/api/beam\_fn\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZm5fYXBpX3BiMl9ncnBjLnB5) | | |
   | [...s/python/apache\_beam/examples/snippets/snippets.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvc25pcHBldHMucHk=) | | |
   | [...e\_beam/portability/api/beam\_interactive\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMi5weQ==) | | |
   | [...sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | | |
   | [...ks/python/apache\_beam/runners/worker/operations.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvb3BlcmF0aW9ucy5weQ==) | | |
   | [.../srcs/sdks/python/apache\_beam/utils/annotations.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvYW5ub3RhdGlvbnMucHk=) | | |
   | [...ython/apache\_beam/io/gcp/experimental/spannerio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2V4cGVyaW1lbnRhbC9zcGFubmVyaW8ucHk=) | | |
   | [...srcs/sdks/python/apache\_beam/dataframe/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL19faW5pdF9fLnB5) | | |
   | [...python/apache\_beam/io/gcp/experimental/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2V4cGVyaW1lbnRhbC9fX2luaXRfXy5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...29f41c0](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on a change in pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

TheNeuralBit commented on a change in pull request #14135:
URL: https://github.com/apache/beam/pull/14135#discussion_r591956347



##########
File path: sdks/python/apache_beam/dataframe/transforms.py
##########
@@ -282,30 +282,38 @@ def __repr__(self, indent=0):
                 self.outputs))
 
     # First define some helper functions.
-    def output_is_partitioned_by(expr, stage, partitioning):
-      if partitioning == partitionings.Nothing():
-        # Always satisfied.
-        return True
-      elif stage.partitioning == partitionings.Singleton():
-        # Within a stage, the singleton partitioning is trivially preserved.
-        return True
-      elif expr in stage.inputs:
+    def output_partitioning_in_stage(expr, stage):
+      """ Return the output partitioning of expr when computed in stage,
+      or returns None if the expression cannot be computed in this stage.
+      """
+      if expr in stage.inputs or expr in inputs:
         # Inputs are all partitioned by stage.partitioning.
-        return stage.partitioning.is_subpartitioning_of(partitioning)
-      elif expr.preserves_partition_by().is_subpartitioning_of(partitioning):
-        # Here expr preserves at least the requested partitioning; its outputs
-        # will also have this partitioning iff its inputs do.
-        if expr.requires_partition_by().is_subpartitioning_of(partitioning):
-          # If expr requires at least this partitioning, we will arrange such
-          # that its inputs satisfy this.
-          return True
-        else:
-          # Otherwise, recursively check all the inputs.
-          return all(
-              output_is_partitioned_by(arg, stage, partitioning)
-              for arg in expr.args())
-      else:
-        return False
+        return stage.partitioning
+
+      # Anything that's not an input must have arguments
+      assert len(expr.args())
+
+      arg_partitionings = set(
+          output_partitioning_in_stage(arg, stage) for arg in expr.args()
+          if not is_scalar(arg))
+
+      # TODO: what does it mean for all args to be 0? only scalar arguments? the

Review comment:
       No we didn't run into this case. I think this makes sense based on our discussion yesterday - there's currently no way to create an expression that operates on only scalar inputs. The only thing you can do with a `DeferredScalar` is use it in arithmetic with a `DeferredFrame`. If/when we support arithmetic on DeferredScalars I think we'd hit this. Probably we should just return expr.preserves_partition_by() in this case.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (c54cc08) into [master](https://codecov.io/gh/apache/beam/commit/1ad764b33d91bc78af54a53babf73b59aede04ad?el=desc) (1ad764b) will **increase** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.18%   +0.02%     
   ==========================================
     Files         469      469              
     Lines       58954    58969      +15     
   ==========================================
   + Hits        49024    49054      +30     
   + Misses       9930     9915      -15     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ild/srcs/sdks/python/apache\_beam/tools/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdG9vbHMvX19pbml0X18ucHk=) | | |
   | [...beam/runners/portability/local\_job\_service\_main.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9sb2NhbF9qb2Jfc2VydmljZV9tYWluLnB5) | | |
   | [...n/apache\_beam/examples/complete/game/user\_score.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvZ2FtZS91c2VyX3Njb3JlLnB5) | | |
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | | |
   | [...s/sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy90ZXN0L19faW5pdF9fLnB5) | | |
   | [...he\_beam/testing/benchmarks/nexmark/nexmark\_perf.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19wZXJmLnB5) | | |
   | [...examples/snippets/transforms/aggregation/latest.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9sYXRlc3QucHk=) | | |
   | [...ld/srcs/sdks/python/apache\_beam/coders/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL19faW5pdF9fLnB5) | | |
   | [...thon/apache\_beam/runners/worker/channel\_factory.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvY2hhbm5lbF9mYWN0b3J5LnB5) | | |
   | [...ks/python/apache\_beam/examples/cookbook/filters.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svZmlsdGVycy5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [1ad764b...c54cc08](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (0446559) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14135   +/-   ##
   =======================================
     Coverage   83.04%   83.05%           
   =======================================
     Files         469      469           
     Lines       58473    58499   +26     
   =======================================
   + Hits        48561    48588   +27     
   + Misses       9912     9911    -1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...hon/apache\_beam/examples/wordcount\_with\_metrics.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X3dpdGhfbWV0cmljcy5weQ==) | | |
   | [...dks/python/apache\_beam/ml/gcp/naturallanguageml.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvZ2NwL25hdHVyYWxsYW5ndWFnZW1sLnB5) | | |
   | [...nchmarks/chicago\_taxi/tfdv\_analyze\_and\_validate.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL2NoaWNhZ29fdGF4aS90ZmR2X2FuYWx5emVfYW5kX3ZhbGlkYXRlLnB5) | | |
   | [...on/apache\_beam/runners/worker/statesampler\_slow.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc3RhdGVzYW1wbGVyX3Nsb3cucHk=) | | |
   | [...38/build/srcs/sdks/python/apache\_beam/io/fileio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZWlvLnB5) | | |
   | [...ild/srcs/sdks/python/apache\_beam/runners/runner.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9ydW5uZXIucHk=) | | |
   | [...apache\_beam/runners/dataflow/internal/apiclient.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9hcGljbGllbnQucHk=) | | |
   | [...sdks/python/apache\_beam/transforms/environments.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9lbnZpcm9ubWVudHMucHk=) | | |
   | [...che\_beam/examples/flink/flink\_streaming\_impulse.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvZmxpbmsvZmxpbmtfc3RyZWFtaW5nX2ltcHVsc2UucHk=) | | |
   | [...m/testing/benchmarks/nexmark/models/auction\_bid.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbW9kZWxzL2F1Y3Rpb25fYmlkLnB5) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...0446559](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (0446559) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14135   +/-   ##
   =======================================
     Coverage   83.04%   83.05%           
   =======================================
     Files         469      469           
     Lines       58473    58499   +26     
   =======================================
   + Hits        48561    48588   +27     
   + Misses       9912     9911    -1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ld/srcs/sdks/python/apache\_beam/utils/histogram.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaGlzdG9ncmFtLnB5) | | |
   | [...python/apache\_beam/examples/wordcount\_debugging.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RlYnVnZ2luZy5weQ==) | | |
   | [...on/apache\_beam/runners/portability/flink\_runner.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbGlua19ydW5uZXIucHk=) | | |
   | [...runners/interactive/options/interactive\_options.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9vcHRpb25zL2ludGVyYWN0aXZlX29wdGlvbnMucHk=) | | |
   | [...ld/srcs/sdks/python/apache\_beam/transforms/core.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jb3JlLnB5) | | |
   | [...beam/portability/api/beam\_artifact\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fYXJ0aWZhY3RfYXBpX3BiMl9ncnBjLnB5) | | |
   | [...examples/snippets/transforms/aggregation/latest.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9sYXRlc3QucHk=) | | |
   | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | | |
   | [...ache\_beam/runners/dataflow/test\_dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy90ZXN0X2RhdGFmbG93X3J1bm5lci5weQ==) | | |
   | [...s/python/apache\_beam/typehints/sharded\_key\_type.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3NoYXJkZWRfa2V5X3R5cGUucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...0446559](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] aromanenko-dev commented on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

aromanenko-dev commented on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-800473198


   Well, my point is that with many small commits it's very difficult to keep a git history of atomic and independent commits (ideally, every commit has to be easy to revert in case if it's needed). Also, we tend to add a Jira prefix to commit message to make it easier for a search in the future. Imho, ideally, we have to have only **one** "well tested" commit per one Jira despite the number of lines and changes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

TheNeuralBit commented on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-800291507


   The force push I did was my attempt to clean up commits - squashing together fixups. The remaining commits are largely independent portions of the refactor, along with some small commits fixing bugs that the refactor exposed, and some bulk renames.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (29f41c0) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   - Coverage   83.04%   83.02%   -0.02%     
   ==========================================
     Files         469      469              
     Lines       58473    58482       +9     
   ==========================================
   - Hits        48561    48557       -4     
   - Misses       9912     9925      +13     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...cs/sdks/python/apache\_beam/transforms/userstate.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91c2Vyc3RhdGUucHk=) | | |
   | [...srcs/sdks/python/apache\_beam/coders/avro\_record.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL2F2cm9fcmVjb3JkLnB5) | | |
   | [...ks/python/apache\_beam/runners/worker/opcounters.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvb3Bjb3VudGVycy5weQ==) | | |
   | [...beam/runners/dataflow/internal/clients/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9jbGllbnRzL19faW5pdF9fLnB5) | | |
   | [...build/srcs/sdks/python/apache\_beam/utils/shared.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc2hhcmVkLnB5) | | |
   | [...thon/apache\_beam/examples/complete/autocomplete.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvYXV0b2NvbXBsZXRlLnB5) | | |
   | [...s/sdks/python/apache\_beam/dataframe/expressions.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2V4cHJlc3Npb25zLnB5) | | |
   | [...d/srcs/sdks/python/apache\_beam/transforms/stats.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9zdGF0cy5weQ==) | | |
   | [...dks/python/apache\_beam/io/gcp/internal/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL19faW5pdF9fLnB5) | | |
   | [...\_beam/io/gcp/datastore/v1new/adaptive\_throttler.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9hZGFwdGl2ZV90aHJvdHRsZXIucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...29f41c0](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (29f41c0) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   - Coverage   83.04%   83.02%   -0.02%     
   ==========================================
     Files         469      469              
     Lines       58473    58482       +9     
   ==========================================
   - Hits        48561    48557       -4     
   - Misses       9912     9925      +13     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../srcs/sdks/python/apache\_beam/transforms/window.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy93aW5kb3cucHk=) | | |
   | [...flow/internal/clients/dataflow/message\_matchers.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9jbGllbnRzL2RhdGFmbG93L21lc3NhZ2VfbWF0Y2hlcnMucHk=) | | |
   | [...38/build/srcs/sdks/python/apache\_beam/io/fileio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZWlvLnB5) | | |
   | [...ks/python/apache\_beam/runners/interactive/utils.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS91dGlscy5weQ==) | | |
   | [...d/srcs/sdks/python/apache\_beam/transforms/stats.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9zdGF0cy5weQ==) | | |
   | [...hon/apache\_beam/runners/dataflow/internal/names.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9uYW1lcy5weQ==) | | |
   | [...sdks/python/apache\_beam/runners/direct/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvX19pbml0X18ucHk=) | | |
   | [...ld/srcs/sdks/python/apache\_beam/utils/histogram.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaGlzdG9ncmFtLnB5) | | |
   | [...am/testing/benchmarks/nexmark/models/field\_name.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbW9kZWxzL2ZpZWxkX25hbWUucHk=) | | |
   | [.../build/srcs/sdks/python/apache\_beam/utils/retry.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcmV0cnkucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...29f41c0](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (1b3e231) into [master](https://codecov.io/gh/apache/beam/commit/08a9d54b60cd4743afeb61e697fa6016279b1ac6?el=desc) (08a9d54) will **increase** coverage by `0.03%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.18%   +0.03%     
   ==========================================
     Files         469      469              
     Lines       58972    58989      +17     
   ==========================================
   + Hits        49037    49069      +32     
   + Misses       9935     9920      -15     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...am/examples/snippets/transforms/elementwise/map.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS9tYXAucHk=) | | |
   | [.../srcs/sdks/python/apache\_beam/runners/sdf\_utils.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9zZGZfdXRpbHMucHk=) | | |
   | [...ache\_beam/examples/snippets/transforms/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9fX2luaXRfXy5weQ==) | | |
   | [...rcs/sdks/python/apache\_beam/transforms/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9fX2luaXRfXy5weQ==) | | |
   | [...ples/snippets/transforms/aggregation/groupbykey.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9ncm91cGJ5a2V5LnB5) | | |
   | [...e\_beam/portability/api/beam\_interactive\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMi5weQ==) | | |
   | [...sdks/python/apache\_beam/dataframe/partitionings.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL3BhcnRpdGlvbmluZ3MucHk=) | | |
   | [...build/srcs/sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | | |
   | [...dks/python/apache\_beam/portability/api/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL19faW5pdF9fLnB5) | | |
   | [...d/srcs/sdks/python/apache\_beam/transforms/stats.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9zdGF0cy5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [0ad65c6...1b3e231](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (29f41c0) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   - Coverage   83.04%   83.02%   -0.02%     
   ==========================================
     Files         469      469              
     Lines       58473    58482       +9     
   ==========================================
   - Hits        48561    48557       -4     
   - Misses       9912     9925      +13     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ers/interactive/testing/integration/screen\_diff.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS90ZXN0aW5nL2ludGVncmF0aW9uL3NjcmVlbl9kaWZmLnB5) | | |
   | [...python/apache\_beam/examples/wordcount\_debugging.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RlYnVnZ2luZy5weQ==) | | |
   | [...dks/python/apache\_beam/transforms/external\_java.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbF9qYXZhLnB5) | | |
   | [.../apache\_beam/runners/dataflow/internal/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9fX2luaXRfXy5weQ==) | | |
   | [...\_beam/testing/benchmarks/nexmark/queries/query6.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTYucHk=) | | |
   | [...\_beam/testing/benchmarks/nexmark/queries/query5.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTUucHk=) | | |
   | [...dks/python/apache\_beam/options/pipeline\_options.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vb3B0aW9ucy9waXBlbGluZV9vcHRpb25zLnB5) | | |
   | [...d/srcs/sdks/python/apache\_beam/internal/pickler.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvcGlja2xlci5weQ==) | | |
   | [...python/apache\_beam/io/gcp/experimental/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2V4cGVyaW1lbnRhbC9fX2luaXRfXy5weQ==) | | |
   | [...apache\_beam/examples/cookbook/custom\_ptransform.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svY3VzdG9tX3B0cmFuc2Zvcm0ucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...29f41c0](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] commented on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] commented on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (29f41c0) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   - Coverage   83.04%   83.02%   -0.02%     
   ==========================================
     Files         469      469              
     Lines       58473    58482       +9     
   ==========================================
   - Hits        48561    48557       -4     
   - Misses       9912     9925      +13     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...sdks/python/apache\_beam/io/watermark\_estimators.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vd2F0ZXJtYXJrX2VzdGltYXRvcnMucHk=) | | |
   | [...apache\_beam/portability/api/beam\_runner\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcnVubmVyX2FwaV9wYjIucHk=) | | |
   | [...pache\_beam/runners/portability/artifact\_service.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9hcnRpZmFjdF9zZXJ2aWNlLnB5) | | |
   | [...python/apache\_beam/typehints/typecheck\_test\_py3.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3R5cGVjaGVja190ZXN0X3B5My5weQ==) | | |
   | [...s/python/apache\_beam/examples/cookbook/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svX19pbml0X18ucHk=) | | |
   | [...sdks/python/apache\_beam/io/restriction\_trackers.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vcmVzdHJpY3Rpb25fdHJhY2tlcnMucHk=) | | |
   | [...dks/python/apache\_beam/io/gcp/internal/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL19faW5pdF9fLnB5) | | |
   | [...srcs/sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | | |
   | [...8/build/srcs/sdks/python/apache\_beam/utils/urns.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvdXJucy5weQ==) | | |
   | [...srcs/sdks/python/apache\_beam/io/gcp/tests/utils.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3Rlc3RzL3V0aWxzLnB5) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...29f41c0](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (0446559) into [master](https://codecov.io/gh/apache/beam/commit/1ad764b33d91bc78af54a53babf73b59aede04ad?el=desc) (1ad764b) will **decrease** coverage by `0.09%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   - Coverage   83.15%   83.05%   -0.10%     
   ==========================================
     Files         469      469              
     Lines       58954    58499     -455     
   ==========================================
   - Hits        49024    48588     -436     
   + Misses       9930     9911      -19     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../apache\_beam/testing/benchmarks/nexmark/monitor.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbW9uaXRvci5weQ==) | | |
   | [.../sdks/python/apache\_beam/internal/metrics/cells.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9jZWxscy5weQ==) | | |
   | [...build/srcs/sdks/python/apache\_beam/dataframe/io.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2lvLnB5) | | |
   | [...flow/internal/clients/dataflow/message\_matchers.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9jbGllbnRzL2RhdGFmbG93L21lc3NhZ2VfbWF0Y2hlcnMucHk=) | | |
   | [...rcs/sdks/python/apache\_beam/io/aws/s3filesystem.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXdzL3MzZmlsZXN5c3RlbS5weQ==) | | |
   | [...apache\_beam/testing/benchmarks/nexmark/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvX19pbml0X18ucHk=) | | |
   | [.../srcs/sdks/python/apache\_beam/typehints/schemas.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3NjaGVtYXMucHk=) | | |
   | [...eam/portability/api/beam\_expansion\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZXhwYW5zaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | | |
   | [...d/srcs/sdks/python/apache\_beam/testing/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9fX2luaXRfXy5weQ==) | | |
   | [...apache\_beam/runners/portability/portable\_runner.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9wb3J0YWJsZV9ydW5uZXIucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [1ad764b...c54cc08](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (c54cc08) into [master](https://codecov.io/gh/apache/beam/commit/1ad764b33d91bc78af54a53babf73b59aede04ad?el=desc) (1ad764b) will **increase** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.18%   +0.02%     
   ==========================================
     Files         469      469              
     Lines       58954    58969      +15     
   ==========================================
   + Hits        49024    49054      +30     
   + Misses       9930     9915      -15     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...s/python/apache\_beam/io/aws/clients/s3/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXdzL2NsaWVudHMvczMvX19pbml0X18ucHk=) | | |
   | [...gcp/datastore/v1new/datastore\_write\_it\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9kYXRhc3RvcmVfd3JpdGVfaXRfcGlwZWxpbmUucHk=) | | |
   | [...runners/interactive/options/interactive\_options.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9vcHRpb25zL2ludGVyYWN0aXZlX29wdGlvbnMucHk=) | | |
   | [...ache\_beam/portability/api/beam\_artifact\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fYXJ0aWZhY3RfYXBpX3BiMi5weQ==) | | |
   | [...pache\_beam/portability/api/beam\_fn\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZm5fYXBpX3BiMl9ncnBjLnB5) | | |
   | [...srcs/sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | | |
   | [...dks/python/apache\_beam/transforms/create\_source.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jcmVhdGVfc291cmNlLnB5) | | |
   | [...ython/apache\_beam/runners/direct/bundle\_factory.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvYnVuZGxlX2ZhY3RvcnkucHk=) | | |
   | [...srcs/sdks/python/apache\_beam/coders/avro\_record.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL2F2cm9fcmVjb3JkLnB5) | | |
   | [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [1ad764b...c54cc08](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on a change in pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

TheNeuralBit commented on a change in pull request #14135:
URL: https://github.com/apache/beam/pull/14135#discussion_r591984211



##########
File path: sdks/python/apache_beam/dataframe/transforms.py
##########
@@ -331,8 +339,7 @@ def expr_to_stages(expr):
       required_partitioning = expr.requires_partition_by()
       for stage in common_stages([expr_to_stages(arg) for arg in expr.args()
                                   if arg not in inputs]):
-        if all(output_is_partitioned_by(arg, stage, required_partitioning)
-               for arg in expr.args() if not is_scalar(arg)):
+        if output_partitioning_in_stage(expr, stage) is not None:

Review comment:
       Good idea, done.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (0446559) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14135   +/-   ##
   =======================================
     Coverage   83.04%   83.05%           
   =======================================
     Files         469      469           
     Lines       58473    58499   +26     
   =======================================
   + Hits        48561    48588   +27     
   + Misses       9912     9911    -1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...srcs/sdks/python/apache\_beam/examples/wordcount.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50LnB5) | | |
   | [...ks/python/apache\_beam/runners/worker/statecache.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc3RhdGVjYWNoZS5weQ==) | | |
   | [...sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | | |
   | [...s/python/apache\_beam/testing/pipeline\_verifiers.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9waXBlbGluZV92ZXJpZmllcnMucHk=) | | |
   | [...ild/srcs/sdks/python/apache\_beam/runners/runner.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9ydW5uZXIucHk=) | | |
   | [...pache\_beam/runners/portability/artifact\_service.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9hcnRpZmFjdF9zZXJ2aWNlLnB5) | | |
   | [...build/srcs/sdks/python/apache\_beam/io/parquetio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vcGFycXVldGlvLnB5) | | |
   | [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | | |
   | [...rcs/sdks/python/apache\_beam/io/hadoopfilesystem.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaGFkb29wZmlsZXN5c3RlbS5weQ==) | | |
   | [...s/python/apache\_beam/examples/snippets/snippets.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvc25pcHBldHMucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...0446559](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] aromanenko-dev edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

aromanenko-dev edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-800473198


   Well, my point is that with many small commits it's very difficult to keep a git history of atomic and independent commits (ideally, every commit has to be easy to revert in case if it's needed). Also, we tend to add a Jira prefix to commit message to make it easier for a search in the future. Imho, ideally, we have to have only **one** "well tested" commit per one Jira despite the number of lines and changes (of course it always will be exceptions there).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] robertwb commented on a change in pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

robertwb commented on a change in pull request #14135:
URL: https://github.com/apache/beam/pull/14135#discussion_r591137449



##########
File path: sdks/python/apache_beam/dataframe/expressions.py
##########
@@ -281,7 +321,7 @@ def requires_partition_by(self):
     return partitionings.Nothing()
 
   def preserves_partition_by(self):
-    return partitionings.Nothing()
+    return partitionings.Index()

Review comment:
       Similarly. 

##########
File path: sdks/python/apache_beam/dataframe/expressions.py
##########
@@ -245,7 +285,7 @@ def requires_partition_by(self):
     return partitionings.Nothing()
 
   def preserves_partition_by(self):
-    return partitionings.Nothing()
+    return partitionings.Index()

Review comment:
       In the new hierarchy, Nothing should still be valid, right? Maybe that just sounds weird though. (E.g. one may require nothing and preserve everything.) 

##########
File path: sdks/python/apache_beam/dataframe/expressions_test.py
##########
@@ -53,6 +54,61 @@ def test_expression_proxy_error(self):
     with self.assertRaises(TypeError):
       expressions.ComputedExpression('add', lambda a, b: a + b, [a, b])
 
+  def test_output_partitioning_preserves_singleton(self):

Review comment:
       This reads like "test that output partitioning preserves singleton partitioning." Maybe rename to `test_preserves_singleton_ouptput_partitioning`. Similarly elsewhere. 

##########
File path: sdks/python/apache_beam/dataframe/expressions_test.py
##########
@@ -53,6 +54,61 @@ def test_expression_proxy_error(self):
     with self.assertRaises(TypeError):
       expressions.ComputedExpression('add', lambda a, b: a + b, [a, b])
 
+  def test_output_partitioning_preserves_singleton(self):
+    a = expressions.PlaceholderExpression(1)
+    b = expressions.PlaceholderExpression(2)
+
+    preserves_singleton = expressions.ComputedExpression(
+        'add',
+        lambda a,

Review comment:
       Put this in ()'s or suppress yapf. (Or maybe just pass None, or a dummy `lambda: None`)

##########
File path: sdks/python/apache_beam/dataframe/expressions.py
##########
@@ -312,6 +352,11 @@ def __init__(
       requires_partition_by: The required (common) partitioning of the args.
       preserves_partition_by: The level of partitioning preserved.
     """
+    assert preserves_partition_by != partitionings.Nothing(), (
+        "Preserving Nothing() partitioning is not allowed. Any expression "
+        "can trivially preserve at least Singleton() partitioning. If you "
+        "intend to indicate this expression can preserve _any_ partioning, use "
+        "Index() instead.")

Review comment:
       What if we renamed Nothing to Arbitrary. Then it would make sense to preserve Arbitrary partitioning, and also require Arbitrary partitioning. 

##########
File path: sdks/python/apache_beam/dataframe/transforms.py
##########
@@ -331,8 +339,7 @@ def expr_to_stages(expr):
       required_partitioning = expr.requires_partition_by()
       for stage in common_stages([expr_to_stages(arg) for arg in expr.args()
                                   if arg not in inputs]):
-        if all(output_is_partitioned_by(arg, stage, required_partitioning)
-               for arg in expr.args() if not is_scalar(arg)):
+        if output_partitioning_in_stage(expr, stage) is not None:

Review comment:
       +1 to combining the logic. Maybe a helper `is_computable_in_stage(expr, stage)` whose implementation is `output_partitioning_in_stage(expr, stage) is not None` would make this easier to follow. 

##########
File path: sdks/python/apache_beam/dataframe/expressions.py
##########
@@ -312,6 +352,11 @@ def __init__(
       requires_partition_by: The required (common) partitioning of the args.
       preserves_partition_by: The level of partitioning preserved.
     """
+    assert preserves_partition_by != partitionings.Nothing(), (
+        "Preserving Nothing() partitioning is not allowed. Any expression "
+        "can trivially preserve at least Singleton() partitioning. If you "
+        "intend to indicate this expression can preserve _any_ partioning, use "
+        "Index() instead.")

Review comment:
       This assumes we will never have anything between Index and Nothing. Maybe that's a fair assumption. 

##########
File path: sdks/python/apache_beam/dataframe/transforms.py
##########
@@ -282,30 +282,38 @@ def __repr__(self, indent=0):
                 self.outputs))
 
     # First define some helper functions.
-    def output_is_partitioned_by(expr, stage, partitioning):
-      if partitioning == partitionings.Nothing():
-        # Always satisfied.
-        return True
-      elif stage.partitioning == partitionings.Singleton():
-        # Within a stage, the singleton partitioning is trivially preserved.
-        return True
-      elif expr in stage.inputs:
+    def output_partitioning_in_stage(expr, stage):
+      """ Return the output partitioning of expr when computed in stage,

Review comment:
       Remove leading space.

##########
File path: sdks/python/apache_beam/dataframe/transforms.py
##########
@@ -282,30 +282,38 @@ def __repr__(self, indent=0):
                 self.outputs))
 
     # First define some helper functions.
-    def output_is_partitioned_by(expr, stage, partitioning):
-      if partitioning == partitionings.Nothing():
-        # Always satisfied.
-        return True
-      elif stage.partitioning == partitionings.Singleton():
-        # Within a stage, the singleton partitioning is trivially preserved.
-        return True
-      elif expr in stage.inputs:
+    def output_partitioning_in_stage(expr, stage):
+      """ Return the output partitioning of expr when computed in stage,
+      or returns None if the expression cannot be computed in this stage.
+      """
+      if expr in stage.inputs or expr in inputs:
         # Inputs are all partitioned by stage.partitioning.
-        return stage.partitioning.is_subpartitioning_of(partitioning)
-      elif expr.preserves_partition_by().is_subpartitioning_of(partitioning):
-        # Here expr preserves at least the requested partitioning; its outputs
-        # will also have this partitioning iff its inputs do.
-        if expr.requires_partition_by().is_subpartitioning_of(partitioning):
-          # If expr requires at least this partitioning, we will arrange such
-          # that its inputs satisfy this.
-          return True
-        else:
-          # Otherwise, recursively check all the inputs.
-          return all(
-              output_is_partitioned_by(arg, stage, partitioning)
-              for arg in expr.args())
-      else:
-        return False
+        return stage.partitioning
+
+      # Anything that's not an input must have arguments
+      assert len(expr.args())
+
+      arg_partitionings = set(
+          output_partitioning_in_stage(arg, stage) for arg in expr.args()
+          if not is_scalar(arg))
+
+      # TODO: what does it mean for all args to be 0? only scalar arguments? the

Review comment:
       len(arg_partitionings) to be 0? Yes, I think in that case this should be a scalar. Did we not run into this case? 

##########
File path: sdks/python/apache_beam/dataframe/partitionings_test.py
##########
@@ -36,8 +36,8 @@ class PartitioningsTest(unittest.TestCase):
   }).set_index(['shape', 'color', 'size'])
 
   def test_index_is_subpartition(self):
-    ordered_list = [Nothing(), Index([3]), Index([1, 3]), Index(), Singleton()]
-    for loose, strict in zip(ordered_list[:1], ordered_list[1:]):
+    ordered_list = [Singleton(), Index([3]), Index([1, 3]), Index(), Nothing()]
+    for loose, strict in zip(ordered_list[:-1], ordered_list[1:]):

Review comment:
       Ack.

##########
File path: sdks/python/apache_beam/dataframe/expressions_test.py
##########
@@ -53,6 +54,61 @@ def test_expression_proxy_error(self):
     with self.assertRaises(TypeError):
       expressions.ComputedExpression('add', lambda a, b: a + b, [a, b])
 
+  def test_output_partitioning_preserves_singleton(self):
+    a = expressions.PlaceholderExpression(1)
+    b = expressions.PlaceholderExpression(2)
+
+    preserves_singleton = expressions.ComputedExpression(
+        'add',
+        lambda a,
+        b: a + b, [a, b],
+        requires_partition_by=partitionings.Nothing(),
+        preserves_partition_by=partitionings.Singleton())
+
+    for partitioning in (partitionings.Singleton(), ):
+      self.assertEqual(
+          expressions.output_partitioning(preserves_singleton, partitioning),
+          partitioning,
+          f"Should preserve {partitioning}")
+
+    for partitioning in (partitionings.Index([0]),
+                         partitionings.Index(),
+                         partitionings.Nothing()):
+      self.assertEqual(
+          expressions.output_partitioning(preserves_singleton, partitioning),
+          partitionings.Nothing(),
+          f"Should NOT preserve {partitioning}")
+
+  def test_output_partitioning_preserves_index(self):
+    a = expressions.PlaceholderExpression(1)
+    b = expressions.PlaceholderExpression(2)
+
+    preserves_index = expressions.ComputedExpression(

Review comment:
       `preserves_partial_index`? Also, an example would be helpful (e.g. something that replaces level 2 of the index with a column).

##########
File path: sdks/python/apache_beam/dataframe/expressions_test.py
##########
@@ -53,6 +54,61 @@ def test_expression_proxy_error(self):
     with self.assertRaises(TypeError):
       expressions.ComputedExpression('add', lambda a, b: a + b, [a, b])
 
+  def test_output_partitioning_preserves_singleton(self):
+    a = expressions.PlaceholderExpression(1)
+    b = expressions.PlaceholderExpression(2)
+
+    preserves_singleton = expressions.ComputedExpression(

Review comment:
       Maybe call this `preserves_only_singleton`? And give an example (like setting the index from a column). 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] robertwb commented on a change in pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

robertwb commented on a change in pull request #14135:
URL: https://github.com/apache/beam/pull/14135#discussion_r592554780



##########
File path: sdks/python/apache_beam/dataframe/transforms.py
##########
@@ -282,30 +282,38 @@ def __repr__(self, indent=0):
                 self.outputs))
 
     # First define some helper functions.
-    def output_is_partitioned_by(expr, stage, partitioning):
-      if partitioning == partitionings.Nothing():
-        # Always satisfied.
-        return True
-      elif stage.partitioning == partitionings.Singleton():
-        # Within a stage, the singleton partitioning is trivially preserved.
-        return True
-      elif expr in stage.inputs:
+    def output_partitioning_in_stage(expr, stage):
+      """ Return the output partitioning of expr when computed in stage,
+      or returns None if the expression cannot be computed in this stage.
+      """
+      if expr in stage.inputs or expr in inputs:
         # Inputs are all partitioned by stage.partitioning.
-        return stage.partitioning.is_subpartitioning_of(partitioning)
-      elif expr.preserves_partition_by().is_subpartitioning_of(partitioning):
-        # Here expr preserves at least the requested partitioning; its outputs
-        # will also have this partitioning iff its inputs do.
-        if expr.requires_partition_by().is_subpartitioning_of(partitioning):
-          # If expr requires at least this partitioning, we will arrange such
-          # that its inputs satisfy this.
-          return True
-        else:
-          # Otherwise, recursively check all the inputs.
-          return all(
-              output_is_partitioned_by(arg, stage, partitioning)
-              for arg in expr.args())
-      else:
-        return False
+        return stage.partitioning
+
+      # Anything that's not an input must have arguments
+      assert len(expr.args())
+
+      arg_partitionings = set(
+          output_partitioning_in_stage(arg, stage) for arg in expr.args()
+          if not is_scalar(arg))
+
+      # TODO: what does it mean for all args to be 0? only scalar arguments? the

Review comment:
       Sounds good for now.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (5ba8319) into [master](https://codecov.io/gh/apache/beam/commit/08a9d54b60cd4743afeb61e697fa6016279b1ac6?el=desc) (08a9d54) will **increase** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.17%   +0.02%     
   ==========================================
     Files         469      469              
     Lines       58972    58989      +17     
   ==========================================
   + Hits        49037    49066      +29     
   + Misses       9935     9923      -12     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ython/apache\_beam/examples/kafkataxi/kafka\_taxi.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMva2Fma2F0YXhpL2thZmthX3RheGkucHk=) | | |
   | [...am/examples/snippets/transforms/aggregation/top.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi90b3AucHk=) | | |
   | [...ers/interactive/display/pipeline\_graph\_renderer.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3BpcGVsaW5lX2dyYXBoX3JlbmRlcmVyLnB5) | | |
   | [...teractive/testing/integration/notebook\_executor.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS90ZXN0aW5nL2ludGVncmF0aW9uL25vdGVib29rX2V4ZWN1dG9yLnB5) | | |
   | [.../sdks/python/apache\_beam/examples/snippets/util.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdXRpbC5weQ==) | | |
   | [...38/build/srcs/sdks/python/apache\_beam/io/fileio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZWlvLnB5) | | |
   | [...thon/apache\_beam/runners/worker/channel\_factory.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvY2hhbm5lbF9mYWN0b3J5LnB5) | | |
   | [...on/apache\_beam/runners/worker/statesampler\_slow.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc3RhdGVzYW1wbGVyX3Nsb3cucHk=) | | |
   | [...pache\_beam/runners/portability/artifact\_service.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9hcnRpZmFjdF9zZXJ2aWNlLnB5) | | |
   | [...y38/build/srcs/sdks/python/apache\_beam/io/vcfio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdmNmaW8ucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [0ad65c6...5ba8319](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (29f41c0) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   - Coverage   83.04%   83.02%   -0.02%     
   ==========================================
     Files         469      469              
     Lines       58473    58482       +9     
   ==========================================
   - Hits        48561    48557       -4     
   - Misses       9912     9925      +13     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...s/python/apache\_beam/testing/synthetic\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9zeW50aGV0aWNfcGlwZWxpbmUucHk=) | | |
   | [.../runners/portability/fn\_api\_runner/translations.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3RyYW5zbGF0aW9ucy5weQ==) | | |
   | [...ild/srcs/sdks/python/apache\_beam/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWV0cmljcy9tZXRyaWMucHk=) | | |
   | [...ld/srcs/sdks/python/apache\_beam/utils/histogram.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaGlzdG9ncmFtLnB5) | | |
   | [.../apache\_beam/options/pipeline\_options\_validator.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vb3B0aW9ucy9waXBlbGluZV9vcHRpb25zX3ZhbGlkYXRvci5weQ==) | | |
   | [...rcs/sdks/python/apache\_beam/runners/job/manager.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9qb2IvbWFuYWdlci5weQ==) | | |
   | [...am/examples/snippets/transforms/elementwise/map.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS9tYXAucHk=) | | |
   | [...ld/srcs/sdks/python/apache\_beam/io/gcp/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL19faW5pdF9fLnB5) | | |
   | [...python/apache\_beam/io/gcp/experimental/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2V4cGVyaW1lbnRhbC9fX2luaXRfXy5weQ==) | | |
   | [...on/apache\_beam/ml/gcp/naturallanguageml\_test\_it.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvZ2NwL25hdHVyYWxsYW5ndWFnZW1sX3Rlc3RfaXQucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...d8eb470](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (d8eb470) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14135   +/-   ##
   =======================================
     Coverage   83.04%   83.05%           
   =======================================
     Files         469      469           
     Lines       58473    58482    +9     
   =======================================
   + Hits        48561    48574   +13     
   + Misses       9912     9908    -4     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...am/testing/benchmarks/chicago\_taxi/trainer/task.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL2NoaWNhZ29fdGF4aS90cmFpbmVyL3Rhc2sucHk=) | | |
   | [...s/python/apache\_beam/ml/gcp/videointelligenceml.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvZ2NwL3ZpZGVvaW50ZWxsaWdlbmNlbWwucHk=) | | |
   | [...xamples/snippets/transforms/elementwise/flatmap.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS9mbGF0bWFwLnB5) | | |
   | [...build/srcs/sdks/python/apache\_beam/io/parquetio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vcGFycXVldGlvLnB5) | | |
   | [...pache\_beam/examples/cookbook/bigquery\_tornadoes.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svYmlncXVlcnlfdG9ybmFkb2VzLnB5) | | |
   | [.../sdks/python/apache\_beam/internal/metrics/cells.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9jZWxscy5weQ==) | | |
   | [...\_beam/io/gcp/datastore/v1new/adaptive\_throttler.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9hZGFwdGl2ZV90aHJvdHRsZXIucHk=) | | |
   | [...s/snippets/transforms/aggregation/combineperkey.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9jb21iaW5lcGVya2V5LnB5) | | |
   | [...beam/testing/benchmarks/nexmark/queries/query10.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTEwLnB5) | | |
   | [...ild/srcs/sdks/python/apache\_beam/io/gcp/spanner.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3NwYW5uZXIucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...d8eb470](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (d8eb470) into [master](https://codecov.io/gh/apache/beam/commit/2e9e9a8e2815458eee382bbef64facea05c90eca?el=desc) (2e9e9a8) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14135   +/-   ##
   =======================================
     Coverage   83.04%   83.05%           
   =======================================
     Files         469      469           
     Lines       58473    58482    +9     
   =======================================
   + Hits        48561    48574   +13     
   + Misses       9912     9908    -4     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...am/examples/snippets/transforms/aggregation/min.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9taW4ucHk=) | | |
   | [...\_beam/io/gcp/datastore/v1new/adaptive\_throttler.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9hZGFwdGl2ZV90aHJvdHRsZXIucHk=) | | |
   | [...ild/srcs/sdks/python/apache\_beam/io/filesystems.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZXN5c3RlbXMucHk=) | | |
   | [.../sdks/python/apache\_beam/examples/snippets/util.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdXRpbC5weQ==) | | |
   | [...n/apache\_beam/typehints/typed\_pipeline\_test\_py3.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3R5cGVkX3BpcGVsaW5lX3Rlc3RfcHkzLnB5) | | |
   | [...s/sdks/python/apache\_beam/testing/test\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3BpcGVsaW5lLnB5) | | |
   | [...nternal/clients/cloudbuild/cloudbuild\_v1\_client.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9jbGllbnRzL2Nsb3VkYnVpbGQvY2xvdWRidWlsZF92MV9jbGllbnQucHk=) | | |
   | [...python/apache\_beam/examples/complete/distribopt.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvZGlzdHJpYm9wdC5weQ==) | | |
   | [...ers/dataflow/dataflow\_exercise\_metrics\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19leGVyY2lzZV9tZXRyaWNzX3BpcGVsaW5lLnB5) | | |
   | [...s/sdks/python/apache\_beam/io/gcp/bigquery\_tools.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3Rvb2xzLnB5) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [2e9e9a8...0446559](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on a change in pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

TheNeuralBit commented on a change in pull request #14135:
URL: https://github.com/apache/beam/pull/14135#discussion_r591707315



##########
File path: sdks/python/apache_beam/dataframe/expressions.py
##########
@@ -245,7 +285,7 @@ def requires_partition_by(self):
     return partitionings.Nothing()
 
   def preserves_partition_by(self):
-    return partitionings.Nothing()
+    return partitionings.Index()

Review comment:
       Yeah my motivation here was just that `preserves=Nothing()` is potentially confusing. In retrospect maybe it's not much of a concern since `PartitioningSession` should catch incorrect specifications. Regardless - I like your idea (in the next comment) to solve this with naming.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

TheNeuralBit commented on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-790110226


   Run PythonDocker PreCommit


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on a change in pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

TheNeuralBit commented on a change in pull request #14135:
URL: https://github.com/apache/beam/pull/14135#discussion_r591984531



##########
File path: sdks/python/apache_beam/dataframe/expressions_test.py
##########
@@ -53,6 +54,61 @@ def test_expression_proxy_error(self):
     with self.assertRaises(TypeError):
       expressions.ComputedExpression('add', lambda a, b: a + b, [a, b])
 
+  def test_output_partitioning_preserves_singleton(self):
+    a = expressions.PlaceholderExpression(1)
+    b = expressions.PlaceholderExpression(2)
+
+    preserves_singleton = expressions.ComputedExpression(
+        'add',
+        lambda a,

Review comment:
       Made these into real functions that yapf doesn't mess up.

##########
File path: sdks/python/apache_beam/dataframe/expressions_test.py
##########
@@ -53,6 +54,61 @@ def test_expression_proxy_error(self):
     with self.assertRaises(TypeError):
       expressions.ComputedExpression('add', lambda a, b: a + b, [a, b])
 
+  def test_output_partitioning_preserves_singleton(self):
+    a = expressions.PlaceholderExpression(1)
+    b = expressions.PlaceholderExpression(2)
+
+    preserves_singleton = expressions.ComputedExpression(

Review comment:
       Done

##########
File path: sdks/python/apache_beam/dataframe/expressions_test.py
##########
@@ -53,6 +54,61 @@ def test_expression_proxy_error(self):
     with self.assertRaises(TypeError):
       expressions.ComputedExpression('add', lambda a, b: a + b, [a, b])
 
+  def test_output_partitioning_preserves_singleton(self):
+    a = expressions.PlaceholderExpression(1)
+    b = expressions.PlaceholderExpression(2)
+
+    preserves_singleton = expressions.ComputedExpression(
+        'add',
+        lambda a,
+        b: a + b, [a, b],
+        requires_partition_by=partitionings.Nothing(),
+        preserves_partition_by=partitionings.Singleton())
+
+    for partitioning in (partitionings.Singleton(), ):
+      self.assertEqual(
+          expressions.output_partitioning(preserves_singleton, partitioning),
+          partitioning,
+          f"Should preserve {partitioning}")
+
+    for partitioning in (partitionings.Index([0]),
+                         partitionings.Index(),
+                         partitionings.Nothing()):
+      self.assertEqual(
+          expressions.output_partitioning(preserves_singleton, partitioning),
+          partitionings.Nothing(),
+          f"Should NOT preserve {partitioning}")
+
+  def test_output_partitioning_preserves_index(self):
+    a = expressions.PlaceholderExpression(1)
+    b = expressions.PlaceholderExpression(2)
+
+    preserves_index = expressions.ComputedExpression(

Review comment:
       Done

##########
File path: sdks/python/apache_beam/dataframe/transforms.py
##########
@@ -282,30 +282,38 @@ def __repr__(self, indent=0):
                 self.outputs))
 
     # First define some helper functions.
-    def output_is_partitioned_by(expr, stage, partitioning):
-      if partitioning == partitionings.Nothing():
-        # Always satisfied.
-        return True
-      elif stage.partitioning == partitionings.Singleton():
-        # Within a stage, the singleton partitioning is trivially preserved.
-        return True
-      elif expr in stage.inputs:
+    def output_partitioning_in_stage(expr, stage):
+      """ Return the output partitioning of expr when computed in stage,

Review comment:
       Done




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (5ba8319) into [master](https://codecov.io/gh/apache/beam/commit/08a9d54b60cd4743afeb61e697fa6016279b1ac6?el=desc) (08a9d54) will **increase** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.17%   +0.02%     
   ==========================================
     Files         469      469              
     Lines       58972    58989      +17     
   ==========================================
   + Hits        49037    49066      +29     
   + Misses       9935     9923      -12     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ython/apache\_beam/examples/kafkataxi/kafka\_taxi.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMva2Fma2F0YXhpL2thZmthX3RheGkucHk=) | | |
   | [...am/examples/snippets/transforms/aggregation/top.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi90b3AucHk=) | | |
   | [...ers/interactive/display/pipeline\_graph\_renderer.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3BpcGVsaW5lX2dyYXBoX3JlbmRlcmVyLnB5) | | |
   | [...teractive/testing/integration/notebook\_executor.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS90ZXN0aW5nL2ludGVncmF0aW9uL25vdGVib29rX2V4ZWN1dG9yLnB5) | | |
   | [.../sdks/python/apache\_beam/examples/snippets/util.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdXRpbC5weQ==) | | |
   | [...38/build/srcs/sdks/python/apache\_beam/io/fileio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZWlvLnB5) | | |
   | [...thon/apache\_beam/runners/worker/channel\_factory.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvY2hhbm5lbF9mYWN0b3J5LnB5) | | |
   | [...on/apache\_beam/runners/worker/statesampler\_slow.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc3RhdGVzYW1wbGVyX3Nsb3cucHk=) | | |
   | [...pache\_beam/runners/portability/artifact\_service.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9hcnRpZmFjdF9zZXJ2aWNlLnB5) | | |
   | [...y38/build/srcs/sdks/python/apache\_beam/io/vcfio.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdmNmaW8ucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [0ad65c6...5ba8319](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14135: [BEAM-11881] Fix partitioning ordering for DataFrames

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #14135:
URL: https://github.com/apache/beam/pull/14135#issuecomment-789406058


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=h1) Report
   > Merging [#14135](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=desc) (c54cc08) into [master](https://codecov.io/gh/apache/beam/commit/1ad764b33d91bc78af54a53babf73b59aede04ad?el=desc) (1ad764b) will **increase** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14135/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #14135      +/-   ##
   ==========================================
   + Coverage   83.15%   83.18%   +0.02%     
   ==========================================
     Files         469      469              
     Lines       58954    58969      +15     
   ==========================================
   + Hits        49024    49054      +30     
   + Misses       9930     9915      -15     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...s/python/apache\_beam/testing/pipeline\_verifiers.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9waXBlbGluZV92ZXJpZmllcnMucHk=) | | |
   | [.../srcs/sdks/python/apache\_beam/internal/gcp/auth.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvZ2NwL2F1dGgucHk=) | | |
   | [...cs/sdks/python/apache\_beam/typehints/decorators.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL2RlY29yYXRvcnMucHk=) | | |
   | [...nternal/clients/dataflow/dataflow\_v1b3\_messages.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9jbGllbnRzL2RhdGFmbG93L2RhdGFmbG93X3YxYjNfbWVzc2FnZXMucHk=) | | |
   | [...examples/snippets/transforms/aggregation/sample.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9zYW1wbGUucHk=) | | |
   | [...cs/sdks/python/apache\_beam/portability/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvX19pbml0X18ucHk=) | | |
   | [...build/srcs/sdks/python/apache\_beam/utils/shared.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc2hhcmVkLnB5) | | |
   | [...ks/python/apache\_beam/runners/worker/sideinputs.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2lkZWlucHV0cy5weQ==) | | |
   | [...beam/runners/interactive/display/pipeline\_graph.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3BpcGVsaW5lX2dyYXBoLnB5) | | |
   | [...thon/apache\_beam/io/aws/clients/s3/boto3\_client.py](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXdzL2NsaWVudHMvczMvYm90bzNfY2xpZW50LnB5) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14135/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=footer). Last update [1ad764b...c54cc08](https://codecov.io/gh/apache/beam/pull/14135?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org