You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/03/11 16:49:00 UTC
[jira] [Work logged] (BEAM-2888) Runner Comparison / Capability
Matrix revamp
[ https://issues.apache.org/jira/browse/BEAM-2888?focusedWorklogId=564734&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-564734 ]
ASF GitHub Bot logged work on BEAM-2888:
----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Mar/21 16:48
Start Date: 11/Mar/21 16:48
Worklog Time Spent: 10m
Work Description: codecov[bot] commented on pull request #13492:
URL: https://github.com/apache/beam/pull/13492#issuecomment-796880041
# [Codecov](https://codecov.io/gh/apache/beam/pull/13492?src=pr&el=h1) Report
> Merging [#13492](https://codecov.io/gh/apache/beam/pull/13492?src=pr&el=desc) (bc91fa1) into [master](https://codecov.io/gh/apache/beam/commit/10f5cbc6d7eb6de41c24b13471390588b1dd2b1c?el=desc) (10f5cbc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/13492?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #13492 +/- ##
==========================================
- Coverage 83.15% 83.14% -0.01%
==========================================
Files 469 469
Lines 58914 58914
==========================================
- Hits 48989 48986 -3
- Misses 9925 9928 +3
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/13492?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...s/sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy90ZXN0L19faW5pdF9fLnB5) | | |
| [...dks/python/apache\_beam/options/pipeline\_options.py](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vb3B0aW9ucy9waXBlbGluZV9vcHRpb25zLnB5) | | |
| [...\_beam/io/gcp/datastore/v1new/adaptive\_throttler.py](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9hZGFwdGl2ZV90aHJvdHRsZXIucHk=) | | |
| [.../apache\_beam/testing/benchmarks/nexmark/monitor.py](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbW9uaXRvci5weQ==) | | |
| [...dks/python/apache\_beam/examples/wordcount\_xlang.py](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X3hsYW5nLnB5) | | |
| [...on/apache\_beam/io/gcp/bigquery\_io\_read\_pipeline.py](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X2lvX3JlYWRfcGlwZWxpbmUucHk=) | | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_perf.py](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19wZXJmLnB5) | | |
| [...ython/apache\_beam/io/gcp/tests/bigquery\_matcher.py](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3Rlc3RzL2JpZ3F1ZXJ5X21hdGNoZXIucHk=) | | |
| [...pache\_beam/portability/api/beam\_fn\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZm5fYXBpX3BiMl9ncnBjLnB5) | | |
| [.../srcs/sdks/python/apache\_beam/internal/gcp/auth.py](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvZ2NwL2F1dGgucHk=) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/13492/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13492?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13492?src=pr&el=footer). Last update [10f5cbc...bc91fa1](https://codecov.io/gh/apache/beam/pull/13492?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 564734)
Time Spent: 3h 20m (was: 3h 10m)
> Runner Comparison / Capability Matrix revamp
> --------------------------------------------
>
> Key: BEAM-2888
> URL: https://issues.apache.org/jira/browse/BEAM-2888
> Project: Beam
> Issue Type: Improvement
> Components: website
> Reporter: Kenneth Knowles
> Priority: P3
> Labels: gsod, gsod2019
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> Discussion: https://lists.apache.org/thread.html/8aff7d70c254356f2dae3109fb605e0b60763602225a877d3dadf8b7@%3Cdev.beam.apache.org%3E
> Summarizing that discussion, we have a lot of issues/wishes. Some can be addressed as one-off and some need a unified reorganization of the runner comparison.
> Basic corrections:
> - Remove rows that impossible to not support (ParDo)
> - Remove rows where "support" doesn't really make sense (Composite transforms)
> - Deduplicate rows are actually the same model feature (all non-merging windowing / all merging windowing)
> - Clearly separate rows that represent optimizations (Combine)
> - Correct rows in the wrong place (Timers are actually a "what...?" row)
> - Separate or remove rows have not been designed ([Meta]Data driven triggers, retractions)
> - Rename rows with names that appear no where else (Timestamp control, which is called a TimestampCombiner in Java)
> - Switch to a more distinct color scheme for full/partial support (currently just solid/faded colors)
> - Switch to something clearer than "~" for partial support, versus ✘ and ✓ for none and full.
> - Correct Gearpump support for merging windows (see BEAM-2759)
> - Correct Spark support for non-merging and merging windows (see BEAM-2499)
> Minor rewrites:
> - Lump all the basic stuff (ParDo, GroupByKey, Read, Window) into one row
> - Make sections as users see them, like "ParDo" / "side Inputs" not "What?" / "side inputs"
> - Add rows for non-model things, like portability framework support, metrics backends, etc
> Bigger rewrites:
> - Add versioning to the comparison, as in BEAM-166
> - Find a way to fit in a plain English summary of runner's support in Beam. It should come first, as it is what new users need before getting to details.
> - Find a way to describe production readiness of runners and/or testimonials of who is using it in production.
> - Have a place to compare non-model differences between runners
> Changes requiring engineering efforts:
> - Gather and add quantitative runner metrics, perhaps Nexmark results for mid-level, smaller benchmarks for measuring aspects of specific features, and larger end-to-end benchmarks to get an idea how it might actually perform on a use case
> - Tighter coupling of the matrix portion of the comparison with tags on ValidatesRunner tests
> If you care to address some aspect of this, please reach out and/or just file a subtask and address it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)