[jira] [Updated] (BEAM-2888) Runner Comparison / Capability Matrix revamp

["Pablo Estrada (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pablo Estrada updated BEAM-2888:
--------------------------------
    Labels: gsoc2022 gsod gsod2019 gsod2022 mentor  (was: gsod gsod2019 gsod2022 mentor)

> Runner Comparison / Capability Matrix revamp
> --------------------------------------------
>
>                 Key: BEAM-2888
>                 URL: https://issues.apache.org/jira/browse/BEAM-2888
>             Project: Beam
>          Issue Type: Improvement
>          Components: website
>            Reporter: Kenneth Knowles
>            Priority: P3
>              Labels: gsoc2022, gsod, gsod2019, gsod2022, mentor
>          Time Spent: 19h
>  Remaining Estimate: 0h
>
> Discussion: https://lists.apache.org/thread.html/8aff7d70c254356f2dae3109fb605e0b60763602225a877d3dadf8b7@%3Cdev.beam.apache.org%3E
> Summarizing that discussion, we have a lot of issues/wishes. Some can be addressed as one-off and some need a unified reorganization of the runner comparison.
> Basic corrections:
>  - Remove rows that impossible to not support (ParDo)
>  - Remove rows where "support" doesn't really make sense (Composite transforms)
>  - Deduplicate rows are actually the same model feature (all non-merging windowing / all merging windowing)
>  - Clearly separate rows that represent optimizations (Combine)
>  - Correct rows in the wrong place (Timers are actually a "what...?" row)
>  - Separate or remove rows have not been designed ([Meta]Data driven triggers, retractions)
>  - Rename rows with names that appear no where else (Timestamp control, which is called a TimestampCombiner in Java)
>  - Switch to a more distinct color scheme for full/partial support (currently just solid/faded colors)
>  - Switch to something clearer than "~" for partial support, versus ✘ and ✓ for none and full.
>  - Correct Gearpump support for merging windows (see BEAM-2759)
>  - Correct Spark support for non-merging and merging windows (see BEAM-2499)
> Minor rewrites:
>  - Lump all the basic stuff (ParDo, GroupByKey, Read, Window) into one row
>  - Make sections as users see them, like "ParDo" / "side Inputs" not "What?" / "side inputs"
>  - Add rows for non-model things, like portability framework support, metrics backends, etc
> Bigger rewrites:
>  - Add versioning to the comparison, as in BEAM-166
>  - Find a way to fit in a plain English summary of runner's support in Beam. It should come first, as it is what new users need before getting to details.
>  - Find a way to describe production readiness of runners and/or testimonials of who is using it in production.
>  - Have a place to compare non-model differences between runners
> Changes requiring engineering efforts:
>  - Gather and add quantitative runner metrics, perhaps Nexmark results for mid-level, smaller benchmarks for measuring aspects of specific features, and larger end-to-end benchmarks to get an idea how it might actually perform on a use case
>  - Tighter coupling of the matrix portion of the comparison with tags on ValidatesRunner tests
> If you care to address some aspect of this, please reach out and/or just file a subtask and address it.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)