You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by da...@apache.org on 2025/09/05 15:05:32 UTC
(beam) branch users/damccorm/prism-capabilities created (now bd13ac405bb)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a change to branch users/damccorm/prism-capabilities
in repository https://gitbox.apache.org/repos/asf/beam.git
at bd13ac405bb Add prism capability matrix
This branch includes the following new commits:
new bd13ac405bb Add prism capability matrix
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(beam) 01/01: Add prism capability matrix
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/prism-capabilities
in repository https://gitbox.apache.org/repos/asf/beam.git
commit bd13ac405bb98dcc13c0da6ddd3acd9ff68feb70
Author: Danny Mccormick <da...@google.com>
AuthorDate: Fri Sep 5 11:05:13 2025 -0400
Add prism capability matrix
---
website/www/site/data/capability_matrix.yaml | 158 +++++++++++++++++++++++++++
1 file changed, 158 insertions(+)
diff --git a/website/www/site/data/capability_matrix.yaml b/website/www/site/data/capability_matrix.yaml
index e6fd51a9bb1..c1da306b9cb 100644
--- a/website/www/site/data/capability_matrix.yaml
+++ b/website/www/site/data/capability_matrix.yaml
@@ -14,6 +14,8 @@ capability-matrix:
columns:
- class: dataflow
name: Google Cloud Dataflow
+ - class: prism
+ name: Prism Local Runner
- class: flink
name: Apache Flink
- class: spark-rdd
@@ -50,6 +52,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: Batch mode uses large bundle sizes. Streaming uses smaller bundle sizes.
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: Supported with per-element transformation.
- class: flink
l1: "Yes"
l2: fully supported
@@ -93,6 +99,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: fully supported
@@ -136,6 +146,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: fully supported
@@ -179,6 +193,10 @@ capability-matrix:
l1: "Yes"
l2: "efficient execution"
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: "fully supported"
@@ -222,6 +240,10 @@ capability-matrix:
l1: "Partially"
l2: supported via inlining
l3: Currently composite transformations are inlined during execution. The structure is later recreated from the names, but other transform level information (if added to the model) will be lost.
+ - class: prism
+ l1: "Yes"
+ l2: supported via inlining
+ l3: ""
- class: flink
l1: "Partially"
l2: supported via inlining
@@ -265,6 +287,10 @@ capability-matrix:
l1: "Yes"
l2: some size restrictions in streaming
l3: Batch mode supports a distributed implementation, but streaming mode may force some size restrictions. Neither mode is able to push lookups directly up into key-based sources.
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: some size restrictions in streaming
@@ -308,6 +334,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: Support includes autotuning features (https://cloud.google.com/dataflow/service/dataflow-service-desc#autotuning-features).
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: fully supported
@@ -351,6 +381,10 @@ capability-matrix:
l1: "Partially"
l2: ""
l3: Gauge metrics are not supported. All other metric types are supported.
+ - class: prism
+ l1: "Partially"
+ l2: ""
+ l3: StringSet metrics are not supported. All other metric types are supported.
- class: flink
l1: "Partially"
l2: All metrics types are supported.
@@ -394,6 +428,10 @@ capability-matrix:
l1: "Partially"
l2: non-merging windows
l3: "State is supported for non-merging windows. The MapState, SetState, and MultimapState state types are supported in the following scenarios: Java pipelines that don't use Streaming Engine; Java pipelines that use Streaming Engine and version 2.58.0 or later of the Java SDK. SetState, MapState, and MultimapState are not supported for pipelines that use Runner v2."
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Partially"
l2: non-merging windows
@@ -446,6 +484,10 @@ capability-matrix:
l1: "Partially"
l2: Only Dataflow Runner V2 supports this.
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Partially"
l2: Only portable Flink Runner supports this.
@@ -489,6 +531,10 @@ capability-matrix:
l1: "Partially"
l2: Only Dataflow Runner V2 supports this.
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Partially"
l2: Only portable Flink Runner supports this.
@@ -532,6 +578,10 @@ capability-matrix:
l1: "Partially"
l2: Only Dataflow Runner v2 supports this.
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Partially"
l2: Only portable Flink Runner supports this.
@@ -575,6 +625,10 @@ capability-matrix:
l1: "Partially"
l2: Only Dataflow Runner V2 supports this.
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "No"
l2:
@@ -618,6 +672,10 @@ capability-matrix:
l1: "Partially"
l2: Only Dataflow Runner V2 supports this.
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "No"
l2:
@@ -670,6 +728,10 @@ capability-matrix:
l1: "Yes"
l2:
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Partially"
l2:
@@ -713,6 +775,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "No"
l2:
@@ -756,6 +822,10 @@ capability-matrix:
l1: "Yes"
l2:
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Partially"
l2:
@@ -799,6 +869,10 @@ capability-matrix:
l1: "No"
l2:
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "No"
l2:
@@ -842,6 +916,10 @@ capability-matrix:
l1: "Partially"
l2: Only Dataflow Runner V2 supports this.
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "No"
l2:
@@ -894,6 +972,10 @@ capability-matrix:
l1: "Yes"
l2: default
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: default
+ l3: ""
- class: flink
l1: "Yes"
l2: supported
@@ -929,6 +1011,10 @@ capability-matrix:
l1: "Yes"
l2: built-in
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: supported
@@ -964,6 +1050,10 @@ capability-matrix:
l1: "Yes"
l2: built-in
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: supported
@@ -999,6 +1089,10 @@ capability-matrix:
l1: "Yes"
l2: built-in
l3: ""
+ - class: prism
+ l1: "No"
+ l2:
+ l3:
- class: flink
l1: "Yes"
l2: supported
@@ -1034,6 +1128,10 @@ capability-matrix:
l1: "Yes"
l2: supported
l3: ""
+ - class: prism
+ l1: "No"
+ l2:
+ l3:
- class: flink
l1: "Yes"
l2: supported
@@ -1069,6 +1167,10 @@ capability-matrix:
l1: "Yes"
l2: supported
l3: ""
+ - class: prism
+ l1: "No"
+ l2:
+ l3:
- class: flink
l1: "Yes"
l2: supported
@@ -1104,6 +1206,10 @@ capability-matrix:
l1: "Yes"
l2: supported
l3: ""
+ - class: prism
+ l1: "No"
+ l2: "Not supported"
+ l3: ""
- class: flink
l1: "Yes"
l2: supported
@@ -1149,6 +1255,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: Fully supported in streaming mode. In batch mode, intermediate trigger firings are effectively meaningless.
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: fully supported
@@ -1185,6 +1295,10 @@ capability-matrix:
l1: "Yes"
l2: yes in streaming, fixed granularity in batch
l3: Fully supported in streaming mode. In batch mode, currently watermark progress jumps from the beginning of time to the end of time once the input has been fully consumed, thus no additional triggering granularity is available.
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: fully supported
@@ -1221,6 +1335,10 @@ capability-matrix:
l1: "Yes"
l2: yes in streaming, fixed granularity in batch
l3: Fully supported in streaming mode. In batch mode, from the perspective of triggers, processing time currently jumps from the beginning of time to the end of time once the input has been fully consumed, thus no additional triggering granularity is available.
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: fully supported
@@ -1257,6 +1375,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: Fully supported in streaming mode. In batch mode, elements are processed in the largest bundles possible, so count-based triggers are effectively meaningless.
+ - class: prism
+ l1: "No"
+ l2: partially supported
+ l3: "Prism will double fire on count triggers"
- class: flink
l1: "Yes"
l2: fully supported
@@ -1293,6 +1415,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: fully supported
@@ -1329,6 +1455,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: Fully supported in streaming mode. In batch mode no data is ever late.
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: fully supported
@@ -1365,6 +1495,10 @@ capability-matrix:
l1: "Partially"
l2: non-merging windows
l3: Dataflow supports timers in non-merging windows.
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Partially"
l2: non-merging windows
@@ -1410,6 +1544,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: ""
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: fully supported
@@ -1446,6 +1584,10 @@ capability-matrix:
l1: "Yes"
l2: fully supported
l3: Requires that the accumulated pane fits in memory, after being passed through the combiner (if relevant)
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
- class: flink
l1: "Yes"
l2: fully supported
@@ -1491,6 +1633,10 @@ capability-matrix:
l1: "Partially"
l2:
l3: Dataflow has a native drain operation, but it does not work in the presence of event time timer loops. Final implemention pending model support.
+ - class: prism
+ l1: "No"
+ l2:
+ l3:
- class: flink
l1: "Partially"
l2:
@@ -1526,6 +1672,10 @@ capability-matrix:
l1: "No"
l2:
l3:
+ - class: prism
+ l1: "No"
+ l2:
+ l3:
- class: flink
l1: "Partially"
l2:
@@ -1561,6 +1711,14 @@ capability-matrix:
l1: "Partially"
l2:
l3: Dataflow performs different shuffling algorithms for batch and streaming. Dataflow guarantees key-ordered delivery in streaming, though not in batch.
+ - class: prism
+ l1: "Yes"
+ l2: fully supported
+ l3: ""
+ - class: prism
+ l1: "Unverified"
+ l2:
+ l3:
- class: flink
l1: "Partially"
l2: