You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2019/06/07 19:40:17 UTC
[beam] branch asf-site updated: Publishing website 2019/06/07
19:40:05 at commit cd7f4e2
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 7d16712 Publishing website 2019/06/07 19:40:05 at commit cd7f4e2
7d16712 is described below
commit 7d16712ab8e0c1c40a2a22b513d6a1ec573ff359
Author: jenkins <bu...@apache.org>
AuthorDate: Fri Jun 7 19:40:06 2019 +0000
Publishing website 2019/06/07 19:40:05 at commit cd7f4e2
---
.../runners/capability-matrix/index.html | 2186 ++++++++++++++------
1 file changed, 1560 insertions(+), 626 deletions(-)
diff --git a/website/generated-content/documentation/runners/capability-matrix/index.html b/website/generated-content/documentation/runners/capability-matrix/index.html
index 8601de7..3525962 100644
--- a/website/generated-content/documentation/runners/capability-matrix/index.html
+++ b/website/generated-content/documentation/runners/capability-matrix/index.html
@@ -6622,78 +6622,51 @@ limitations under the License.
<td class="cap-summary color-blank cap-blank" colspan="5"></td>
</tr>
-</table>
-</div>
-
-
-<!-- Full details table -->
-
-
-
-
-
-
-<!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-<div id="cap-full" style="display:none">
-<table class="cap">
-
- <tr class="cap" id="cap-full-what">
- <th class="cap color-metadata format-category" colspan="8" style="color:#ca1">
+ <tr class="cap-summary" id="cap-summary-misc">
+ <th class="cap-summary color-metadata format-category" colspan="8" style="color:#aaa">
- <div class="cap-toggle" onclick="ToggleTables(0, "cap-summary-what")">
- (click to collapse details)
+ <div class="cap-toggle" onclick="ToggleTables(1, "cap-full-misc")">
+ (click to expand details)
</div>
- What is being computed?
+ Additional common features not yet part of the Beam model
</th>
</tr>
- <tr class="cap">
- <th class="cap color-capability"></th>
+ <tr class="cap-summary">
+ <th class="cap-summary color-capability"></th>
- <th class="cap color-platform format-platform" style="color:#ec3">Beam Model</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">Beam Model</th>
- <th class="cap color-platform format-platform" style="color:#ec3">Google Cloud Dataflow</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">Google Cloud Dataflow</th>
- <th class="cap color-platform format-platform" style="color:#ec3">Apache Flink</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">Apache Flink</th>
- <th class="cap color-platform format-platform" style="color:#ec3">Apache Spark</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">Apache Spark</th>
- <th class="cap color-platform format-platform" style="color:#ec3">Apache Apex</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">Apache Apex</th>
- <th class="cap color-platform format-platform" style="color:#ec3">Apache Gearpump</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">Apache Gearpump</th>
- <th class="cap color-platform format-platform" style="color:#ec3">Apache Hadoop MapReduce</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">Apache Hadoop MapReduce</th>
- <th class="cap color-platform format-platform" style="color:#ec3">JStorm</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">JStorm</th>
- <th class="cap color-platform format-platform" style="color:#ec3">IBM Streams</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">IBM Streams</th>
- <th class="cap color-platform format-platform" style="color:#ec3">Apache Samza</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">Apache Samza</th>
- <th class="cap color-platform format-platform" style="color:#ec3">Apache Nemo</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">Apache Nemo</th>
- <th class="cap color-platform format-platform" style="color:#ec3">Hazelcast Jet</th>
+ <th class="cap-summary color-platform format-platform" style="color:#bbb">Hazelcast Jet</th>
</tr>
- <tr class="cap">
- <th class="cap color-capability format-capability" style="color:#ec3">ParDo</th>
+ <tr class="cap-summary">
+ <th class="cap-summary color-capability format-capability" style="color:#bbb">Drain</th>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ccc;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6706,12 +6679,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: element-wise processing</b></center><br />Element-wise transformation parameterized by a chunk of user code. Elements are processed in bundles, with initialization and termination hooks. Bundle size is chosen by the runner and cannot be controlled by user code. ParDo processes a main input PCollection one element at a time, but provides side input access to additional PCollections.
+<center><b>~</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ccc;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6724,12 +6697,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Batch mode uses large bundle sizes. Streaming uses smaller bundle sizes.
+<center><b>~</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ccc;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6742,12 +6715,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />ParDo itself, as per-element transformation with UDFs, is fully supported by Flink for both batch and streaming.
+<center><b>~</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6760,12 +6733,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />ParDo applies per-element transformations as Spark FlatMapFunction.
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6778,12 +6751,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Supported through Apex operator that wraps the function and processes data as single element bundles.
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6796,12 +6769,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Gearpump wraps the per-element transformation function into processor execution.
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6814,12 +6787,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6832,12 +6805,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6850,12 +6823,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6868,12 +6841,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Supported with per-element transformation.
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6886,12 +6859,17 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>✕</b></center>
</td>
+ </tr>
+
+ <tr class="cap-summary">
+ <th class="cap-summary color-capability format-capability" style="color:#bbb">Checkpoint</th>
+
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ccc;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6904,17 +6882,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>~</b></center>
</td>
- </tr>
-
- <tr class="cap">
- <th class="cap color-capability format-capability" style="color:#ec3">GroupByKey</th>
-
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6927,12 +6900,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: key grouping</b></center><br />Grouping of key-value pairs per key, window, and pane. (See also other tabs.)
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ccc;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6945,12 +6918,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>~</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ccc;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6963,12 +6936,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Uses Flink's keyBy for key grouping. When grouping by window in streaming (creating the panes) the Flink runner uses the Beam code. This guarantees support for all windowing and triggering mechanisms.
+<center><b>~</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6981,12 +6954,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: fully supported in batch mode</b></center><br />Using Spark's <tt>groupByKey</tt>. GroupByKey with multiple trigger firings in streaming mode is a work in progress.
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -6999,12 +6972,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Apex runner uses the Beam code for grouping by window and thereby has support for all windowing and triggering mechanisms. Runner does not implement partitioning yet (BEAM-838)
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7017,12 +6990,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Use Gearpump's groupBy and window for key grouping and translate Beam's windowing and triggering to Gearpump's internal implementation.
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7035,12 +7008,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7053,12 +7026,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ccc;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7071,12 +7044,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>~</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7089,12 +7062,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Uses Samza's partitionBy for key grouping and Beam's logic for window aggregation and triggering.
+<center><b>✕</b></center>
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap-summary" style="background-color:#ddd;border-color:#aaa"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7107,12 +7080,27 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>✕</b></center>
</td>
-
+ </tr>
+
+ <tr class="cap-summary">
+ <td class="cap-summary color-blank cap-blank" colspan="5"></td>
+ </tr>
+
+</table>
+</div>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+
+<!-- Full details table -->
+
+
+
+
+
+
+<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7125,13 +7113,50 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
-</td>
-
+<div id="cap-full" style="display:none">
+<table class="cap">
+
+ <tr class="cap" id="cap-full-what">
+ <th class="cap color-metadata format-category" colspan="8" style="color:#ca1">
+
+ <div class="cap-toggle" onclick="ToggleTables(0, "cap-summary-what")">
+ (click to collapse details)
+ </div>
+
+ What is being computed?
+ </th>
+ </tr>
+ <tr class="cap">
+ <th class="cap color-capability"></th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">Beam Model</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">Google Cloud Dataflow</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">Apache Flink</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">Apache Spark</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">Apache Apex</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">Apache Gearpump</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">Apache Hadoop MapReduce</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">JStorm</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">IBM Streams</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">Apache Samza</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">Apache Nemo</th>
+
+ <th class="cap color-platform format-platform" style="color:#ec3">Hazelcast Jet</th>
+
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#ec3">Flatten</th>
+ <th class="cap color-capability format-capability" style="color:#ec3">ParDo</th>
@@ -7148,7 +7173,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: collection concatenation</b></center><br />Concatenates multiple homogenously typed collections together.
+<center><b>Yes: element-wise processing</b></center><br />Element-wise transformation parameterized by a chunk of user code. Elements are processed in bundles, with initialization and termination hooks. Bundle size is chosen by the runner and cannot be controlled by user code. ParDo processes a main input PCollection one element at a time, but provides side input access to additional PCollections.
</td>
@@ -7166,7 +7191,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Batch mode uses large bundle sizes. Streaming uses smaller bundle sizes.
</td>
@@ -7184,7 +7209,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: fully supported</b></center><br />ParDo itself, as per-element transformation with UDFs, is fully supported by Flink for both batch and streaming.
</td>
@@ -7202,7 +7227,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: fully supported</b></center><br />ParDo applies per-element transformations as Spark FlatMapFunction.
</td>
@@ -7220,7 +7245,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Supported through Apex operator that wraps the function and processes data as single element bundles.
</td>
@@ -7238,7 +7263,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Gearpump wraps the per-element transformation function into processor execution.
</td>
@@ -7310,7 +7335,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Supported with per-element transformation.
</td>
@@ -7352,7 +7377,7 @@ limitations under the License.
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#ec3">Combine</th>
+ <th class="cap color-capability format-capability" style="color:#ec3">GroupByKey</th>
@@ -7369,7 +7394,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: associative & commutative aggregation</b></center><br />Application of an associative, commutative operation over all values ("globally") or over all values associated with each key ("per key"). Can be implemented using ParDo, but often more efficient implementations exist.
+<center><b>Yes: key grouping</b></center><br />Grouping of key-value pairs per key, window, and pane. (See also other tabs.)
</td>
@@ -7387,7 +7412,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: efficient execution</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -7405,12 +7430,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Uses a combiner for pre-aggregation for batch and streaming.
+<center><b>Yes: fully supported</b></center><br />Uses Flink's keyBy for key grouping. When grouping by window in streaming (creating the panes) the Flink runner uses the Beam code. This guarantees support for all windowing and triggering mechanisms.
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7423,7 +7448,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Using Spark's <tt>combineByKey</tt> and <tt>aggregate</tt> functions.
+<center><b>Partially: fully supported in batch mode</b></center><br />Using Spark's <tt>groupByKey</tt>. GroupByKey with multiple trigger firings in streaming mode is a work in progress.
</td>
@@ -7441,7 +7466,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Default Beam translation. Currently no efficient pre-aggregation (BEAM-935).
+<center><b>Yes: fully supported</b></center><br />Apex runner uses the Beam code for grouping by window and thereby has support for all windowing and triggering mechanisms. Runner does not implement partitioning yet (BEAM-838)
</td>
@@ -7459,7 +7484,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Use Gearpump's groupBy and window for key grouping and translate Beam's windowing and triggering to Gearpump's internal implementation.
</td>
@@ -7531,7 +7556,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Use combiner for efficient pre-aggregation.
+<center><b>Yes: fully supported</b></center><br />Uses Samza's partitionBy for key grouping and Beam's logic for window aggregation and triggering.
</td>
@@ -7549,7 +7574,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Batch mode uses pre-aggregation
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -7567,13 +7592,13 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Batch mode uses pre-aggregation
+<center><b>Yes: fully supported</b></center><br />
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#ec3">Composite Transforms</th>
+ <th class="cap color-capability format-capability" style="color:#ec3">Flatten</th>
@@ -7590,12 +7615,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: user-defined transformation subgraphs</b></center><br />Allows easy extensibility for library writers. In the near future, we expect there to be more information provided at this level -- customized metadata hooks for monitoring, additional runtime/environment hooks, etc.
+<center><b>Yes: collection concatenation</b></center><br />Concatenates multiple homogenously typed collections together.
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7608,12 +7633,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supported via inlining</b></center><br />Currently composite transformations are inlined during execution. The structure is later recreated from the names, but other transform level information (if added to the model) will be lost.
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7626,12 +7651,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supported via inlining</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7644,12 +7669,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supported via inlining</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7662,12 +7687,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supported via inlining</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7680,7 +7705,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supported via inlining</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -7721,7 +7746,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7734,12 +7759,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supported via inlining</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7752,7 +7777,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supported via inlining</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -7775,7 +7800,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -7788,13 +7813,13 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supported via inlining</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#ec3">Side Inputs</th>
+ <th class="cap color-capability format-capability" style="color:#ec3">Combine</th>
@@ -7811,7 +7836,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: additional elements available during DoFn execution</b></center><br />Side inputs are additional <tt>PCollections</tt> whose contents are computed during pipeline execution and then made accessible to DoFn code. The exact shape of the side input depends both on the <tt>PCollectionView</tt> used to describe the access pattern (interable, map, singleton) and the window of the element from the main input that is currently being processed.
+<center><b>Yes: associative & commutative aggregation</b></center><br />Application of an associative, commutative operation over all values ("globally") or over all values associated with each key ("per key"). Can be implemented using ParDo, but often more efficient implementations exist.
</td>
@@ -7829,7 +7854,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: some size restrictions in streaming</b></center><br />Batch mode supports a distributed implementation, but streaming mode may force some size restrictions. Neither mode is able to push lookups directly up into key-based sources.
+<center><b>Yes: efficient execution</b></center><br />
</td>
@@ -7847,7 +7872,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: some size restrictions in streaming</b></center><br />Batch mode supports a distributed implementation, but streaming mode may force some size restrictions. Neither mode is able to push lookups directly up into key-based sources.
+<center><b>Yes: fully supported</b></center><br />Uses a combiner for pre-aggregation for batch and streaming.
</td>
@@ -7865,7 +7890,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Using Spark's broadcast variables. In streaming mode, side inputs may update but only between micro-batches.
+<center><b>Yes: fully supported</b></center><br />Using Spark's <tt>combineByKey</tt> and <tt>aggregate</tt> functions.
</td>
@@ -7883,7 +7908,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: size restrictions</b></center><br />No distributed implementation and therefore size restrictions.
+<center><b>Yes: fully supported</b></center><br />Default Beam translation. Currently no efficient pre-aggregation (BEAM-935).
</td>
@@ -7901,7 +7926,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Implemented by merging side input as a normal stream in Gearpump
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -7937,7 +7962,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: some size restrictions</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -7973,7 +7998,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Uses Samza's broadcast operator to distribute the side inputs.
+<center><b>Yes: fully supported</b></center><br />Use combiner for efficient pre-aggregation.
</td>
@@ -7991,12 +8016,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Batch mode uses pre-aggregation
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8009,13 +8034,13 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: with restrictions</b></center><br />Supported only when the side input source is bounded and windowing uses global window
+<center><b>Yes: fully supported</b></center><br />Batch mode uses pre-aggregation
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#ec3">Source API</th>
+ <th class="cap color-capability format-capability" style="color:#ec3">Composite Transforms</th>
@@ -8032,12 +8057,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: user-defined sources</b></center><br />Allows users to provide additional input sources. Supports both bounded and unbounded data. Includes hooks necessary to provide efficient parallelization (size estimation, progress information, dynamic splitting, etc).
+<center><b>Yes: user-defined transformation subgraphs</b></center><br />Allows easy extensibility for library writers. In the near future, we expect there to be more information provided at this level -- customized metadata hooks for monitoring, additional runtime/environment hooks, etc.
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8050,12 +8075,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Support includes autotuning features (https://cloud.google.com/dataflow/service/dataflow-service-desc#autotuning-features).
+<center><b>Partially: supported via inlining</b></center><br />Currently composite transformations are inlined during execution. The structure is later recreated from the names, but other transform level information (if added to the model) will be lost.
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8068,12 +8093,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: supported via inlining</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8086,12 +8111,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: supported via inlining</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8104,12 +8129,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: supported via inlining</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8122,12 +8147,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: supported via inlining</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8140,7 +8165,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: bounded source only</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -8163,7 +8188,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8176,12 +8201,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: supported via inlining</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8194,7 +8219,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: supported via inlining</b></center><br />
</td>
@@ -8217,7 +8242,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8230,17 +8255,17 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: supported via inlining</b></center><br />
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#ec3">Splittable DoFn (SDF)</th>
+ <th class="cap color-capability format-capability" style="color:#ec3">Side Inputs</th>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8253,7 +8278,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: DoFn where processing of each element can be split for parallelism, or suspended and resumed</b></center><br />Allows users to develop DoFn's that process a single element in portions ("restrictions"), executed in parallel or sequentially. This supersedes the unbounded and bounded `Source` APIs by supporting all of their features on a per-element basis. See http://s.apache.org/splittable-do-fn. Design is in progress on achieving parity with Source API regarding prog [...]
+<center><b>Yes: additional elements available during DoFn execution</b></center><br />Side inputs are additional <tt>PCollections</tt> whose contents are computed during pipeline execution and then made accessible to DoFn code. The exact shape of the side input depends both on the <tt>PCollectionView</tt> used to describe the access pattern (interable, map, singleton) and the window of the element from the main input that is currently being processed.
</td>
@@ -8271,7 +8296,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: </b></center><br />Does not yet support autotuning features of the Source API.
+<center><b>Yes: some size restrictions in streaming</b></center><br />Batch mode supports a distributed implementation, but streaming mode may force some size restrictions. Neither mode is able to push lookups directly up into key-based sources.
</td>
@@ -8289,12 +8314,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: </b></center><br />
+<center><b>Yes: some size restrictions in streaming</b></center><br />Batch mode supports a distributed implementation, but streaming mode may force some size restrictions. Neither mode is able to push lookups directly up into key-based sources.
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8307,12 +8332,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supports bounded-per-element SDFs</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Using Spark's broadcast variables. In streaming mode, side inputs may update but only between micro-batches.
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8325,12 +8350,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supports bounded-per-element SDFs</b></center><br />implementation in streaming mode coming soon
+<center><b>Yes: size restrictions</b></center><br />No distributed implementation and therefore size restrictions.
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8343,12 +8368,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supports bounded-per-element SDFs</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Implemented by merging side input as a normal stream in Gearpump
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8361,12 +8386,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8379,12 +8404,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>Yes: some size restrictions</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8397,12 +8422,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8415,12 +8440,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: supports bounded-per-element SDFs</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Uses Samza's broadcast operator to distribute the side inputs.
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8433,12 +8458,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8451,17 +8476,17 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>Partially: with restrictions</b></center><br />Supported only when the side input source is bounded and windowing uses global window
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#ec3">Metrics</th>
+ <th class="cap color-capability format-capability" style="color:#ec3">Source API</th>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8474,12 +8499,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: user-provided metrics</b></center><br />Allow transforms to gather simple metrics across bundles in a <tt>PTransform</tt>. Provide a mechanism to obtain both committed and attempted metrics. Semantically similar to using an additional output, but support partial results as the transform executes, and support both committed and attempted values. Will likely want to augment <tt>Metrics</tt> to be more useful for processing unbounded data by making them windowed.
+<center><b>Yes: user-defined sources</b></center><br />Allows users to provide additional input sources. Supports both bounded and unbounded data. Includes hooks necessary to provide efficient parallelization (size estimation, progress information, dynamic splitting, etc).
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8492,12 +8517,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: In batch mode, Dataflow supports committed and attempted Counters and Distributions.</b></center><br />Gauge metrics are not supported in batch mode. Metrics are not yet supported at all in streaming mode, but this support is coming soon ([BEAM-2059](https://issues.apache.org/jira/browse/BEAM-2059)).
+<center><b>Yes: fully supported</b></center><br />Support includes autotuning features (https://cloud.google.com/dataflow/service/dataflow-service-desc#autotuning-features).
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8510,12 +8535,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: All metrics types are supported.</b></center><br />Only attempted values are supported. No committed values for metrics.
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8528,12 +8553,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: All metric types are supported.</b></center><br />Only attempted values are supported. No committed values for metrics.
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8546,12 +8571,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: Not implemented in runner.</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8564,7 +8589,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />not implemented
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -8582,12 +8607,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: Only attempted counters are supported</b></center><br />
+<center><b>Partially: bounded source only</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8600,12 +8625,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: Metrics are only supported in local mode.</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8618,12 +8643,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: All metrics types are supported.</b></center><br />Only attempted values are supported. No committed values for metrics.
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8636,12 +8661,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: Counter and Gauge are supported.</b></center><br />Only attempted values are supported. No committed values for metrics.
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8654,12 +8679,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8672,17 +8697,17 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: All metrics types supported, both in batching and streaming mode.</b></center><br />Doesn't differentiate between committed and attempted values.
+<center><b>Yes: fully supported</b></center><br />
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#ec3">Stateful Processing</th>
+ <th class="cap color-capability format-capability" style="color:#ec3">Splittable DoFn (SDF)</th>
- <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8695,12 +8720,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: storage per key, per window</b></center><br />Allows fine-grained access to per-key, per-window persistent state. Necessary for certain use cases (e.g. high-volume windows which store large amounts of data, but typically only access small portions of it; complex state machines; etc.) that are not easily or efficiently addressed via <tt>Combine</tt> or <tt>GroupByKey</tt>+<tt>ParDo</tt>.
+<center><b>Partially: DoFn where processing of each element can be split for parallelism, or suspended and resumed</b></center><br />Allows users to develop DoFn's that process a single element in portions ("restrictions"), executed in parallel or sequentially. This supersedes the unbounded and bounded `Source` APIs by supporting all of their features on a per-element basis. See http://s.apache.org/splittable-do-fn. Design is in progress on achieving parity with Source API regarding prog [...]
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8713,12 +8738,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />State is supported for non-merging windows. SetState and MapState are not yet supported.
+<center><b>Yes: </b></center><br />Does not yet support autotuning features of the Source API.
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8731,12 +8756,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />State is supported for non-merging windows. SetState and MapState are not yet supported.
+<center><b>Yes: </b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8749,7 +8774,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />Spark supports per-key state with <tt>mapWithState()</tt> so support should be straightforward.
+<center><b>Partially: supports bounded-per-element SDFs</b></center><br />
</td>
@@ -8767,12 +8792,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />State is supported for non-merging windows. SetState and MapState are not yet supported.
+<center><b>Partially: supports bounded-per-element SDFs</b></center><br />implementation in streaming mode coming soon
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8785,12 +8810,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>Partially: supports bounded-per-element SDFs</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8803,12 +8828,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />
+<center><b>No: not implemented</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8821,12 +8846,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />
+<center><b>No: not implemented</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8839,7 +8864,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />
+<center><b>No: not implemented</b></center><br />
</td>
@@ -8857,7 +8882,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />States are backed up by either rocksDb KV store or in-memory hash map, and persist using changelog.
+<center><b>Partially: supports bounded-per-element SDFs</b></center><br />
</td>
@@ -8880,7 +8905,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8893,60 +8918,17 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />
+<center><b>No: not implemented</b></center><br />
</td>
</tr>
<tr class="cap">
- <td class="cap color-blank cap-blank" colspan="5"></td>
- </tr>
-
- <tr class="cap" id="cap-full-where">
- <th class="cap color-metadata format-category" colspan="8" style="color:#37d">
-
- <div class="cap-toggle" onclick="ToggleTables(0, "cap-summary-where")">
- (click to collapse details)
- </div>
-
- Where in event time?
- </th>
- </tr>
- <tr class="cap">
- <th class="cap color-capability"></th>
-
- <th class="cap color-platform format-platform" style="color:#59f">Beam Model</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">Google Cloud Dataflow</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">Apache Flink</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">Apache Spark</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">Apache Apex</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">Apache Gearpump</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">Apache Hadoop MapReduce</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">JStorm</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">IBM Streams</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">Apache Samza</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">Apache Nemo</th>
-
- <th class="cap color-platform format-platform" style="color:#59f">Hazelcast Jet</th>
-
- </tr>
-
- <tr class="cap">
- <th class="cap color-capability format-capability" style="color:#59f">Global windows</th>
+ <th class="cap color-capability format-capability" style="color:#ec3">Metrics</th>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8959,12 +8941,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: all time</b></center><br />The default window which covers all of time. (Basically how traditional batch cases fit in the model.)
+<center><b>Partially: user-provided metrics</b></center><br />Allow transforms to gather simple metrics across bundles in a <tt>PTransform</tt>. Provide a mechanism to obtain both committed and attempted metrics. Semantically similar to using an additional output, but support partial results as the transform executes, and support both committed and attempted values. Will likely want to augment <tt>Metrics</tt> to be more useful for processing unbounded data by making them windowed.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8977,12 +8959,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: default</b></center><br />
+<center><b>Partially: In batch mode, Dataflow supports committed and attempted Counters and Distributions.</b></center><br />Gauge metrics are not supported in batch mode. Metrics are not yet supported at all in streaming mode, but this support is coming soon ([BEAM-2059](https://issues.apache.org/jira/browse/BEAM-2059)).
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -8995,12 +8977,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: All metrics types are supported.</b></center><br />Only attempted values are supported. No committed values for metrics.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9013,12 +8995,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: All metric types are supported.</b></center><br />Only attempted values are supported. No committed values for metrics.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9031,12 +9013,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>No: Not implemented in runner.</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9049,12 +9031,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>No</b></center><br />not implemented
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9067,12 +9049,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: Only attempted counters are supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9085,12 +9067,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: Metrics are only supported in local mode.</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9103,12 +9085,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: All metrics types are supported.</b></center><br />Only attempted values are supported. No committed values for metrics.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9121,12 +9103,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: Counter and Gauge are supported.</b></center><br />Only attempted values are supported. No committed values for metrics.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9139,12 +9121,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>No: not implemented</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9157,17 +9139,17 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: All metrics types supported, both in batching and streaming mode.</b></center><br />Doesn't differentiate between committed and attempted values.
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#59f">Fixed windows</th>
+ <th class="cap color-capability format-capability" style="color:#ec3">Stateful Processing</th>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#ec3;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9180,12 +9162,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: periodic, non-overlapping</b></center><br />Fixed-size, timestamp-based windows. (Hourly, Daily, etc)
+<center><b>Yes: storage per key, per window</b></center><br />Allows fine-grained access to per-key, per-window persistent state. Necessary for certain use cases (e.g. high-volume windows which store large amounts of data, but typically only access small portions of it; complex state machines; etc.) that are not easily or efficiently addressed via <tt>Combine</tt> or <tt>GroupByKey</tt>+<tt>ParDo</tt>.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9198,12 +9180,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: built-in</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />State is supported for non-merging windows. SetState and MapState are not yet supported.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9216,12 +9198,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />State is supported for non-merging windows. SetState and MapState are not yet supported.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9234,12 +9216,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>No: not implemented</b></center><br />Spark supports per-key state with <tt>mapWithState()</tt> so support should be straightforward.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9252,12 +9234,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />State is supported for non-merging windows. SetState and MapState are not yet supported.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9270,12 +9252,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>No: not implemented</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9288,12 +9270,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9306,12 +9288,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9324,12 +9306,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9342,12 +9324,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />States are backed up by either rocksDb KV store or in-memory hash map, and persist using changelog.
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9360,12 +9342,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>No: not implemented</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
+ <td width="25%" class="cap" style="background-color:#fe5;border-color:#ca1"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -9378,13 +9360,56 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#59f">Sliding windows</th>
+ <td class="cap color-blank cap-blank" colspan="5"></td>
+ </tr>
+
+ <tr class="cap" id="cap-full-where">
+ <th class="cap color-metadata format-category" colspan="8" style="color:#37d">
+
+ <div class="cap-toggle" onclick="ToggleTables(0, "cap-summary-where")">
+ (click to collapse details)
+ </div>
+
+ Where in event time?
+ </th>
+ </tr>
+ <tr class="cap">
+ <th class="cap color-capability"></th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">Beam Model</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">Google Cloud Dataflow</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">Apache Flink</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">Apache Spark</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">Apache Apex</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">Apache Gearpump</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">Apache Hadoop MapReduce</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">JStorm</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">IBM Streams</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">Apache Samza</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">Apache Nemo</th>
+
+ <th class="cap color-platform format-platform" style="color:#59f">Hazelcast Jet</th>
+
+ </tr>
+
+ <tr class="cap">
+ <th class="cap color-capability format-capability" style="color:#59f">Global windows</th>
@@ -9401,7 +9426,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: periodic, overlapping</b></center><br />Possibly overlapping fixed-size timestamp-based windows (Every minute, use the last ten minutes of data.)
+<center><b>Yes: all time</b></center><br />The default window which covers all of time. (Basically how traditional batch cases fit in the model.)
</td>
@@ -9419,7 +9444,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: built-in</b></center><br />
+<center><b>Yes: default</b></center><br />
</td>
@@ -9605,7 +9630,7 @@ limitations under the License.
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#59f">Session windows</th>
+ <th class="cap color-capability format-capability" style="color:#59f">Fixed windows</th>
@@ -9622,7 +9647,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: activity-based</b></center><br />Based on bursts of activity separated by a gap size. Different per key.
+<center><b>Yes: periodic, non-overlapping</b></center><br />Fixed-size, timestamp-based windows. (Hourly, Daily, etc)
</td>
@@ -9826,7 +9851,7 @@ limitations under the License.
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#59f">Custom windows</th>
+ <th class="cap color-capability format-capability" style="color:#59f">Sliding windows</th>
@@ -9843,7 +9868,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: user-defined windows</b></center><br />All windows must implement <tt>BoundedWindow</tt>, which specifies a max timestamp. Each <tt>WindowFn</tt> assigns elements to an associated window.
+<center><b>Yes: periodic, overlapping</b></center><br />Possibly overlapping fixed-size timestamp-based windows (Every minute, use the last ten minutes of data.)
</td>
@@ -9861,7 +9886,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Yes: built-in</b></center><br />
</td>
@@ -10047,7 +10072,7 @@ limitations under the License.
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#59f">Custom merging windows</th>
+ <th class="cap color-capability format-capability" style="color:#59f">Session windows</th>
@@ -10064,7 +10089,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: user-defined merging windows</b></center><br />A custom <tt>WindowFn</tt> additionally specifies whether and how to merge windows.
+<center><b>Yes: activity-based</b></center><br />Based on bursts of activity separated by a gap size. Different per key.
</td>
@@ -10082,7 +10107,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: supported</b></center><br />
+<center><b>Yes: built-in</b></center><br />
</td>
@@ -10268,7 +10293,7 @@ limitations under the License.
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#59f">Timestamp control</th>
+ <th class="cap color-capability format-capability" style="color:#59f">Custom windows</th>
@@ -10285,7 +10310,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: output timestamp for window panes</b></center><br />For a grouping transform, such as GBK or Combine, an OutputTimeFn specifies (1) how to combine input timestamps within a window and (2) how to merge aggregated timestamps when windows merge.
+<center><b>Yes: user-defined windows</b></center><br />All windows must implement <tt>BoundedWindow</tt>, which specifies a max timestamp. Each <tt>WindowFn</tt> assigns elements to an associated window.
</td>
@@ -10489,54 +10514,11 @@ limitations under the License.
</tr>
<tr class="cap">
- <td class="cap color-blank cap-blank" colspan="5"></td>
- </tr>
-
- <tr class="cap" id="cap-full-when">
- <th class="cap color-metadata format-category" colspan="8" style="color:#6a4">
-
- <div class="cap-toggle" onclick="ToggleTables(0, "cap-summary-when")">
- (click to collapse details)
- </div>
-
- When in processing time?
- </th>
- </tr>
- <tr class="cap">
- <th class="cap color-capability"></th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">Beam Model</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">Google Cloud Dataflow</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">Apache Flink</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">Apache Spark</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">Apache Apex</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">Apache Gearpump</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">Apache Hadoop MapReduce</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">JStorm</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">IBM Streams</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">Apache Samza</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">Apache Nemo</th>
-
- <th class="cap color-platform format-platform" style="color:#8c6">Hazelcast Jet</th>
-
- </tr>
-
- <tr class="cap">
- <th class="cap color-capability format-capability" style="color:#8c6">Configurable triggering</th>
+ <th class="cap color-capability format-capability" style="color:#59f">Custom merging windows</th>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10549,12 +10531,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: user customizable</b></center><br />Triggering may be specified by the user (instead of simply driven by hardcoded defaults).
+<center><b>Yes: user-defined merging windows</b></center><br />A custom <tt>WindowFn</tt> additionally specifies whether and how to merge windows.
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10567,12 +10549,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Fully supported in streaming mode. In batch mode, intermediate trigger firings are effectively meaningless.
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10585,12 +10567,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10603,12 +10585,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10621,12 +10603,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10639,12 +10621,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10657,12 +10639,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: batch-only runner</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10675,12 +10657,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10693,12 +10675,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10711,12 +10693,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10729,12 +10711,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10747,17 +10729,17 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#8c6">Event-time triggers</th>
+ <th class="cap color-capability format-capability" style="color:#59f">Timestamp control</th>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10770,12 +10752,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: relative to event time</b></center><br />Triggers that fire in response to event-time completeness signals, such as watermarks progressing.
+<center><b>Yes: output timestamp for window panes</b></center><br />For a grouping transform, such as GBK or Combine, an OutputTimeFn specifies (1) how to combine input timestamps within a window and (2) how to merge aggregated timestamps when windows merge.
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10788,12 +10770,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: yes in streaming, fixed granularity in batch</b></center><br />Fully supported in streaming mode. In batch mode, currently watermark progress jumps from the beginning of time to the end of time once the input has been fully consumed, thus no additional triggering granularity is available.
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10806,12 +10788,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10824,12 +10806,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10842,12 +10824,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10860,12 +10842,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10878,12 +10860,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10896,12 +10878,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10914,12 +10896,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10932,12 +10914,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10950,12 +10932,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#59f;border-color:#37d"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -10968,13 +10950,56 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: supported</b></center><br />
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#8c6">Processing-time triggers</th>
+ <td class="cap color-blank cap-blank" colspan="5"></td>
+ </tr>
+
+ <tr class="cap" id="cap-full-when">
+ <th class="cap color-metadata format-category" colspan="8" style="color:#6a4">
+
+ <div class="cap-toggle" onclick="ToggleTables(0, "cap-summary-when")">
+ (click to collapse details)
+ </div>
+
+ When in processing time?
+ </th>
+ </tr>
+ <tr class="cap">
+ <th class="cap color-capability"></th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">Beam Model</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">Google Cloud Dataflow</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">Apache Flink</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">Apache Spark</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">Apache Apex</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">Apache Gearpump</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">Apache Hadoop MapReduce</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">JStorm</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">IBM Streams</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">Apache Samza</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">Apache Nemo</th>
+
+ <th class="cap color-platform format-platform" style="color:#8c6">Hazelcast Jet</th>
+
+ </tr>
+
+ <tr class="cap">
+ <th class="cap color-capability format-capability" style="color:#8c6">Configurable triggering</th>
@@ -10991,7 +11016,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: relative to processing time</b></center><br />Triggers that fire in response to processing-time advancing.
+<center><b>Yes: user customizable</b></center><br />Triggering may be specified by the user (instead of simply driven by hardcoded defaults).
</td>
@@ -11009,7 +11034,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: yes in streaming, fixed granularity in batch</b></center><br />Fully supported in streaming mode. In batch mode, from the perspective of triggers, processing time currently jumps from the beginning of time to the end of time once the input has been fully consumed, thus no additional triggering granularity is available.
+<center><b>Yes: fully supported</b></center><br />Fully supported in streaming mode. In batch mode, intermediate trigger firings are effectively meaningless.
</td>
@@ -11032,7 +11057,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11045,7 +11070,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: This is Spark streaming's native model</b></center><br />Spark processes streams in micro-batches. The micro-batch size is actually a pre-set, fixed, time interval. Currently, the runner takes the first window size in the pipeline and sets it's size as the batch interval. Any following window operations will be considered processing time windows and will affect triggering.
+<center><b>No</b></center><br />
</td>
@@ -11099,7 +11124,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />
+<center><b>No: batch-only runner</b></center><br />
</td>
@@ -11195,7 +11220,7 @@ limitations under the License.
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#8c6">Count triggers</th>
+ <th class="cap color-capability format-capability" style="color:#8c6">Event-time triggers</th>
@@ -11212,7 +11237,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: every N elements</b></center><br />Triggers that fire after seeing at least N elements.
+<center><b>Yes: relative to event time</b></center><br />Triggers that fire in response to event-time completeness signals, such as watermarks progressing.
</td>
@@ -11230,7 +11255,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Fully supported in streaming mode. In batch mode, elements are processed in the largest bundles possible, so count-based triggers are effectively meaningless.
+<center><b>Yes: yes in streaming, fixed granularity in batch</b></center><br />Fully supported in streaming mode. In batch mode, currently watermark progress jumps from the beginning of time to the end of time once the input has been fully consumed, thus no additional triggering granularity is available.
</td>
@@ -11289,7 +11314,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11302,7 +11327,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -11416,11 +11441,11 @@ limitations under the License.
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#8c6">[Meta]data driven triggers</th>
+ <th class="cap color-capability format-capability" style="color:#8c6">Processing-time triggers</th>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11433,12 +11458,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: in response to data<br />(<a href="https://issues.apache.org/jira/browse/BEAM-101">BEAM-101</a>)</b></center><br />Triggers that fire in response to attributes of the data being processed.
+<center><b>Yes: relative to processing time</b></center><br />Triggers that fire in response to processing-time advancing.
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11451,12 +11476,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: yes in streaming, fixed granularity in batch</b></center><br />Fully supported in streaming mode. In batch mode, from the perspective of triggers, processing time currently jumps from the beginning of time to the end of time once the input has been fully consumed, thus no additional triggering granularity is available.
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11469,12 +11494,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11487,12 +11512,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: This is Spark streaming's native model</b></center><br />Spark processes streams in micro-batches. The micro-batch size is actually a pre-set, fixed, time interval. Currently, the runner takes the first window size in the pipeline and sets it's size as the batch interval. Any following window operations will be considered processing time windows and will affect triggering.
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11505,7 +11530,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -11523,7 +11548,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>No</b></center><br />
</td>
@@ -11546,7 +11571,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11559,12 +11584,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11577,12 +11602,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11595,12 +11620,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11613,12 +11638,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11631,13 +11656,13 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#8c6">Composite triggers</th>
+ <th class="cap color-capability format-capability" style="color:#8c6">Count triggers</th>
@@ -11654,7 +11679,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: compositions of one or more sub-triggers</b></center><br />Triggers which compose other triggers in more complex structures, such as logical AND, logical OR, early/on-time/late, etc.
+<center><b>Yes: every N elements</b></center><br />Triggers that fire after seeing at least N elements.
</td>
@@ -11672,7 +11697,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Fully supported in streaming mode. In batch mode, elements are processed in the largest bundles possible, so count-based triggers are effectively meaningless.
</td>
@@ -11858,11 +11883,11 @@ limitations under the License.
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#8c6">Allowed lateness</th>
+ <th class="cap color-capability format-capability" style="color:#8c6">[Meta]data driven triggers</th>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11875,12 +11900,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: event-time bound on window lifetimes</b></center><br />A way to bound the useful lifetime of a window (in event time), after which any unemitted results may be materialized, the window contents may be garbage collected, and any addtional late data that arrive for the window may be discarded.
+<center><b>No: in response to data<br />(<a href="https://issues.apache.org/jira/browse/BEAM-101">BEAM-101</a>)</b></center><br />Triggers that fire in response to attributes of the data being processed.
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11893,12 +11918,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Fully supported in streaming mode. In batch mode no data is ever late.
+<center><b>No: pending model support</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11911,7 +11936,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>No: pending model support</b></center><br />
</td>
@@ -11929,12 +11954,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />
+<center><b>No: pending model support</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11947,12 +11972,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>No: pending model support</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -11965,7 +11990,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>No: pending model support</b></center><br />
</td>
@@ -11988,7 +12013,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12001,12 +12026,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>No: pending model support</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12019,12 +12044,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>No: pending model support</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12037,12 +12062,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>No: pending model support</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12055,12 +12080,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>No: pending model support</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12073,13 +12098,13 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>No: pending model support</b></center><br />
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#8c6">Timers</th>
+ <th class="cap color-capability format-capability" style="color:#8c6">Composite triggers</th>
@@ -12096,12 +12121,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: delayed processing callbacks</b></center><br />A fine-grained mechanism for performing work at some point in the future, in either the event-time or processing-time domain. Useful for orchestrating delayed events, timeouts, etc in complex state per-key, per-window state machines.
+<center><b>Yes: compositions of one or more sub-triggers</b></center><br />Triggers which compose other triggers in more complex structures, such as logical AND, logical OR, early/on-time/late, etc.
</td>
- <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12114,12 +12139,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />Dataflow supports timers in non-merging windows.
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12132,7 +12157,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />The Flink Runner supports timers in non-merging windows.
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -12150,12 +12175,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>No</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12168,7 +12193,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -12186,7 +12211,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>No</b></center><br />
</td>
@@ -12209,7 +12234,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12222,12 +12247,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12240,12 +12265,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12258,12 +12283,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />The Samza Runner supports timers in non-merging windows.
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12276,12 +12301,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: not implemented</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12294,60 +12319,17 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Partially: non-merging windows</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
</tr>
<tr class="cap">
- <td class="cap color-blank cap-blank" colspan="5"></td>
- </tr>
-
- <tr class="cap" id="cap-full-how">
- <th class="cap color-metadata format-category" colspan="8" style="color:#b55">
-
- <div class="cap-toggle" onclick="ToggleTables(0, "cap-summary-how")">
- (click to collapse details)
- </div>
-
- How do refinements relate?
- </th>
- </tr>
- <tr class="cap">
- <th class="cap color-capability"></th>
-
- <th class="cap color-platform format-platform" style="color:#d77">Beam Model</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">Google Cloud Dataflow</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">Apache Flink</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">Apache Spark</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">Apache Apex</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">Apache Gearpump</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">Apache Hadoop MapReduce</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">JStorm</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">IBM Streams</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">Apache Samza</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">Apache Nemo</th>
-
- <th class="cap color-platform format-platform" style="color:#d77">Hazelcast Jet</th>
-
- </tr>
-
- <tr class="cap">
- <th class="cap color-capability format-capability" style="color:#d77">Discarding</th>
+ <th class="cap color-capability format-capability" style="color:#8c6">Allowed lateness</th>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12360,12 +12342,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: panes discard elements when fired</b></center><br />Elements are discarded from accumulated state as their pane is fired.
+<center><b>Yes: event-time bound on window lifetimes</b></center><br />A way to bound the useful lifetime of a window (in event time), after which any unemitted results may be materialized, the window contents may be garbage collected, and any addtional late data that arrive for the window may be discarded.
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12378,12 +12360,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Fully supported in streaming mode. In batch mode no data is ever late.
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12401,7 +12383,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12414,12 +12396,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Spark streaming natively discards elements after firing.
+<center><b>No</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12437,7 +12419,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12455,7 +12437,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12468,12 +12450,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: batch-only runner</b></center><br />
+<center><b>No</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12491,7 +12473,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12509,7 +12491,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12527,7 +12509,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12545,7 +12527,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12564,11 +12546,11 @@ limitations under the License.
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#d77">Accumulating</th>
+ <th class="cap color-capability format-capability" style="color:#8c6">Timers</th>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#8c6;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12581,12 +12563,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: panes accumulate elements across firings</b></center><br />Elements are accumulated in state across multiple pane firings for the same window.
+<center><b>Yes: delayed processing callbacks</b></center><br />A fine-grained mechanism for performing work at some point in the future, in either the event-time or processing-time domain. Useful for orchestrating delayed events, timeouts, etc in complex state per-key, per-window state machines.
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12599,12 +12581,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Requires that the accumulated pane fits in memory, after being passed through the combiner (if relevant)
+<center><b>Partially: non-merging windows</b></center><br />Dataflow supports timers in non-merging windows.
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12617,12 +12599,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />The Flink Runner supports timers in non-merging windows.
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12635,12 +12617,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />
+<center><b>No: not implemented</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12653,12 +12635,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />Size restriction, see combine support.
+<center><b>No: not implemented</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12671,12 +12653,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />
+<center><b>No: not implemented</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12694,7 +12676,7 @@ limitations under the License.
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12707,12 +12689,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12725,12 +12707,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12743,12 +12725,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />The Samza Runner supports timers in non-merging windows.
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12761,12 +12743,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>No: not implemented</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#ae8;border-color:#6a4"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12779,17 +12761,60 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>Yes: fully supported</b></center><br />
+<center><b>Partially: non-merging windows</b></center><br />
</td>
</tr>
<tr class="cap">
- <th class="cap color-capability format-capability" style="color:#d77">Accumulating & Retracting</th>
+ <td class="cap color-blank cap-blank" colspan="5"></td>
+ </tr>
+
+ <tr class="cap" id="cap-full-how">
+ <th class="cap color-metadata format-category" colspan="8" style="color:#b55">
+
+ <div class="cap-toggle" onclick="ToggleTables(0, "cap-summary-how")">
+ (click to collapse details)
+ </div>
+
+ How do refinements relate?
+ </th>
+ </tr>
+ <tr class="cap">
+ <th class="cap color-capability"></th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">Beam Model</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">Google Cloud Dataflow</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">Apache Flink</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">Apache Spark</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">Apache Apex</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">Apache Gearpump</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">Apache Hadoop MapReduce</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">JStorm</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">IBM Streams</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">Apache Samza</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">Apache Nemo</th>
+
+ <th class="cap color-platform format-platform" style="color:#d77">Hazelcast Jet</th>
+
+ </tr>
+
+ <tr class="cap">
+ <th class="cap color-capability format-capability" style="color:#d77">Discarding</th>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12802,12 +12827,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: accumulation plus retraction of old panes<br />(<a href="https://issues.apache.org/jira/browse/BEAM-91">BEAM-91</a>)</b></center><br />Elements are accumulated across multiple pane firings and old emitted values are retracted. Also known as "backsies" ;-D
+<center><b>Yes: panes discard elements when fired</b></center><br />Elements are discarded from accumulated state as their pane is fired.
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12820,12 +12845,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12838,12 +12863,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12856,12 +12881,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />Spark streaming natively discards elements after firing.
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12874,12 +12899,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12892,7 +12917,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
@@ -12910,12 +12935,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No</b></center><br />
+<center><b>No: batch-only runner</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12928,12 +12953,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12946,12 +12971,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12964,12 +12989,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -12982,12 +13007,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
</td>
- <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -13000,7 +13025,916 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<center><b>No: pending model support</b></center><br />
+<center><b>Yes: fully supported</b></center><br />
+</td>
+
+ </tr>
+
+ <tr class="cap">
+ <th class="cap color-capability format-capability" style="color:#d77">Accumulating</th>
+
+
+
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Yes: panes accumulate elements across firings</b></center><br />Elements are accumulated in state across multiple pane firings for the same window.
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Yes: fully supported</b></center><br />Requires that the accumulated pane fits in memory, after being passed through the combiner (if relevant)
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Yes: fully supported</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Yes: fully supported</b></center><br />Size restriction, see combine support.
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Yes: fully supported</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Yes: fully supported</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Yes: fully supported</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Yes: fully supported</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#d77;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Yes: fully supported</b></center><br />
+</td>
+
+ </tr>
+
+ <tr class="cap">
+ <th class="cap color-capability format-capability" style="color:#d77">Accumulating & Retracting</th>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: accumulation plus retraction of old panes<br />(<a href="https://issues.apache.org/jira/browse/BEAM-91">BEAM-91</a>)</b></center><br />Elements are accumulated across multiple pane firings and old emitted values are retracted. Also known as "backsies" ;-D
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: pending model support</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: pending model support</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: pending model support</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: pending model support</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: pending model support</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: pending model support</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: pending model support</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: pending model support</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: pending model support</b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#b55"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: pending model support</b></center><br />
+</td>
+
+ </tr>
+
+ <tr class="cap">
+ <td class="cap color-blank cap-blank" colspan="5"></td>
+ </tr>
+
+ <tr class="cap" id="cap-full-misc">
+ <th class="cap color-metadata format-category" colspan="8" style="color:#aaa">
+
+ <div class="cap-toggle" onclick="ToggleTables(0, "cap-summary-misc")">
+ (click to collapse details)
+ </div>
+
+ Additional common features not yet part of the Beam model
+ </th>
+ </tr>
+ <tr class="cap">
+ <th class="cap color-capability"></th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">Beam Model</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">Google Cloud Dataflow</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">Apache Flink</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">Apache Spark</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">Apache Apex</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">Apache Gearpump</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">Apache Hadoop MapReduce</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">JStorm</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">IBM Streams</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">Apache Samza</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">Apache Nemo</th>
+
+ <th class="cap color-platform format-platform" style="color:#bbb">Hazelcast Jet</th>
+
+ </tr>
+
+ <tr class="cap">
+ <th class="cap color-capability format-capability" style="color:#bbb">Drain</th>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ccc;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Partially: </b></center><br />APIs and semantics for draining a pipeline are under discussion. This would cause incomplete aggregations to be emitted regardless of trigger and tagged with metadata indicating it is incomplated.
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ccc;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Partially: </b></center><br />Dataflow has a native drain operation, but it does not work in the presence of event time timer loops. Final implemention pending model support.
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ccc;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Partially: </b></center><br />Flink supports taking a "savepoint" of the pipeline and shutting the pipeline down after its completion.
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+ </tr>
+
+ <tr class="cap">
+ <th class="cap color-capability format-capability" style="color:#bbb">Checkpoint</th>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ccc;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Partially: </b></center><br />APIs and semantics for saving a pipeline checkpoint are under discussion. This would be a runner-specific materialization of the pipeline state required to resume or duplicate the pipeline.
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>No: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ccc;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Partially: </b></center><br />Flink has a native savepoint capability.
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ccc;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Partially: </b></center><br />Spark has a native savepoint capability.
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ccc;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>Partially: </b></center><br />Samza has a native checkpoint capability.
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
+</td>
+
+
+
+ <td width="25%" class="cap" style="background-color:#ddd;border-color:#aaa"><!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<center><b>: </b></center><br />
</td>
</tr>