You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2020/06/21 00:02:16 UTC
[beam] branch asf-site updated: Publishing website 2020/06/21
00:02:01 at commit 3384f62
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 6b6815f Publishing website 2020/06/21 00:02:01 at commit 3384f62
6b6815f is described below
commit 6b6815fe2ab6fa213980eb0feb8c7090adee6060
Author: jenkins <us...@infra.apache.org>
AuthorDate: Sun Jun 21 00:02:01 2020 +0000
Publishing website 2020/06/21 00:02:01 at commit 3384f62
---
website/generated-content/contribute/index.html | 2 +-
website/generated-content/contribute/index.xml | 8 +-
.../contribute/ptransform-style-guide/index.html | 2 +-
.../contribute/release-guide/index.html | 5 +-
website/generated-content/documentation/index.html | 2 +-
.../documentation/runners/apex/index.html | 22 -----
.../runners/capability-matrix/index.html | 106 +++++----------------
.../documentation/runners/dataflow/index.html | 2 +-
.../documentation/runners/direct/index.html | 2 +-
.../documentation/runners/flink/index.html | 2 +-
.../documentation/runners/jet/index.html | 2 +-
.../documentation/runners/mapreduce/index.html | 2 +-
.../documentation/runners/nemo/index.html | 2 +-
.../documentation/runners/samza/index.html | 2 +-
.../documentation/runners/spark/index.html | 2 +-
.../sdks/java/testing/nexmark/index.html | 25 +----
.../documentation/sdks/python-streaming/index.html | 2 +-
.../get-started/beam-overview/index.html | 2 +-
website/generated-content/get-started/index.xml | 67 +------------
.../get-started/quickstart-java/index.html | 16 +---
.../get-started/quickstart-py/index.html | 2 +-
.../get-started/wordcount-example/index.html | 28 +++---
website/generated-content/index.html | 2 +-
.../roadmap/apex-runner/index.html | 7 --
.../roadmap/connectors-go-sdk/index.html | 2 +-
.../roadmap/connectors-java-sdk/index.html | 2 +-
.../roadmap/connectors-multi-sdk/index.html | 2 +-
.../roadmap/connectors-python-sdk/index.html | 2 +-
.../roadmap/dataflow-runner/index.html | 2 +-
.../generated-content/roadmap/euphoria/index.html | 2 +-
.../roadmap/flink-runner/index.html | 2 +-
.../generated-content/roadmap/go-sdk/index.html | 2 +-
website/generated-content/roadmap/index.html | 2 +-
website/generated-content/roadmap/index.xml | 19 +---
.../generated-content/roadmap/java-sdk/index.html | 2 +-
.../roadmap/nemo-runner/index.html | 2 +-
.../roadmap/portability/index.html | 2 +-
.../roadmap/python-sdk/index.html | 2 +-
.../roadmap/samza-runner/index.html | 2 +-
.../roadmap/spark-runner/index.html | 2 +-
website/generated-content/roadmap/sql/index.html | 2 +-
website/generated-content/sitemap.xml | 2 +-
42 files changed, 81 insertions(+), 286 deletions(-)
diff --git a/website/generated-content/contribute/index.html b/website/generated-content/contribute/index.html
index 15e7ee5..68a4e51 100644
--- a/website/generated-content/contribute/index.html
+++ b/website/generated-content/contribute/index.html
@@ -2,7 +2,7 @@
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
-<a href=https://stackoverflow.com/questions/tagged/apache-beam>stackoverflow</a></li><li>review proposed design ideas on <a href=/community/contact-us/>dev@beam.apache.org</a></li><li>improve the documentation</li><li>file <a href=https://issues.apache.org/jira/projects/BEAM/issues>bug reports</a></li><li>test releases</li><li>review <a href=https://github.com/apache/beam/pulls>changes</a></li><li>write new examples</li><li>improve your favorite language SDK (Java, Python, Go, etc)</li>< [...]
+<a href=https://stackoverflow.com/questions/tagged/apache-beam>stackoverflow</a></li><li>review proposed design ideas on <a href=/community/contact-us/>dev@beam.apache.org</a></li><li>improve the documentation</li><li>file <a href=https://issues.apache.org/jira/projects/BEAM/issues>bug reports</a></li><li>test releases</li><li>review <a href=https://github.com/apache/beam/pulls>changes</a></li><li>write new examples</li><li>improve your favorite language SDK (Java, Python, Go, etc)</li>< [...]
Cloud Dataflow, etc)</li><li>improve or add IO connectors</li><li>add new transform libraries (statistics, ML, image processing, etc)</li><li>work on the core programming model (what is a Beam pipeline and how does it
run?)</li><li>improve the developer experience (for example, Windows guides)</li><li>add answers to the <a href=https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ>contribution FAQ</a></li><li>organize local meetups of users or contributors to Apache Beam</li></ul><p>Most importantly, if you have an idea of how to contribute, then do it!</p><h2 id=contributing-code>Contributing code</h2><p>Below is a tutorial for contributing code to Beam, covering our tools and typical process in
detail.</p><h3 id=prerequisites>Prerequisites</h3><p>To contribute code, you need</p><ul><li>a GitHub account</li><li>a Linux, macOS, or Microsoft Windows development environment with Java JDK 8 installed</li><li><a href=https://www.docker.com/>Docker</a> installed for some tasks including building worker containers and testing this website
diff --git a/website/generated-content/contribute/index.xml b/website/generated-content/contribute/index.xml
index 94a1a47..337ef0d 100644
--- a/website/generated-content/contribute/index.xml
+++ b/website/generated-content/contribute/index.xml
@@ -1092,7 +1092,7 @@ Thanks everyone!
<li>
<p>Tasks included</p>
<ol>
-<li>Run Java quickstart with Direct Runner, Apex local runner, Flink local runner, Spark local runner and Dataflow runner.</li>
+<li>Run Java quickstart with Direct Runner, Flink local runner, Spark local runner and Dataflow runner.</li>
<li>Run Java Mobile Games(UserScore, HourlyTeamScore, Leaderboard) with Dataflow runner.</li>
<li>Create a PR to trigger python validation job, including
<ul>
@@ -1130,10 +1130,6 @@ Thanks everyone!
<pre><code>./gradlew :runners:direct-java:runQuickstartJavaDirect \
-Prepourl=https://repository.apache.org/content/repositories/orgapachebeam-${KEY} \
-Pver=${RELEASE_VERSION}
-</code></pre><p>Apex Local Runner</p>
-<pre><code>./gradlew :runners:apex:runQuickstartJavaApex \
--Prepourl=https://repository.apache.org/content/repositories/orgapachebeam-${KEY} \
--Pver=${RELEASE_VERSION}
</code></pre><p>Flink Local Runner</p>
<pre><code>./gradlew :runners:flink:1.10:runQuickstartJavaFlinkLocal \
-Prepourl=https://repository.apache.org/content/repositories/orgapachebeam-${KEY} \
@@ -2226,7 +2222,7 @@ One advantage of putting a parameter into transform configuration is, it can be
For a user, a transform that logged an error but succeeded is silent data loss.</li>
</ul>
<h3 id="performance">Performance</h3>
-<p>Many runners optimize chains of <code>ParDo</code>s in ways that improve performance if the <code>ParDo</code>s emit a small to moderate number of elements per input element, or have relatively cheap per-element processing (e.g. Dataflow&rsquo;s &ldquo;fusion&rdquo;, Apex &ldquo;compute locality&rdquo;), but limit parallelization if these assumptions are violated. In that case you may need a &ldquo;fusion break&rdquo; (<code>Reshuffle.of() [...]
+<p>Many runners optimize chains of <code>ParDo</code>s in ways that improve performance if the <code>ParDo</code>s emit a small to moderate number of elements per input element, or have relatively cheap per-element processing (e.g. Dataflow&rsquo;s &ldquo;fusion&rdquo;), but limit parallelization if these assumptions are violated. In that case you may need a &ldquo;fusion break&rdquo; (<code>Reshuffle.of()</code>) to improve the parallelizability [...]
<ul>
<li>If the transform includes a <code>ParDo</code> that outputs a potentially large number of elements per input element, apply a fusion break after this <code>ParDo</code> to make sure downstream transforms can process its output in parallel.</li>
<li>If the transform includes a <code>ParDo</code> that takes a very long time to process an element, insert a fusion break before this <code>ParDo</code> to make sure all or most elements can be processed in parallel regardless of how its input <code>PCollection</code> was produced.</li>
diff --git a/website/generated-content/contribute/ptransform-style-guide/index.html b/website/generated-content/contribute/ptransform-style-guide/index.html
index e617e73..358c43e 100644
--- a/website/generated-content/contribute/ptransform-style-guide/index.html
+++ b/website/generated-content/contribute/ptransform-style-guide/index.html
@@ -14,7 +14,7 @@ One advantage of putting a parameter into transform configuration is, it can be
*Exception: *It may be valuable to catch the error, log a message, and rethrow it, if you’re able to provide valuable context that the original error doesn’t have.</li><li>Never, ever, ever do this:
<code>catch(...) { log an error; return null or false or otherwise ignore; }</code>
<strong>Rule of thumb: if a bundle didn’t fail, its output must be correct and complete.</strong>
-For a user, a transform that logged an error but succeeded is silent data loss.</li></ul><h3 id=performance>Performance</h3><p>Many runners optimize chains of <code>ParDo</code>s in ways that improve performance if the <code>ParDo</code>s emit a small to moderate number of elements per input element, or have relatively cheap per-element processing (e.g. Dataflow’s “fusion”, Apex “compute locality”), but limit parallelization if these assumptions are violated [...]
+For a user, a transform that logged an error but succeeded is silent data loss.</li></ul><h3 id=performance>Performance</h3><p>Many runners optimize chains of <code>ParDo</code>s in ways that improve performance if the <code>ParDo</code>s emit a small to moderate number of elements per input element, or have relatively cheap per-element processing (e.g. Dataflow’s “fusion”), but limit parallelization if these assumptions are violated. In that case you may need a “ [...]
E.g. when expanding a filepattern into files, log what the filepattern was and how many parts it was split into; when executing a query, log the query and log how many results it produced.</li></ul><p>Do not:</p><ul><li>Do not log at <code>INFO</code> per element or per bundle. <code>DEBUG</code>/<code>TRACE</code> may be okay because these levels are disabled by default.</li><li>Avoid logging data payloads that may contain sensitive information, or sanitize them before logging (e.g. use [...]
Strive to make such incompatible behavior changes cause a compile error (e.g. it’s better to introduce a new transform for a new behavior and deprecate and then delete the old one (in a new major version), than to silently change the behavior of an existing transform), or at least a runtime error.</li><li>If the behavior of the transform stays the same and you’re merely changing implementation or API - do not change API of the transform in a way that will make a user’s [...]
<span class=kd>private</span> <span class=kd>final</span> <span class=n>TupleTag</span><span class=o><</span><span class=n>Moo</span><span class=o>></span> <span class=n>mooTag</span> <span class=o>=</span> <span class=k>new</span> <span class=n>TupleTag</span><span class=o><</span><span class=n>Moo</span><span class=o>>()</span> <span class=o>{};</span>
diff --git a/website/generated-content/contribute/release-guide/index.html b/website/generated-content/contribute/release-guide/index.html
index 479f258..0d428db 100644
--- a/website/generated-content/contribute/release-guide/index.html
+++ b/website/generated-content/contribute/release-guide/index.html
@@ -250,10 +250,7 @@ There are no disapproving votes.
Thanks everyone!
</code></pre><h3 id=run-validation-tests>Run validation tests</h3><p>All tests listed in this <a href=https://s.apache.org/beam-release-validation>spreadsheet</a></p><p>Since there are a bunch of tests, we recommend you running validations using automation script. In case of script failure, you can still run all of them manually.</p><h4 id=run-validations-using-run_rc_validationsh>Run validations using run_rc_validation.sh</h4><ul><li><p>Script: <a href=https://github.com/apache/beam/blo [...]
<a href=https://github.com/apache/beam/blob/master/release/src/main/scripts/script.config>script.config</a></li><li>Then run<pre><code>./beam/release/src/main/scripts/run_rc_validation.sh
-</code></pre></li></ol></li><li><p>Tasks included</p><ol><li>Run Java quickstart with Direct Runner, Apex local runner, Flink local runner, Spark local runner and Dataflow runner.</li><li>Run Java Mobile Games(UserScore, HourlyTeamScore, Leaderboard) with Dataflow runner.</li><li>Create a PR to trigger python validation job, including<ul><li>Python quickstart in batch and streaming mode with direct runner and Dataflow runner.</li><li>Python Mobile Games(UserScore, HourlyTeamScore) with d [...]
--Prepourl=https://repository.apache.org/content/repositories/orgapachebeam-${KEY} \
--Pver=${RELEASE_VERSION}
-</code></pre><p>Apex Local Runner</p><pre><code>./gradlew :runners:apex:runQuickstartJavaApex \
+</code></pre></li></ol></li><li><p>Tasks included</p><ol><li>Run Java quickstart with Direct Runner, Flink local runner, Spark local runner and Dataflow runner.</li><li>Run Java Mobile Games(UserScore, HourlyTeamScore, Leaderboard) with Dataflow runner.</li><li>Create a PR to trigger python validation job, including<ul><li>Python quickstart in batch and streaming mode with direct runner and Dataflow runner.</li><li>Python Mobile Games(UserScore, HourlyTeamScore) with direct runner and Da [...]
-Prepourl=https://repository.apache.org/content/repositories/orgapachebeam-${KEY} \
-Pver=${RELEASE_VERSION}
</code></pre><p>Flink Local Runner</p><pre><code>./gradlew :runners:flink:1.10:runQuickstartJavaFlinkLocal \
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/index.html
index c420b8a..642a4e8 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Learn about Beam</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Speci [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/documentation/runners/apex/index.html b/website/generated-content/documentation/runners/apex/index.html
deleted file mode 100644
index 5039117..0000000
--- a/website/generated-content/documentation/runners/apex/index.html
+++ /dev/null
@@ -1,22 +0,0 @@
-<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Apache Apex Runner</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Spe [...]
-<span class=sr-only>Toggle navigation</span>
-<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
-They are not required for Apex in embedded mode (see <a href=/get-started/quickstart-java/>quickstart</a>).</p><h2 id=apex-runner-prerequisites>Apex Runner prerequisites</h2><p>You may set up your own Hadoop cluster. Beam does not require anything extra to launch the pipelines on YARN.
-An optional Apex installation may be useful for monitoring and troubleshooting.
-The Apex CLI can be <a href=https://apex.apache.org/docs/apex/apex_development_setup/>built</a> or
-obtained as binary build.
-For more download options see <a href=https://apex.apache.org/downloads.html>distribution information on the Apache Apex website</a>.</p><h2 id=running-wordcount-with-apex>Running wordcount with Apex</h2><p>Typically the build environment is separate from the target YARN cluster. In such case, it is necessary to build a fat jar that will include all dependencies. Ensure that <code>hadoop.version</code> in <code>pom.xml</code> matches the version of your YARN cluster and then build the ja [...]
-</code></pre><p>Copy the resulting <code>target/word-count-beam-bundled-0.1.jar</code> to the cluster and submit the application using:</p><pre><code>java -cp word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount --inputFile=/etc/profile --output=/tmp/counts --embeddedExecution=false --configFile=beam-runners-apex.properties --runner=ApexRunner
-</code></pre><p>If the build environment is setup as cluster client, it is possible to run the example directly:</p><pre><code>mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="--inputFile=/etc/profile --output=/tmp/counts --runner=ApexRunner --embeddedExecution=false --configFile=beam-runners-apex.properties" -Papex-runner
-</code></pre><p>The application will run asynchronously. Check status with <code>yarn application -list -appStates ALL</code></p><p>The configuration file is optional, it can be used to influence how Apex operators are deployed into YARN containers.
-The following example will reduce the number of required containers by collocating the operators into the same container
-and lower the heap memory per operator - suitable for execution in a single node Hadoop sandbox.</p><pre><code>apex.application.*.operator.*.attr.MEMORY_MB=64
-apex.stream.*.prop.locality=CONTAINER_LOCAL
-apex.application.*.operator.*.attr.TIMEOUT_WINDOW_COUNT=1200
-</code></pre><p>This example uses local files. To use a distributed file system (HDFS, S3 etc.),
-it is necessary to augment the build to include the respective file system provider.</p><h2 id=montoring-progress-of-your-job>Montoring progress of your job</h2><p>Depending on your installation, you may be able to monitor the progress of your job on the Hadoop cluster. Alternatively, you have following options:</p><ul><li>YARN : Using YARN web UI generally running on 8088 on the node running resource manager.</li><li>Apex command-line interface: <a href=https://apex.apache.org/docs/apex [...]
-</code></pre></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=footer__cols__col><div class=footer__cols__col__logo><img src=/images/beam_logo_circle.svg class=footer__logo alt="Beam logo"></div><div class=footer__cols__col__logo><img src=/images/apache_logo_circle.svg class=footer__logo alt="Apache logo"></div></div><div class="footer__cols__col footer__cols__col--md"><div class=footer__cols__col__title>Start</div><div class=footer__cols__c [...]
-<a href=http://www.apache.org>The Apache Software Foundation</a>
-| <a href=/privacy_policy>Privacy Policy</a>
-| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/documentation/runners/capability-matrix/index.html b/website/generated-content/documentation/runners/capability-matrix/index.html
index f4ac9ef..fbb5dcc 100644
--- a/website/generated-content/documentation/runners/capability-matrix/index.html
+++ b/website/generated-content/documentation/runners/capability-matrix/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Apache Beam Capability Matrix</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
@@ -12,7 +12,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>GroupByKey</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
@@ -24,7 +23,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Flatten</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
@@ -36,7 +34,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Combine</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
@@ -48,13 +45,11 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Composite Transforms</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
@@ -71,14 +66,12 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#ec3>Source API</th><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
@@ -90,7 +83,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ec3;border-color:#ca1><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b>
@@ -102,7 +94,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
@@ -118,10 +109,9 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#ca1><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#fe5;border-color:#ca1><center><b>~</b>
-<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-where><th class="cap-summary color-metadata format-category" colspan=8 style=color:#37d><div class=cap-toggle onclick='ToggleTables(1,"cap-full-where")'>(click to expand details)</div>Where in event time?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" styl [...]
+<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-where><th class="cap-summary color-metadata format-category" colspan=8 style=color:#37d><div class=cap-toggle onclick='ToggleTables(1,"cap-full-where")'>(click to expand details)</div>Where in event time?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" styl [...]
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
@@ -132,7 +122,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#59f>Fixed windows</th><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
@@ -144,7 +133,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#59f>Sliding windows</th><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
@@ -156,7 +144,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#59f>Session windows</th><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
@@ -168,7 +155,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#59f>Custom windows</th><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
@@ -180,7 +166,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#59f>Custom merging windows</th><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
@@ -192,7 +177,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#59f>Timestamp control</th><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
@@ -204,13 +188,11 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#59f;border-color:#37d><center><b>✓</b>
-<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-when><th class="cap-summary color-metadata format-category" colspan=8 style=color:#6a4><div class=cap-toggle onclick='ToggleTables(1,"cap-full-when")'>(click to expand details)</div>When in processing time?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" st [...]
+<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-when><th class="cap-summary color-metadata format-category" colspan=8 style=color:#6a4><div class=cap-toggle onclick='ToggleTables(1,"cap-full-when")'>(click to expand details)</div>When in processing time?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" st [...]
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
@@ -222,7 +204,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
@@ -234,7 +215,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
@@ -246,7 +226,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
@@ -264,13 +243,11 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#8c6>Composite triggers</th><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
@@ -282,7 +259,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#8c6;border-color:#6a4><center><b>✓</b>
@@ -295,18 +271,16 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#6a4><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ae8;border-color:#6a4><center><b>~</b>
-<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-how><th class="cap-summary color-metadata format-category" colspan=8 style=color:#b55><div class=cap-toggle onclick='ToggleTables(1,"cap-full-how")'>(click to expand details)</div>How do refinements relate?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" st [...]
+<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-how><th class="cap-summary color-metadata format-category" colspan=8 style=color:#b55><div class=cap-toggle onclick='ToggleTables(1,"cap-full-how")'>(click to expand details)</div>How do refinements relate?</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary color-platform format-platform" st [...]
<b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#faa;border-color:#b55><center><b>~</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b>
@@ -318,7 +292,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#d77;border-color:#b55><center><b>✓</b>
@@ -336,8 +309,7 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#b55><center><b>✕</b>
-<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-misc><th class="cap-summary color-metadata format-category" colspan=8 style=color:#aaa><div class=cap-toggle onclick='ToggleTables(1,"cap-full-misc")'>(click to expand details)</div>Additional common features not yet part of the Beam model</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary c [...]
+<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr><tr class=cap-summary id=cap-summary-misc><th class="cap-summary color-metadata format-category" colspan=8 style=color:#aaa><div class=cap-toggle onclick='ToggleTables(1,"cap-full-misc")'>(click to expand details)</div>Additional common features not yet part of the Beam model</th></tr><tr class=cap-summary><th class="cap-summary color-capability"></th><th class="cap-summary c [...]
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ccc;border-color:#aaa><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ccc;border-color:#aaa><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
@@ -347,7 +319,6 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
<b></b></center></td></tr><tr class=cap-summary><th class="cap-summary color-capability format-capability" style=color:#bbb>Checkpoint</th><td width=25% class=cap-summary style=background-color:#ccc;border-color:#aaa><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ccc;border-color:#aaa><center><b>~</b>
@@ -356,17 +327,15 @@
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
-<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ccc;border-color:#aaa><center><b>~</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
<b></b></center></td><td width=25% class=cap-summary style=background-color:#ddd;border-color:#aaa><center><b>✕</b>
-<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr></table></div><div id=cap-full style=display:none><table class=cap><tr class=cap id=cap-full-what><th class="cap color-metadata format-category" colspan=8 style=color:#ca1><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-what")'>(click to collapse details)</div>What is being computed?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color- [...]
+<b></b></center></td></tr><tr class=cap-summary><td class="cap-summary color-blank cap-blank" colspan=5></td></tr></table></div><div id=cap-full style=display:none><table class=cap><tr class=cap id=cap-full-what><th class="cap color-metadata format-category" colspan=8 style=color:#ca1><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-what")'>(click to collapse details)</div>What is being computed?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color- [...]
: element-wise processing</b></center><br>Element-wise transformation parameterized by a chunk of user code. Elements are processed in bundles, with initialization and termination hooks. Bundle size is chosen by the runner and cannot be controlled by user code. ParDo processes a main input PCollection one element at a time, but provides side input access to additional PCollections.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br>Batch mode uses large bundle sizes. Streaming uses smaller bundle sizes.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br>ParDo itself, as per-element transformation with UDFs, is fully supported by Flink for both batch and streaming.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br>ParDo applies per-element transformations as Spark FlatMapFunction.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: fully supported in batch mode</b></center><br>ParDo applies per-element transformations as Spark FlatMapFunction.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
-: fully supported</b></center><br>Supported through Apex operator that wraps the function and processes data as single element bundles.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
@@ -378,7 +347,6 @@
: fully supported</b></center><br>Uses Flink's keyBy for key grouping. When grouping by window in streaming (creating the panes) the Flink runner uses the Beam code. This guarantees support for all windowing and triggering mechanisms.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: fully supported in batch mode</b></center><br>Using Spark's <tt>groupByKey</tt>. GroupByKey with multiple trigger firings in streaming mode is a work in progress.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: fully supported in batch mode</b></center><br>Using Spark's <tt>groupByKey</tt>.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
-: fully supported</b></center><br>Apex runner uses the Beam code for grouping by window and thereby has support for all windowing and triggering mechanisms. Runner does not implement partitioning yet (BEAM-838)</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
@@ -395,14 +363,12 @@
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
-: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#ec3>Combine</th><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: associative &amp; commutative aggregation</b></center><br>Application of an associative, commutative operation over all values ("globally") or over all values associated with each key ("per key"). Can be implemented using ParDo, but often more efficient implementations exist.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: efficient execution</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br>Uses a combiner for pre-aggregation for batch and streaming.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br>Using Spark's <tt>combineByKey</tt> and <tt>aggregate</tt> functions.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: fully supported in batch mode</b></center><br>Using Spark's <tt>Aggregator</tt> and agg function</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
-: fully supported</b></center><br>Default Beam translation. Currently no efficient pre-aggregation (BEAM-935).</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
@@ -413,8 +379,7 @@
: supported via inlining</b></center><br>Currently composite transformations are inlined during execution. The structure is later recreated from the names, but other transform level information (if added to the model) will be lost.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: supported via inlining</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: supported via inlining</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
-: supported via inlining only in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
-: supported via inlining</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
+: supported via inlining only in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: supported via inlining</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
@@ -426,7 +391,6 @@
: some size restrictions in streaming</b></center><br>Batch mode supports a distributed implementation, but streaming mode may force some size restrictions. Neither mode is able to push lookups directly up into key-based sources.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br>Using Spark's broadcast variables. In streaming mode, side inputs may update but only between micro-batches.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: fully supported in batch mode</b></center><br>Using Spark's broadcast variables.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
-: size restrictions</b></center><br>No distributed implementation and therefore size restrictions.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: some size restrictions</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
@@ -437,8 +401,7 @@
: fully supported</b></center><br>Support includes autotuning features (https://cloud.google.com/dataflow/service/dataflow-service-desc#autotuning-features).</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
-: bounded source only</b></center><br>Using Spark's DatasourceV2 API in microbatch mode (Continuous streaming mode is tagged experimental in spark and does not support aggregation).</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
-: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
+: bounded source only</b></center><br>Using Spark's DatasourceV2 API in microbatch mode (Continuous streaming mode is tagged experimental in spark and does not support aggregation).</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: bounded source only</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
@@ -449,8 +412,7 @@
:</b></center><br>Does not yet support autotuning features of the Source API.</td><td width=25% class=cap style=background-color:#ec3;border-color:#ca1><center><b>Yes
:</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: supports bounded-per-element SDFs</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No
-: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
-: supports bounded-per-element SDFs</b></center><br>implementation in streaming mode coming soon</td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No
+: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No
: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No
: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No
: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
@@ -460,8 +422,7 @@
: user-provided metrics</b></center><br>Allow transforms to gather simple metrics across bundles in a <tt>PTransform</tt>. Provide a mechanism to obtain both committed and attempted metrics. Semantically similar to using an additional output, but support partial results as the transform executes, and support both committed and attempted values. Will likely want to augment <tt>Metrics</tt> to be more useful for processing unbounded data by making them windowed.</td><td width=2 [...]
: All metrics types are supported.</b></center><br>Only attempted values are supported. No committed values for metrics.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: All metric types are supported.</b></center><br>Only attempted values are supported. No committed values for metrics.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
-: All metric types are supported in batch mode.</b></center><br>Only attempted values are supported. No committed values for metrics.</td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No
-: Not implemented in runner.</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
+: All metric types are supported in batch mode.</b></center><br>Only attempted values are supported. No committed values for metrics.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: Only attempted counters are supported</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: Metrics are only supported in local mode.</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: All metrics types are supported.</b></center><br>Only attempted values are supported. No committed values for metrics.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
@@ -473,13 +434,12 @@
: non-merging windows</b></center><br>State is supported for non-merging windows. SetState and MapState are not yet supported.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: full support in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No
: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
-: non-merging windows</b></center><br>State is supported for non-merging windows. SetState and MapState are not yet supported.</td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: non-merging windows</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: non-merging windows</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: non-merging windows</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
: non-merging windows</b></center><br>States are backed up by either rocksDb KV store or in-memory hash map, and persist using changelog.</td><td width=25% class=cap style=background-color:#ddd;border-color:#ca1><center><b>No
: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#fe5;border-color:#ca1><center><b>Partially
-: non-merging windows</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-where><th class="cap color-metadata format-category" colspan=8 style=color:#37d><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-where")'>(click to collapse details)</div>Where in event time?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#59f>Beam Model</th><th [...]
+: non-merging windows</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-where><th class="cap color-metadata format-category" colspan=8 style=color:#37d><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-where")'>(click to collapse details)</div>Where in event time?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#59f>Beam Model</th><th [...]
: all time</b></center><br>The default window which covers all of time. (Basically how traditional batch cases fit in the model.)</td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: default</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
@@ -490,7 +450,6 @@
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
-: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#59f>Fixed windows</th><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: periodic, non-overlapping</b></center><br>Fixed-size, timestamp-based windows. (Hourly, Daily, etc)</td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: built-in</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
@@ -502,7 +461,6 @@
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
-: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#59f>Sliding windows</th><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: periodic, overlapping</b></center><br>Possibly overlapping fixed-size timestamp-based windows (Every minute, use the last ten minutes of data.)</td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: built-in</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
@@ -514,7 +472,6 @@
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
-: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#59f>Session windows</th><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: activity-based</b></center><br>Based on bursts of activity separated by a gap size. Different per key.</td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: built-in</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
@@ -526,7 +483,6 @@
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
-: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#59f>Custom windows</th><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: user-defined windows</b></center><br>All windows must implement <tt>BoundedWindow</tt>, which specifies a max timestamp. Each <tt>WindowFn</tt> assigns elements to an associated window.</td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
@@ -538,7 +494,6 @@
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
-: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#59f>Custom merging windows</th><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: user-defined merging windows</b></center><br>A custom <tt>WindowFn</tt> additionally specifies whether and how to merge windows.</td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
@@ -550,7 +505,6 @@
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
-: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#59f>Timestamp control</th><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: output timestamp for window panes</b></center><br>For a grouping transform, such as GBK or Combine, an OutputTimeFn specifies (1) how to combine input timestamps within a window and (2) how to merge aggregated timestamps when windows merge.</td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
@@ -562,14 +516,12 @@
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
-: supported</b></center><br></td><td width=25% class=cap style=background-color:#59f;border-color:#37d><center><b>Yes
-: supported</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-when><th class="cap color-metadata format-category" colspan=8 style=color:#6a4><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-when")'>(click to collapse details)</div>When in processing time?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#8c6>Beam Model</th><th class="c [...]
+: supported</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-when><th class="cap color-metadata format-category" colspan=8 style=color:#6a4><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-when")'>(click to collapse details)</div>When in processing time?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#8c6>Beam Model</th><th class="c [...]
: user customizable</b></center><br>Triggering may be specified by the user (instead of simply driven by hardcoded defaults).</td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br>Fully supported in streaming mode. In batch mode, intermediate trigger firings are effectively meaningless.</td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
-: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
-: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
+: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
: batch-only runner</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
@@ -580,8 +532,7 @@
: yes in streaming, fixed granularity in batch</b></center><br>Fully supported in streaming mode. In batch mode, currently watermark progress jumps from the beginning of time to the end of time once the input has been fully consumed, thus no additional triggering granularity is available.</td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
-: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
-: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
+: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
@@ -591,8 +542,7 @@
: yes in streaming, fixed granularity in batch</b></center><br>Fully supported in streaming mode. In batch mode, from the perspective of triggers, processing time currently jumps from the beginning of time to the end of time once the input has been fully consumed, thus no additional triggering granularity is available.</td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: This is Spark streaming's native model</b></center><br>Spark processes streams in micro-batches. The micro-batch size is actually a pre-set, fixed, time interval. Currently, the runner takes the first window size in the pipeline and sets it's size as the batch interval. Any following window operations will be considered processing time windows and will affect triggering.</td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
-: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
-: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
+: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
@@ -602,8 +552,7 @@
: fully supported</b></center><br>Fully supported in streaming mode. In batch mode, elements are processed in the largest bundles possible, so count-based triggers are effectively meaningless.</td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
-: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
-: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
+: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
@@ -613,7 +562,6 @@
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
-: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
@@ -624,8 +572,7 @@
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
-: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
-: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
+: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
@@ -634,8 +581,7 @@
: event-time bound on window lifetimes</b></center><br>A way to bound the useful lifetime of a window (in event time), after which any unemitted results may be materialized, the window contents may be garbage collected, and any addtional late data that arrive for the window may be discarded.</td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br>Fully supported in streaming mode. In batch mode no data is ever late.</td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
-: no streaming support in the runner</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
-: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
+: no streaming support in the runner</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#8c6;border-color:#6a4><center><b>Yes
@@ -645,19 +591,17 @@
: non-merging windows</b></center><br>Dataflow supports timers in non-merging windows.</td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
: non-merging windows</b></center><br>The Flink Runner supports timers in non-merging windows.</td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
-: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
: non-merging windows</b></center><br></td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
: non-merging windows</b></center><br></td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
: non-merging windows</b></center><br>The Samza Runner supports timers in non-merging windows.</td><td width=25% class=cap style=background-color:#ddd;border-color:#6a4><center><b>No
: not implemented</b></center><br></td><td width=25% class=cap style=background-color:#ae8;border-color:#6a4><center><b>Partially
-: non-merging windows</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-how><th class="cap color-metadata format-category" colspan=8 style=color:#b55><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-how")'>(click to collapse details)</div>How do refinements relate?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#d77>Beam Model</th><t [...]
+: non-merging windows</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-how><th class="cap color-metadata format-category" colspan=8 style=color:#b55><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-how")'>(click to collapse details)</div>How do refinements relate?</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" style=color:#d77>Beam Model</th><t [...]
: panes discard elements when fired</b></center><br>Elements are discarded from accumulated state as their pane is fired.</td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
: fully supported</b></center><br>Spark streaming natively discards elements after firing.</td><td width=25% class=cap style=background-color:#faa;border-color:#b55><center><b>Partially
-: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
-: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
+: fully supported in batch mode</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
: batch-only runner</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
@@ -666,8 +610,7 @@
: fully supported</b></center><br></td></tr><tr class=cap><th class="cap color-capability format-capability" style=color:#d77>Accumulating</th><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
: panes accumulate elements across firings</b></center><br>Elements are accumulated in state across multiple pane firings for the same window.</td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
: fully supported</b></center><br>Requires that the accumulated pane fits in memory, after being passed through the combiner (if relevant)</td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
-: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
-: fully supported</b></center><br>Size restriction, see combine support.</td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
+: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
: fully supported</b></center><br></td><td width=25% class=cap style=background-color:#d77;border-color:#b55><center><b>Yes
@@ -677,21 +620,20 @@
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
-: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
: pending model support</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#b55><center><b>No
-: pending model support</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-misc><th class="cap color-metadata format-category" colspan=8 style=color:#aaa><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-misc")'>(click to collapse details)</div>Additional common features not yet part of the Beam model</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" [...]
+: pending model support</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr><tr class=cap id=cap-full-misc><th class="cap color-metadata format-category" colspan=8 style=color:#aaa><div class=cap-toggle onclick='ToggleTables(0,"cap-summary-misc")'>(click to collapse details)</div>Additional common features not yet part of the Beam model</th></tr><tr class=cap><th class="cap color-capability"></th><th class="cap color-platform format-platform" [...]
:</b></center><br>APIs and semantics for draining a pipeline are under discussion. This would cause incomplete aggregations to be emitted regardless of trigger and tagged with metadata indicating it is incompleted.</td><td width=25% class=cap style=background-color:#ccc;border-color:#aaa><center><b>Partially
:</b></center><br>Dataflow has a native drain operation, but it does not work in the presence of event time timer loops. Final implemention pending model support.</td><td width=25% class=cap style=background-color:#ccc;border-color:#aaa><center><b>Partially
-:</b></center><br>Flink supports taking a "savepoint" of the pipeline and shutting the pipeline down after its completion.</td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;b [...]
+:</b></center><br>Flink supports taking a "savepoint" of the pipeline and shutting the pipeline down after its completion.</td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;b [...]
:</b></center><br>APIs and semantics for saving a pipeline checkpoint are under discussion. This would be a runner-specific materialization of the pipeline state required to resume or duplicate the pipeline.</td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>No
:</b></center><br></td><td width=25% class=cap style=background-color:#ccc;border-color:#aaa><center><b>Partially
:</b></center><br>Flink has a native savepoint capability.</td><td width=25% class=cap style=background-color:#ccc;border-color:#aaa><center><b>Partially
:</b></center><br>Spark has a native savepoint capability.</td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>No
-:</b></center><br>not implemented</td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=backgrou [...]
+:</b></center><br>not implemented</td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ccc;border-color:#aaa><center><b>Partially
:</b></center><br>Samza has a native checkpoint capability.</td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td><td width=25% class=cap style=background-color:#ddd;border-color:#aaa><center><b>:</b></center><br></td></tr><tr class=cap><td class="cap color-blank cap-blank" colspan=5></td></tr></table></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=footer__cols__col><div class=foo [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
diff --git a/website/generated-content/documentation/runners/dataflow/index.html b/website/generated-content/documentation/runners/dataflow/index.html
index 35ec3b2..4b9a402 100644
--- a/website/generated-content/documentation/runners/dataflow/index.html
+++ b/website/generated-content/documentation/runners/dataflow/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Cloud Dataflow Runner</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
begin</em> section of the <a href=https://cloud.google.com/dataflow/docs/quickstarts>Cloud Dataflow quickstart</a>
for your chosen language.</p><ol><li>Select or create a Google Cloud Platform Console project.</li><li>Enable billing for your project.</li><li>Enable the required Google Cloud APIs: Cloud Dataflow, Compute Engine,
Stackdriver Logging, Cloud Storage, Cloud Storage JSON, and Cloud Resource
diff --git a/website/generated-content/documentation/runners/direct/index.html b/website/generated-content/documentation/runners/direct/index.html
index 8857073..b8a5639 100644
--- a/website/generated-content/documentation/runners/direct/index.html
+++ b/website/generated-content/documentation/runners/direct/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Direct Runner</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<span class=o><</span><span class=n>groupId</span><span class=o>></span><span class=n>org</span><span class=o>.</span><span class=na>apache</span><span class=o>.</span><span class=na>beam</span><span class=o></</span><span class=n>groupId</span><span class=o>></span>
<span class=o><</span><span class=n>artifactId</span><span class=o>></span><span class=n>beam</span><span class=o>-</span><span class=n>runners</span><span class=o>-</span><span class=n>direct</span><span class=o>-</span><span class=n>java</span><span class=o></</span><span class=n>artifactId</span><span class=o>></span>
<span class=o><</span><span class=n>version</span><span class=o>></span><span class=n>2</span><span class=o>.</span><span class=na>22</span><span class=o>.</span><span class=na>0</span><span class=o></</span><span class=n>version</span><span class=o>></span>
diff --git a/website/generated-content/documentation/runners/flink/index.html b/website/generated-content/documentation/runners/flink/index.html
index 7cd3aed..a4f4ffa 100644
--- a/website/generated-content/documentation/runners/flink/index.html
+++ b/website/generated-content/documentation/runners/flink/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Apache Flink Runner</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Sp [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
Flink</a>. For execution you can choose between a cluster
execution mode (e.g. Yarn/Kubernetes/Mesos) or a local embedded execution mode
which is useful for testing pipelines.</p><p>The Flink Runner and Flink are suitable for large scale, continuous jobs, and provide:</p><ul><li>A streaming-first runtime that supports both batch processing and data streaming programs</li><li>A runtime that supports very high throughput and low event latency at the same time</li><li>Fault-tolerance with <em>exactly-once</em> processing guarantees</li><li>Natural back-pressure in streaming programs</li><li>Custom memory management for effic [...]
diff --git a/website/generated-content/documentation/runners/jet/index.html b/website/generated-content/documentation/runners/jet/index.html
index d43999e..9af282a 100644
--- a/website/generated-content/documentation/runners/jet/index.html
+++ b/website/generated-content/documentation/runners/jet/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Hazelcast Jet Runner</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain S [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
Jet</a>.</p><p>The Jet Runner and Jet are suitable for large scale continuous jobs and provide:</p><ul><li>Support for both batch (bounded) and streaming (unbounded) data sets</li><li>A runtime that supports very high throughput and low event latency at the same time</li><li>Natural back-pressure in streaming programs</li><li>Distributed massively parallel data processing engine with in memory storage</li></ul><p>It’s important to note that the Jet Runner is currently in an <em>EXP [...]
the capabilities present in Jet:</p><ul><li>Jet has full Fault Tolerance support, the Jet Runner does not; if a job fails it must be restarted</li><li>Internal performance of Jet is extremely high.
The Runner can’t match it as of now because Beam pipeline optimization/surgery has not been fully implemented.</li></ul><p>The <a href=/documentation/runners/capability-matrix/>Beam Capability Matrix</a> documents the
diff --git a/website/generated-content/documentation/runners/mapreduce/index.html b/website/generated-content/documentation/runners/mapreduce/index.html
index c099e6f..62402bc 100644
--- a/website/generated-content/documentation/runners/mapreduce/index.html
+++ b/website/generated-content/documentation/runners/mapreduce/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Apache Hadoop MapReduce Runner</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) an [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-mapreduce</artifactId>
<version>2.22.0</version>
diff --git a/website/generated-content/documentation/runners/nemo/index.html b/website/generated-content/documentation/runners/nemo/index.html
index e16608f..52584d6 100644
--- a/website/generated-content/documentation/runners/nemo/index.html
+++ b/website/generated-content/documentation/runners/nemo/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Apache Nemo Runner</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Spe [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
The Nemo Runner can optimize Beam pipelines with the Nemo compiler through various optimization passes
and execute them in a distributed fashion using the Nemo runtime. You can also deploy a self-contained application
for local mode or run using resource managers like YARN or Mesos.</p><p>The Nemo Runner executes Beam pipelines on top of Apache Nemo, providing:</p><ul><li>Batch and streaming pipelines</li><li>Fault-tolerance</li><li>Integration with YARN and other components of the Apache Hadoop ecosystem</li><li>Support for the various optimizations provided by the Nemo optimizer</li></ul><p>The <a href=/documentation/runners/capability-matrix/>Beam Capability Matrix</a> documents the
diff --git a/website/generated-content/documentation/runners/samza/index.html b/website/generated-content/documentation/runners/samza/index.html
index 0d2dc06..a1d2c25 100644
--- a/website/generated-content/documentation/runners/samza/index.html
+++ b/website/generated-content/documentation/runners/samza/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Apache Samza Runner</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Sp [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<span class=o><</span><span class=n>groupId</span><span class=o>></span><span class=n>org</span><span class=o>.</span><span class=na>apache</span><span class=o>.</span><span class=na>beam</span><span class=o></</span><span class=n>groupId</span><span class=o>></span>
<span class=o><</span><span class=n>artifactId</span><span class=o>></span><span class=n>beam</span><span class=o>-</span><span class=n>runners</span><span class=o>-</span><span class=n>samza</span><span class=o></</span><span class=n>artifactId</span><span class=o>></span>
<span class=o><</span><span class=n>version</span><span class=o>></span><span class=n>2</span><span class=o>.</span><span class=na>22</span><span class=o>.</span><span class=na>0</span><span class=o></</span><span class=n>version</span><span class=o>></span>
diff --git a/website/generated-content/documentation/runners/spark/index.html b/website/generated-content/documentation/runners/spark/index.html
index bee1c25..1ec59d1 100644
--- a/website/generated-content/documentation/runners/spark/index.html
+++ b/website/generated-content/documentation/runners/spark/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Apache Spark Runner</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Sp [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
The Spark Runner can execute Spark pipelines just like a native Spark application; deploying a self-contained application for local mode, running on Spark’s Standalone RM, or using YARN or Mesos.</p><p>The Spark Runner executes Beam pipelines on top of Apache Spark, providing:</p><ul><li>Batch and streaming (and combined) pipelines.</li><li>The same fault-tolerance <a href=https://spark.apache.org/docs/latest/streaming-programming-guide.html#fault-tolerance-semantics>guarantees</a> [...]
the portable functionality of the Spark Runner. Please use the switcher below to
select the appropriate Runner:</p><h2 id=which-runner-to-use-portable-or-non-portable-runner>Which runner to use: portable or non portable runner?</h2><p>Beam and its Runners originally only supported JVM-based languages
diff --git a/website/generated-content/documentation/sdks/java/testing/nexmark/index.html b/website/generated-content/documentation/sdks/java/testing/nexmark/index.html
index 2c7d670..ddf2e86 100644
--- a/website/generated-content/documentation/sdks/java/testing/nexmark/index.html
+++ b/website/generated-content/documentation/sdks/java/testing/nexmark/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Nexmark benchmark suite</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domai [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
queries in <a href=https://web.archive.org/web/20100620010601/http://datalab.cs.pdx.edu/niagaraST/NEXMark/>Nexmark research
paper</a></p><p>These are multiple queries over a three entities model representing on online
auction system:</p><ul><li><strong>Person</strong> represents a person submitting an item for auction and/or making
@@ -61,8 +61,7 @@ or may be published to Pub/Sub or Kafka.</p><p>The query results may be:</p><ul>
</code></pre><p>Queries can be run by their name or by their number (number is still there for backward compatibility, only the queries 0 to 12 have a number)</p><p>Run query <strong>N</strong>:</p><pre><code>--query=N
</code></pre><p>Run query called <strong>PASSTHROUGH</strong>:</p><pre><code>--query=PASSTHROUGH
</code></pre><h3 id=available-suites>Available Suites</h3><p>The suite to run can be chosen using this configuration parameter:</p><pre><code>--suite=SUITE
-</code></pre><p>Available suites are:</p><ul><li>DEFAULT: Test default configuration with query 0.<ul><li>SMOKE: Run all the queries with the default configuration.</li></ul></li><li>STRESS: Like smoke but for 1m events.</li><li>FULL_THROTTLE: Like SMOKE but 100m events.</li></ul><h3 id=apex-runner-specific-configuration>Apex runner specific configuration</h3><pre><code>--manageResources=false --monitorJobs=false
-</code></pre><h3 id=google-cloud-dataflow-runner-specific-configuration>Google Cloud Dataflow runner specific configuration</h3><pre><code>--manageResources=false --monitorJobs=true \
+</code></pre><p>Available suites are:</p><ul><li>DEFAULT: Test default configuration with query 0.<ul><li>SMOKE: Run all the queries with the default configuration.</li></ul></li><li>STRESS: Like smoke but for 1m events.</li><li>FULL_THROTTLE: Like SMOKE but 100m events.</li></ul><h3 id=google-cloud-dataflow-runner-specific-configuration>Google Cloud Dataflow runner specific configuration</h3><pre><code>--manageResources=false --monitorJobs=true \
--enforceEncodability=false --enforceImmutability=false
--project=<your project> \
--zone=<your zone> \
@@ -83,7 +82,7 @@ or may be published to Pub/Sub or Kafka.</p><p>The query results may be:</p><ul>
</code></pre><p>Set topic name which will be used for benchmark results:</p><pre><code>--kafkaResultsTopic=<topic name>
</code></pre><p>Write or/and read events into/from Kafka topic:</p><pre><code>--sourceType=KAFKA
</code></pre><p>Set topic name which will be used for benchmark events:</p><pre><code>--kafkaTopic=<topic name>
-</code></pre><h2 id=current-status>Current status</h2><p>These tables contain statuses of the queries runs in the different runners. Google Cloud Dataflow status is yet to come.</p><h3 id=batch--synthetic--local>Batch / Synthetic / Local</h3><table class="table table-bordered"><tr><th>Query</th><th>Direct</th><th>Spark</th><th>Flink</th><th>Apex</th></tr><tr><td>0</td><td>ok</td><td>ok</td><td>ok</td><td>ok</td></tr><tr><td>1</td><td>ok</td><td>ok</td><td>ok</td><td>ok</td></tr><tr><td>2 [...]
+</code></pre><h2 id=current-status>Current status</h2><p>These tables contain statuses of the queries runs in the different runners. Google Cloud Dataflow status is yet to come.</p><h3 id=batch--synthetic--local>Batch / Synthetic / Local</h3><table class="table table-bordered"><tr><th>Query</th><th>Direct</th><th>Spark</th><th>Flink</th></tr><tr><td>0</td><td>ok</td><td>ok</td><td>ok</td></tr><tr><td>1</td><td>ok</td><td>ok</td><td>ok</td></tr><tr><td>2</td><td>ok</td><td>ok</td><td>ok</ [...]
Here we do it for maximum clarity.</p><p>The direct runner does not have separate batch and streaming modes, but the
Nexmark launch does.</p><p>These parameters leave on many of the DirectRunner’s extra safety checks so the
SMOKE suite can make sure there is nothing broken in the Nexmark suite.</p><p>Batch Mode:</p><pre><code>./gradlew :sdks:java:testing:nexmark:run \
@@ -146,24 +145,6 @@ configure logging.</p><p>Batch Mode:</p><pre><code>./gradlew :sdks:java:testing:
--manageResources=false
--monitorJobs=true
--flinkMaster=[local]"
-</code></pre><h3 id=running-smoke-suite-on-the-apexrunner-local>Running SMOKE suite on the ApexRunner (local)</h3><p>Batch Mode:</p><pre><code>./gradlew :sdks:java:testing:nexmark:run \
- -Pnexmark.runner=":runners:apex" \
- -Pnexmark.args="
- --runner=ApexRunner
- --suite=SMOKE
- --streamTimeout=60
- --streaming=false
- --manageResources=false
- --monitorJobs=true"
-</code></pre><p>Streaming Mode:</p><pre><code>./gradlew :sdks:java:testing:nexmark:run \
- -Pnexmark.runner=":runners:apex" \
- -Pnexmark.args="
- --runner=ApexRunner
- --suite=SMOKE
- --streamTimeout=60
- --streaming=true
- --manageResources=false
- --monitorJobs=true"
</code></pre><h3 id=running-smoke-suite-on-google-cloud-dataflow>Running SMOKE suite on Google Cloud Dataflow</h3><p>Set these up first so the below command is valid</p><pre><code>PROJECT=<your project>
ZONE=<your zone>
STAGING_LOCATION=gs://<a GCS path for staging>
diff --git a/website/generated-content/documentation/sdks/python-streaming/index.html b/website/generated-content/documentation/sdks/python-streaming/index.html
index bfef635..88a8c22 100644
--- a/website/generated-content/documentation/sdks/python-streaming/index.html
+++ b/website/generated-content/documentation/sdks/python-streaming/index.html
@@ -62,7 +62,7 @@ example streaming pipeline. Specify your Cloud Pub/Sub project and input topic
python -m apache_beam.examples.streaming_wordcount \
--input_topic "projects/YOUR_PUBSUB_PROJECT_NAME/topics/YOUR_INPUT_TOPIC" \
--output_topic "projects/YOUR_PUBSUB_PROJECT_NAME/topics/YOUR_OUTPUT_TOPIC" \
- --streaming</code></pre></div><div class=runner-apex><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-flink-local><pre><code>See https://beam.apache.org/roadmap/portability/#python-on-flink for more information.</code></pre></div><div class=runner-flink-cluster><pre><code>See https://beam.apache.org/documentation/runners/flink/ for more information.</code></pre></div><div class=runner-spark><pre><code>See https://beam.apache.org/roadma [...]
+ --streaming</code></pre></div><div class=runner-flink-local><pre><code>See https://beam.apache.org/roadmap/portability/#python-on-flink for more information.</code></pre></div><div class=runner-flink-cluster><pre><code>See https://beam.apache.org/documentation/runners/flink/ for more information.</code></pre></div><div class=runner-spark><pre><code>See https://beam.apache.org/roadmap/portability/#python-on-spark for more information.</code></pre></div><div class=runner-dataflow><pre><c [...]
pip install apache-beam[gcp]
# DataflowRunner requires the --streaming option
diff --git a/website/generated-content/get-started/beam-overview/index.html b/website/generated-content/get-started/beam-overview/index.html
index 9c5f900..44ccaee 100644
--- a/website/generated-content/get-started/beam-overview/index.html
+++ b/website/generated-content/get-started/beam-overview/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Beam Overview</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/get-started/index.xml b/website/generated-content/get-started/index.xml
index 5cd8418..7f291d9 100644
--- a/website/generated-content/get-started/index.xml
+++ b/website/generated-content/get-started/index.xml
@@ -851,7 +851,7 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
<h1 id="apache-beam-overview">Apache Beam Overview</h1>
-<p>Apache Beam is an open source, unified model for defining both batch and streaming data-parallel processing pipelines. Using one of the open source Beam SDKs, you build a program that defines the pipeline. The pipeline is then executed by one of Beam&rsquo;s supported <strong>distributed processing back-ends</strong>, which include <a href="https://apex.apache.org">Apache Apex</a>, <a href="https://flink.apache.org">Apache Flink</a>, <a href="http://spark.a [...]
+<p>Apache Beam is an open source, unified model for defining both batch and streaming data-parallel processing pipelines. Using one of the open source Beam SDKs, you build a program that defines the pipeline. The pipeline is then executed by one of Beam&rsquo;s supported <strong>distributed processing back-ends</strong>, which include <a href="https://flink.apache.org">Apache Flink</a>, <a href="http://spark.apache.org">Apache Spark</a>, and <a href="https://c [...]
<p>Beam is particularly useful for <a href="https://en.wikipedia.org/wiki/Embarassingly_parallel">Embarrassingly Parallel</a> data processing tasks, in which the problem can be decomposed into many smaller bundles of data that can be processed independently and in parallel. You can also use Beam for Extract, Transform, and Load (ETL) tasks and pure data integration. These tasks are useful for moving data between different storage media and data sources, transforming data into a [...]
<h2 id="apache-beam-sdks">Apache Beam SDKs</h2>
<p>The Beam SDKs provide a unified programming model that can represent and transform data sets of any size, whether the input is a finite data set from a batch data source, or an infinite data set from a streaming data source. The Beam SDKs use the same classes to represent both bounded and unbounded data, and the same transforms to operate on that data. You use the Beam SDK of your choice to build a program that defines your data processing pipeline.</p>
@@ -866,7 +866,6 @@ limitations under the License.
<p>The Beam Pipeline Runners translate the data processing pipeline you define with your Beam program into the API compatible with the distributed processing back-end of your choice. When you run your Beam program, you&rsquo;ll need to specify an <a href="/documentation/runners/capability-matrix">appropriate runner</a> for the back-end where you want to execute your pipeline.</p>
<p>Beam currently supports Runners that work with the following distributed processing back-ends:</p>
<ul>
-<li>Apache Apex <img src="/images/logos/runners/apex.png" alt="Apache Apex logo"></li>
<li>Apache Flink <img src="/images/logos/runners/flink.png" alt="Apache Flink logo"></li>
<li>Apache Samza <img src="/images/logos/runners/samza.png" height="20px" alt="Apache Samza logo"></li>
<li>Apache Spark <img src="/images/logos/runners/spark.png" alt="Apache Spark logo"></li>
@@ -1085,7 +1084,7 @@ args System.getProperty(&#34;exec.args&#34;).split()
</li>
</ol>
<h2 id="run-wordcount">Run WordCount</h2>
-<p>A single Beam pipeline can run on multiple Beam <a href="/documentation#runners">runners</a>, including the <a href="/documentation/runners/apex">ApexRunner</a>, <a href="/documentation/runners/flink">FlinkRunner</a>, <a href="/documentation/runners/spark">SparkRunner</a>, <a href="/documentation/runners/nemo">NemoRunner</a>, <a href="/documentation/runners/jet">JetRunner</a>, or <a href="/documentation/runners/dataflow">DataflowRunner</a>. [...]
+<p>A single Beam pipeline can run on multiple Beam <a href="/documentation#runners">runners</a>, including the <a href="/documentation/runners/flink">FlinkRunner</a>, <a href="/documentation/runners/spark">SparkRunner</a>, <a href="/documentation/runners/nemo">NemoRunner</a>, <a href="/documentation/runners/jet">JetRunner</a>, or <a href="/documentation/runners/dataflow">DataflowRunner</a>. The <a href="/documentation/runners/direct">DirectRunner [...]
<p>After you&rsquo;ve chosen which runner you&rsquo;d like to use:</p>
<ol>
<li>Ensure you&rsquo;ve done any runner-specific setup.</li>
@@ -1104,10 +1103,6 @@ args System.getProperty(&#34;exec.args&#34;).split()
<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args=&#34;--inputFile=pom.xml --output=counts&#34; -Pdirect-runner</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
--Dexec.args=&#34;--inputFile=pom.xml --output=counts --runner=ApexRunner&#34; -Papex-runner</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args=&#34;--runner=FlinkRunner --inputFile=pom.xml --output=counts&#34; -Pflink-runner</code></pre>
@@ -1149,10 +1144,6 @@ $ java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordC
<pre><code>PS&gt; mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args=&#34;--inputFile=pom.xml --output=counts&#34; -P direct-runner</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>PS&gt; mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
--D exec.args=&#34;--inputFile=pom.xml --output=counts --runner=ApexRunner&#34; -P apex-runner</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>PS&gt; mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args=&#34;--runner=FlinkRunner --inputFile=pom.xml --output=counts&#34; -P flink-runner</code></pre>
@@ -1228,9 +1219,6 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<div class=runner-direct>
<pre><code>$ ls counts*</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>$ ls counts*</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>$ ls counts*</code></pre>
</div>
@@ -1264,15 +1252,6 @@ limitations: 1
Foundation: 1
...</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>$ cat counts*
-BEAM: 1
-have: 1
-simple: 1
-skip: 4
-PAssert: 1
-...</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>$ more counts*
The: 1
@@ -1510,9 +1489,6 @@ environment&rsquo;s directories.</p>
<div class=runner-direct>
<pre><code>python -m apache_beam.examples.wordcount --input /path/to/inputfile --output /path/to/write/counts</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>This runner is not yet available for the Python SDK.</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>Currently, running wordcount.py on Flink requires a full download of the Beam source code.
See https://beam.apache.org/roadmap/portability/#python-on-flink for more information.</code></pre>
@@ -1775,7 +1751,6 @@ limitations under the License.
<li><a href="#cloud-dataflow-runner">Cloud Dataflow Runner</a></li>
<li><a href="#apache-spark-runner">Apache Spark Runner</a></li>
<li><a href="#apache-flink-runner">Apache Flink Runner</a></li>
-<li><a href="#apache-apex-runner">Apache Apex Runner</a></li>
<li><a href="#apache-nemo-runner">Apache Nemo Runner</a></li>
</ul>
</li>
@@ -2053,10 +2028,6 @@ above section, <a href="#minimalwordcount-example">MinimalWordCount</a>.&l
<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args=&#34;--inputFile=pom.xml --output=counts&#34; -Pdirect-runner</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
--Dexec.args=&#34;--inputFile=pom.xml --output=counts --runner=ApexRunner&#34; -Papex-runner</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args=&#34;--runner=FlinkRunner --inputFile=pom.xml --output=counts&#34; -Pflink-runner</code></pre>
@@ -2096,9 +2067,6 @@ You can monitor the running job by visiting the Flink dashboard at http://&l
<div class=runner-direct>
<pre><code>python -m apache_beam.examples.wordcount --input YOUR_INPUT_FILE --output counts</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>This runner is not yet available for the Python SDK.</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>Currently, running wordcount.py on Flink requires a full download of the Beam source code.
See https://beam.apache.org/roadmap/portability/#python-on-flink for more information.</code></pre>
@@ -2137,9 +2105,6 @@ python -m apache_beam.examples.wordcount --input gs://dataflow-samples/shakespea
<pre><code>$ go install github.com/apache/beam/sdks/go/examples/wordcount
$ wordcount --input &lt;PATH_TO_INPUT_FILE&gt; --output counts</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>This runner is not yet available for the Go SDK.</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>This runner is not yet available for the Go SDK.</code></pre>
</div>
@@ -2332,10 +2297,6 @@ instrumenting your pipeline code.</p>
<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \
-Dexec.args=&#34;--output=counts&#34; -Pdirect-runner</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \
--Dexec.args=&#34;--output=counts --runner=ApexRunner&#34; -Papex-runner</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \
-Dexec.args=&#34;--runner=FlinkRunner --output=counts&#34; -Pflink-runner</code></pre>
@@ -2375,9 +2336,6 @@ You can monitor the running job by visiting the Flink dashboard at http://&l
<div class=runner-direct>
<pre><code>python -m apache_beam.examples.wordcount_debugging --input YOUR_INPUT_FILE --output counts</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>This runner is not yet available for the Python SDK.</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>This runner is not yet available for the Python SDK.</code></pre>
</div>
@@ -2412,9 +2370,6 @@ python -m apache_beam.examples.wordcount_debugging --input gs://dataflow-samples
<pre><code>$ go install github.com/apache/beam/sdks/go/examples/debugging_wordcount
$ debugging_wordcount --input &lt;PATH_TO_INPUT_FILE&gt; --output counts</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>This runner is not yet available for the Go SDK.</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>This runner is not yet available for the Go SDK.</code></pre>
</div>
@@ -2558,11 +2513,6 @@ or DEBUG significantly increases the amount of logs output.</p>
<p><strong>Note:</strong> This section is yet to be added. There is an open issue for this
(<a href="https://issues.apache.org/jira/browse/BEAM-791">BEAM-791</a>).</p>
</blockquote>
-<h4 id="apache-apex-runner">Apache Apex Runner</h4>
-<blockquote>
-<p><strong>Note:</strong> This section is yet to be added. There is an open issue for this
-(<a href="https://issues.apache.org/jira/browse/BEAM-2285">BEAM-2285</a>).</p>
-</blockquote>
<h4 id="apache-nemo-runner">Apache Nemo Runner</h4>
<p>When executing your pipeline with the <code>NemoRunner</code>, most log messages are printed
directly to your local console. You should add <code>Slf4j</code> to your class path to make
@@ -2624,10 +2574,6 @@ pipeline code into smaller sections.</p>
<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WindowedWordCount \
-Dexec.args=&#34;--inputFile=pom.xml --output=counts&#34; -Pdirect-runner</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WindowedWordCount \
--Dexec.args=&#34;--inputFile=pom.xml --output=counts --runner=ApexRunner&#34; -Papex-runner</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WindowedWordCount \
-Dexec.args=&#34;--runner=FlinkRunner --inputFile=pom.xml --output=counts&#34; -Pflink-runner</code></pre>
@@ -2670,9 +2616,6 @@ parameter. using the format <code>PROJECT:DATASET.TABLE</code> or
<div class=runner-direct>
<pre><code>python -m apache_beam.examples.windowed_wordcount --input YOUR_INPUT_FILE --output_table PROJECT:DATASET.TABLE</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>This runner is not yet available for the Python SDK.</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>This runner is not yet available for the Python SDK.</code></pre>
</div>
@@ -2707,9 +2650,6 @@ python -m apache_beam.examples.windowed_wordcount --input YOUR_INPUT_FILE \
<pre><code>$ go install github.com/apache/beam/sdks/go/examples/windowed_wordcount
$ windowed_wordcount --input &lt;PATH_TO_INPUT_FILE&gt; --output counts</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>This runner is not yet available for the Go SDK.</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>This runner is not yet available for the Go SDK.</code></pre>
</div>
@@ -2912,9 +2852,6 @@ frequency count of the words seen in each 15 second window.</p>
--output_topic &#34;projects/YOUR_PUBSUB_PROJECT_NAME/topics/YOUR_OUTPUT_TOPIC&#34; \
--streaming</code></pre>
</div>
-<div class=runner-apex>
-<pre><code>This runner is not yet available for the Python SDK.</code></pre>
-</div>
<div class=runner-flink-local>
<pre><code>This runner is not yet available for the Python SDK.</code></pre>
</div>
diff --git a/website/generated-content/get-started/quickstart-java/index.html b/website/generated-content/get-started/quickstart-java/index.html
index 057f901..5cf589d 100644
--- a/website/generated-content/get-started/quickstart-java/index.html
+++ b/website/generated-content/get-started/quickstart-java/index.html
@@ -60,9 +60,8 @@ d----- 7/19/2018 11:00 PM subprocess
classpath = sourceSets.main.runtimeClasspath
systemProperties System.getProperties()
args System.getProperty("exec.args").split()
-}</code></pre></li><li>Rebuild your project by running:<pre><code>$ gradle build</code></pre></li></ol><h2 id=run-wordcount>Run WordCount</h2><p>A single Beam pipeline can run on multiple Beam <a href=/documentation#runners>runners</a>, including the <a href=/documentation/runners/apex>ApexRunner</a>, <a href=/documentation/runners/flink>FlinkRunner</a>, <a href=/documentation/runners/spark>SparkRunner</a>, <a href=/documentation/runners/nemo>NemoRunner</a>, <a href=/documentation/runner [...]
- -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-apex><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
- -Dexec.args="--inputFile=pom.xml --output=counts --runner=ApexRunner" -Papex-runner</code></pre></div><div class=runner-flink-local><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+}</code></pre></li><li>Rebuild your project by running:<pre><code>$ gradle build</code></pre></li></ol><h2 id=run-wordcount>Run WordCount</h2><p>A single Beam pipeline can run on multiple Beam <a href=/documentation#runners>runners</a>, including the <a href=/documentation/runners/flink>FlinkRunner</a>, <a href=/documentation/runners/spark>SparkRunner</a>, <a href=/documentation/runners/nemo>NemoRunner</a>, <a href=/documentation/runners/jet>JetRunner</a>, or <a href=/documentation/runne [...]
+ -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-flink-local><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--runner=FlinkRunner --inputFile=pom.xml --output=counts" -Pflink-runner</code></pre></div><div class=runner-flink-cluster><pre><code>$ mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--runner=FlinkRunner --flinkMaster=<flink master> --filesToStage=target/word-count-beam-bundled-0.1.jar \
--inputFile=/path/to/quickstart/pom.xml --output=/tmp/counts" -Pflink-runner
@@ -80,8 +79,7 @@ $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
--runner=NemoRunner --inputFile=`pwd`/pom.xml --output=counts</code></pre></div><div class=runner-jet><pre><code>$ mvn package -Pjet-runner
$ java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount \
--runner=JetRunner --jetLocalMode=3 --inputFile=`pwd`/pom.xml --output=counts</code></pre></div><p>For Windows PowerShell:</p><div class=runner-direct><pre><code>PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
- -D exec.args="--inputFile=pom.xml --output=counts" -P direct-runner</code></pre></div><div class=runner-apex><pre><code>PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
- -D exec.args="--inputFile=pom.xml --output=counts --runner=ApexRunner" -P apex-runner</code></pre></div><div class=runner-flink-local><pre><code>PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+ -D exec.args="--inputFile=pom.xml --output=counts" -P direct-runner</code></pre></div><div class=runner-flink-local><pre><code>PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args="--runner=FlinkRunner --inputFile=pom.xml --output=counts" -P flink-runner</code></pre></div><div class=runner-flink-cluster><pre><code>PS> mvn package exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args="--runner=FlinkRunner --flinkMaster=<flink master> --filesToStage=.\target\word-count-beam-bundled-0.1.jar `
--inputFile=C:\path\to\quickstart\pom.xml --output=C:\tmp\counts" -P flink-runner
@@ -103,7 +101,7 @@ PS> java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.
-Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-apex><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-flink-local><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-flink-cluster><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-spark><pre><code>$ gradle clean [...]
-Dexec.args="--inputFile=pom.xml --output=counts" -Pspark-runner</code></pre></div><div class=runner-dataflow><pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--project=<your-gcp-project> --inputFile=gs://apache-beam-samples/shakespeare/* \
- --output=gs://<your-gcs-bucket>/counts" -Pdataflow-runner</code></pre></div><div class=runner-samza-local><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-nemo><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-jet><pre><code>We are working on adding the instruction for this runner!</code></pre></div><h2 id=inspect-the-results>Inspect the results</h2><p>Once the pip [...]
+ --output=gs://<your-gcs-bucket>/counts" -Pdataflow-runner</code></pre></div><div class=runner-samza-local><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-nemo><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-jet><pre><code>We are working on adding the instruction for this runner!</code></pre></div><h2 id=inspect-the-results>Inspect the results</h2><p>Once the pip [...]
api: 9
bundled: 1
old: 4
@@ -111,12 +109,6 @@ Apache: 2
The: 1
limitations: 1
Foundation: 1
-...</code></pre></div><div class=runner-apex><pre><code>$ cat counts*
-BEAM: 1
-have: 1
-simple: 1
-skip: 4
-PAssert: 1
...</code></pre></div><div class=runner-flink-local><pre><code>$ more counts*
The: 1
api: 9
diff --git a/website/generated-content/get-started/quickstart-py/index.html b/website/generated-content/get-started/quickstart-py/index.html
index f26cc7f..70a73c2 100644
--- a/website/generated-content/get-started/quickstart-py/index.html
+++ b/website/generated-content/get-started/quickstart-py/index.html
@@ -9,7 +9,7 @@ administrative privileges.</p><div class=shell-unix><pre><code>pip install --upg
<code>setuptools</code> is installed on your machine. If you do not have <code>setuptools</code>
version 17.1 or newer, run the following command to install it.</p><div class=shell-unix><pre><code>pip install --upgrade setuptools</code></pre></div><div class=shell-PowerShell><pre><code>PS> python -m pip install --upgrade setuptools</code></pre></div><h2 id=get-apache-beam>Get Apache Beam</h2><h3 id=create-and-activate-a-virtual-environment>Create and activate a virtual environment</h3><p>A virtual environment is a directory tree containing its own Python distribution. To create a [...]
Activating it sets some environment variables that point to the virtual
-environment’s directories.</p><p>To activate a virtual environment in Bash, run:</p><div class=shell-unix><pre><code>. /path/to/directory/bin/activate</code></pre></div><div class=shell-PowerShell><pre><code>PS> C:\path\to\directory\Scripts\activate.ps1</code></pre></div><p>That is, execute the <code>activate</code> script under the virtual environment directory you created.</p><p>For instructions using other shells, see the <a href=https://virtualenv.pypa.io/en/stable/userguide [...]
+environment’s directories.</p><p>To activate a virtual environment in Bash, run:</p><div class=shell-unix><pre><code>. /path/to/directory/bin/activate</code></pre></div><div class=shell-PowerShell><pre><code>PS> C:\path\to\directory\Scripts\activate.ps1</code></pre></div><p>That is, execute the <code>activate</code> script under the virtual environment directory you created.</p><p>For instructions using other shells, see the <a href=https://virtualenv.pypa.io/en/stable/userguide [...]
See https://beam.apache.org/roadmap/portability/#python-on-flink for more information.</code></pre></div><div class=runner-flink-cluster><pre><code>Currently, running wordcount.py on Flink requires a full download of the Beam source code.
See https://beam.apache.org/documentation/runners/flink/ for more information.</code></pre></div><div class=runner-spark><pre><code>Currently, running wordcount.py on Spark requires a full download of the Beam source code.
See https://beam.apache.org/roadmap/portability/#python-on-spark for more information.</code></pre></div><div class=runner-dataflow><pre><code># As part of the initial setup, install Google Cloud Platform specific extra components. Make sure you
diff --git a/website/generated-content/get-started/wordcount-example/index.html b/website/generated-content/get-started/wordcount-example/index.html
index e36fa48..bc1fc27 100644
--- a/website/generated-content/get-started/wordcount-example/index.html
+++ b/website/generated-content/get-started/wordcount-example/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Beam WordCount Examples</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domai [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
read text, tokenize the text lines into individual words, and perform a
frequency count on each of those words. The Beam SDKs contain a series of these
four successively more detailed WordCount examples that build on each other. The
@@ -100,8 +100,7 @@ required, they can make your pipeline’s execution more flexible, aid in te
your pipeline, and help make your pipeline’s code reusable.</p><p>This section assumes that you have a good understanding of the basic concepts in
building a pipeline. If you feel that you aren’t at that point yet, read the
above section, <a href=#minimalwordcount-example>MinimalWordCount</a>.</p><p><strong>To run this example in Java:</strong></p><div class=runner-direct><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
- -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-apex><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
- -Dexec.args="--inputFile=pom.xml --output=counts --runner=ApexRunner" -Papex-runner</code></pre></div><div class=runner-flink-local><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+ -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-flink-local><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--runner=FlinkRunner --inputFile=pom.xml --output=counts" -Pflink-runner</code></pre></div><div class=runner-flink-cluster><pre><code>$ mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--runner=FlinkRunner --flinkMaster=<flink master> --filesToStage=target/word-count-beam-bundled-0.1.jar \
--inputFile=/path/to/quickstart/pom.xml --output=/tmp/counts" -Pflink-runner
@@ -115,7 +114,7 @@ You can monitor the running job by visiting the Flink dashboard at http://<fl
-Dexec.args="--inputFile=pom.xml --output=counts --runner=SamzaRunner" -Psamza-runner</code></pre></div><div class=runner-nemo><pre><code>$ mvn package -Pnemo-runner && java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount \
--runner=NemoRunner --inputFile=`pwd`/pom.xml --output=counts</code></pre></div><div class=runner-jet><pre><code>$ mvn package -P jet-runner && java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount \
--runner=JetRunner --jetLocalMode=3 --inputFile=`pwd`/pom.xml --output=counts</code></pre></div><p>To view the full code in Java, see
-<strong><a href=https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/WordCount.java>WordCount</a>.</strong></p><p><strong>To run this example in Python:</strong></p><div class=runner-direct><pre><code>python -m apache_beam.examples.wordcount --input YOUR_INPUT_FILE --output counts</code></pre></div><div class=runner-apex><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-flink-local><pre><code>Cur [...]
+<strong><a href=https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/WordCount.java>WordCount</a>.</strong></p><p><strong>To run this example in Python:</strong></p><div class=runner-direct><pre><code>python -m apache_beam.examples.wordcount --input YOUR_INPUT_FILE --output counts</code></pre></div><div class=runner-flink-local><pre><code>Currently, running wordcount.py on Flink requires a full download of the Beam source code.
See https://beam.apache.org/roadmap/portability/#python-on-flink for more information.</code></pre></div><div class=runner-flink-cluster><pre><code>Currently, running wordcount.py on Flink requires a full download of the Beam source code.
See https://beam.apache.org/documentation/runners/flink/ for more information.</code></pre></div><div class=runner-spark><pre><code>Currently, running wordcount.py on Spark requires a full download of the Beam source code.
See https://beam.apache.org/roadmap/portability/#python-on-spark for more information.</code></pre></div><div class=runner-dataflow><pre><code># As part of the initial setup, install Google Cloud Platform specific extra components.
@@ -127,7 +126,7 @@ python -m apache_beam.examples.wordcount --input gs://dataflow-samples/shakespea
--region YOUR_GCP_REGION \
--temp_location gs://YOUR_GCS_BUCKET/tmp/</code></pre></div><div class=runner-samza-local><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-nemo><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-jet><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><p>To view the full code in Python, see
<strong><a href=https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/wordcount.py>wordcount.py</a>.</strong></p><p><strong>To run this example in Go:</strong></p><div class=runner-direct><pre><code>$ go install github.com/apache/beam/sdks/go/examples/wordcount
-$ wordcount --input <PATH_TO_INPUT_FILE> --output counts</code></pre></div><div class=runner-apex><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-flink-cluster><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-spark><pre><code>This runner is not yet available for the Go SDK.</code></ [...]
+$ wordcount --input <PATH_TO_INPUT_FILE> --output counts</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-flink-cluster><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-spark><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-dataflow><pre><code>$ go install github.com/apache/beam/sdks/go/examples/ [...]
# As part of the initial setup, for non linux users - install package unix before run
$ go get -u golang.org/x/sys/unix
$ wordcount --input gs://dataflow-samples/shakespeare/kinglear.txt \
@@ -261,8 +260,7 @@ values for them. You can then access the options values in your pipeline code.</
<span class=nx>lines</span> <span class=o>:=</span> <span class=nx>textio</span><span class=p>.</span><span class=nf>Read</span><span class=p>(</span><span class=nx>s</span><span class=p>,</span> <span class=o>*</span><span class=nx>input</span><span class=p>)</span>
<span class=o>...</span></code></pre></div></div><h2 id=debuggingwordcount-example>DebuggingWordCount example</h2><p>The DebuggingWordCount example demonstrates some best practices for
instrumenting your pipeline code.</p><p><strong>To run this example in Java:</strong></p><div class=runner-direct><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \
- -Dexec.args="--output=counts" -Pdirect-runner</code></pre></div><div class=runner-apex><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \
- -Dexec.args="--output=counts --runner=ApexRunner" -Papex-runner</code></pre></div><div class=runner-flink-local><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \
+ -Dexec.args="--output=counts" -Pdirect-runner</code></pre></div><div class=runner-flink-local><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \
-Dexec.args="--runner=FlinkRunner --output=counts" -Pflink-runner</code></pre></div><div class=runner-flink-cluster><pre><code>$ mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \
-Dexec.args="--runner=FlinkRunner --flinkMaster=<flink master> --filesToStage=target/word-count-beam-bundled-0.1.jar \
--output=/tmp/counts" -Pflink-runner
@@ -276,7 +274,7 @@ You can monitor the running job by visiting the Flink dashboard at http://<fl
-Dexec.args="--runner=SamzaRunner --output=counts" -Psamza-runner</code></pre></div><div class=runner-nemo><pre><code>$ mvn package -Pnemo-runner && java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.DebuggingWordCount \
--runner=NemoRunner --inputFile=`pwd`/pom.xml --output=counts</code></pre></div><div class=runner-jet><pre><code>$ mvn package -P jet-runner && java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.DebuggingWordCount \
--runner=JetRunner --jetLocalMode=3 --output=counts</code></pre></div><p>To view the full code in Java, see
-<a href=https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/DebuggingWordCount.java>DebuggingWordCount</a>.</p><p><strong>To run this example in Python:</strong></p><div class=runner-direct><pre><code>python -m apache_beam.examples.wordcount_debugging --input YOUR_INPUT_FILE --output counts</code></pre></div><div class=runner-apex><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-flink-local><pr [...]
+<a href=https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/DebuggingWordCount.java>DebuggingWordCount</a>.</p><p><strong>To run this example in Python:</strong></p><div class=runner-direct><pre><code>python -m apache_beam.examples.wordcount_debugging --input YOUR_INPUT_FILE --output counts</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-flink-cl [...]
pip install apache-beam[gcp]
python -m apache_beam.examples.wordcount_debugging --input gs://dataflow-samples/shakespeare/kinglear.txt \
--output gs://YOUR_GCS_BUCKET/counts \
@@ -284,7 +282,7 @@ python -m apache_beam.examples.wordcount_debugging --input gs://dataflow-samples
--project YOUR_GCP_PROJECT \
--temp_location gs://YOUR_GCS_BUCKET/tmp/</code></pre></div><div class=runner-samza-local><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-nemo><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-jet><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><p>To view the full code in Python, see
<strong><a href=https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/wordcount_debugging.py>wordcount_debugging.py</a>.</strong></p><p><strong>To run this example in Go:</strong></p><div class=runner-direct><pre><code>$ go install github.com/apache/beam/sdks/go/examples/debugging_wordcount
-$ debugging_wordcount --input <PATH_TO_INPUT_FILE> --output counts</code></pre></div><div class=runner-apex><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-flink-cluster><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-spark><pre><code>This runner is not yet available for the Go SDK [...]
+$ debugging_wordcount --input <PATH_TO_INPUT_FILE> --output counts</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-flink-cluster><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-spark><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-dataflow><pre><code>$ go install github.com/apache/beam/sdks/go [...]
# As part of the initial setup, for non linux users - install package unix before run
$ go get -u golang.org/x/sys/unix
$ debugging_wordcount --input gs://dataflow-samples/shakespeare/kinglear.txt \
@@ -381,8 +379,7 @@ pipeline with the Cloud Dataflow service, Cloud Logging will contain all “
or higher level logs. Note that changing the default worker log level to TRACE
or DEBUG significantly increases the amount of logs output.</p><h4 id=apache-spark-runner>Apache Spark Runner</h4><blockquote><p><strong>Note:</strong> This section is yet to be added. There is an open issue for this
(<a href=https://issues.apache.org/jira/browse/BEAM-792>BEAM-792</a>).</p></blockquote><h4 id=apache-flink-runner>Apache Flink Runner</h4><blockquote><p><strong>Note:</strong> This section is yet to be added. There is an open issue for this
-(<a href=https://issues.apache.org/jira/browse/BEAM-791>BEAM-791</a>).</p></blockquote><h4 id=apache-apex-runner>Apache Apex Runner</h4><blockquote><p><strong>Note:</strong> This section is yet to be added. There is an open issue for this
-(<a href=https://issues.apache.org/jira/browse/BEAM-2285>BEAM-2285</a>).</p></blockquote><h4 id=apache-nemo-runner>Apache Nemo Runner</h4><p>When executing your pipeline with the <code>NemoRunner</code>, most log messages are printed
+(<a href=https://issues.apache.org/jira/browse/BEAM-791>BEAM-791</a>).</p></blockquote><h4 id=apache-nemo-runner>Apache Nemo Runner</h4><p>When executing your pipeline with the <code>NemoRunner</code>, most log messages are printed
directly to your local console. You should add <code>Slf4j</code> to your class path to make
full use of the logs. In order to observe the logs on each of the driver and the
executor sides, you should observe the folders created by Apache REEF. For example,
@@ -413,8 +410,7 @@ expectations are met.</p><div class=language-java><div class=highlight><pre clas
for an example unit test.</p><h2 id=windowedwordcount-example>WindowedWordCount example</h2><p>The WindowedWordCount example counts words in text just as the previous
examples did, but introduces several advanced concepts.</p><p><strong>New Concepts:</strong></p><ul><li>Unbounded and bounded datasets</li><li>Adding timestamps to data</li><li>Windowing</li><li>Reusing PTransforms over windowed PCollections</li></ul><p>The following sections explain these key concepts in detail, and break down the
pipeline code into smaller sections.</p><p><strong>To run this example in Java:</strong></p><div class=runner-direct><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WindowedWordCount \
- -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-apex><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WindowedWordCount \
- -Dexec.args="--inputFile=pom.xml --output=counts --runner=ApexRunner" -Papex-runner</code></pre></div><div class=runner-flink-local><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WindowedWordCount \
+ -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-flink-local><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WindowedWordCount \
-Dexec.args="--runner=FlinkRunner --inputFile=pom.xml --output=counts" -Pflink-runner</code></pre></div><div class=runner-flink-cluster><pre><code>$ mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WindowedWordCount \
-Dexec.args="--runner=FlinkRunner --flinkMaster=<flink master> --filesToStage=target/word-count-beam-bundled-0.1.jar \
--inputFile=/path/to/quickstart/pom.xml --output=/tmp/counts" -Pflink-runner
@@ -430,7 +426,7 @@ You can monitor the running job by visiting the Flink dashboard at http://<fl
--runner=JetRunner --jetLocalMode=3 --inputFile=`pwd`/pom.xml --output=counts</code></pre></div><p>To view the full code in Java, see
<strong><a href=https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/WindowedWordCount.java>WindowedWordCount</a>.</strong></p><p><strong>To run this example in Python:</strong></p><p>This pipeline writes its results to a BigQuery table <code>--output_table</code>
parameter. using the format <code>PROJECT:DATASET.TABLE</code> or
-<code>DATASET.TABLE</code>.</p><div class=runner-direct><pre><code>python -m apache_beam.examples.windowed_wordcount --input YOUR_INPUT_FILE --output_table PROJECT:DATASET.TABLE</code></pre></div><div class=runner-apex><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-flink-cluster><pre><code>This runner is not yet available for [...]
+<code>DATASET.TABLE</code>.</p><div class=runner-direct><pre><code>python -m apache_beam.examples.windowed_wordcount --input YOUR_INPUT_FILE --output_table PROJECT:DATASET.TABLE</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-flink-cluster><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-spark><pre><code>This runner is not yet available fo [...]
pip install apache-beam[gcp]
python -m apache_beam.examples.windowed_wordcount --input YOUR_INPUT_FILE \
--output_table PROJECT:DATASET.TABLE \
@@ -438,7 +434,7 @@ python -m apache_beam.examples.windowed_wordcount --input YOUR_INPUT_FILE \
--project YOUR_GCP_PROJECT \
--temp_location gs://YOUR_GCS_BUCKET/tmp/</code></pre></div><div class=runner-samza-local><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-nemo><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-jet><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><p>To view the full code in Python, see
<strong><a href=https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/windowed_wordcount.py>windowed_wordcount.py</a>.</strong></p><p><strong>To run this example in Go:</strong></p><div class=runner-direct><pre><code>$ go install github.com/apache/beam/sdks/go/examples/windowed_wordcount
-$ windowed_wordcount --input <PATH_TO_INPUT_FILE> --output counts</code></pre></div><div class=runner-apex><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-flink-cluster><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-spark><pre><code>This runner is not yet available for the Go SDK. [...]
+$ windowed_wordcount --input <PATH_TO_INPUT_FILE> --output counts</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-flink-cluster><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-spark><pre><code>This runner is not yet available for the Go SDK.</code></pre></div><div class=runner-dataflow><pre><code>$ go install github.com/apache/beam/sdks/go/ [...]
# As part of the initial setup, for non linux users - install package unix before run
$ go get -u golang.org/x/sys/unix
$ windowed_wordcount --input gs://dataflow-samples/shakespeare/kinglear.txt \
@@ -552,7 +548,7 @@ fixed window size for this example is 15 seconds. The pipeline outputs the
frequency count of the words seen in each 15 second window.</p><p><strong>New Concepts:</strong></p><ul><li>Reading an unbounded dataset</li><li>Writing unbounded results</li></ul><p><strong>To run this example in Java:</strong></p><blockquote><p><strong>Note:</strong> StreamingWordCount is not yet available for the Java SDK.</p></blockquote><p><strong>To run this example in Python:</strong></p><div class=runner-direct><pre><code>python -m apache_beam.examples.streaming_wordcount \
--input_topic "projects/YOUR_PUBSUB_PROJECT_NAME/topics/YOUR_INPUT_TOPIC" \
--output_topic "projects/YOUR_PUBSUB_PROJECT_NAME/topics/YOUR_OUTPUT_TOPIC" \
- --streaming</code></pre></div><div class=runner-apex><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-flink-cluster><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-spark><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-dataf [...]
+ --streaming</code></pre></div><div class=runner-flink-local><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-flink-cluster><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-spark><pre><code>This runner is not yet available for the Python SDK.</code></pre></div><div class=runner-dataflow><pre><code># As part of the initial setup, install Google Cloud Platform specific extra components.
pip install apache-beam[gcp]
python -m apache_beam.examples.streaming_wordcount \
--runner DataflowRunner \
diff --git a/website/generated-content/index.html b/website/generated-content/index.html
index b066ea5..8a3cff9 100644
--- a/website/generated-content/index.html
+++ b/website/generated-content/index.html
@@ -5,7 +5,7 @@
<a class="button button--primary" href=/get-started/try-apache-beam/>Try Beam</a>
<a class="button button--primary" href=/get-started/downloads/>Download Beam SDK 2.22.0</a></div><div class=hero__ctas><a class=button href=/get-started/quickstart-java/>Java Quickstart</a>
<a class=button href=/get-started/quickstart-py/>Python Quickstart</a>
-<a class=button href=/get-started/quickstart-go/>Go Quickstart</a></div></div></div><div class=hero__cols__col><div class=hero__blog><div class=hero__blog__title>The latest from the blog</div><div class=hero__blog__cards><a class=hero__blog__cards__card href=/blog/beam-2.22.0/><div class=hero__blog__cards__card__title>Apache Beam 2.22.0</div><div class=hero__blog__cards__card__date>Jun 8, 2020</div></a><a class=hero__blog__cards__card href=/blog/beam-katas-kotlin-release/><div class=hero [...]
+<a class=button href=/get-started/quickstart-go/>Go Quickstart</a></div></div></div><div class=hero__cols__col><div class=hero__blog><div class=hero__blog__title>The latest from the blog</div><div class=hero__blog__cards><a class=hero__blog__cards__card href=/blog/beam-2.22.0/><div class=hero__blog__cards__card__title>Apache Beam 2.22.0</div><div class=hero__blog__cards__card__date>Jun 8, 2020</div></a><a class=hero__blog__cards__card href=/blog/beam-katas-kotlin-release/><div class=hero [...]
<a class="button button--primary" href=/get-started/downloads/>Download Beam SDK 2.22.0</a></div><div class=ctas__ctas><a class=button href=/get-started/quickstart-java/>Java Quickstart</a>
<a class=button href=/get-started/quickstart-py/>Python Quickstart</a>
<a class=button href=/get-started/quickstart-go/>Go Quickstart</a></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=footer__cols__col><div class=footer__cols__col__logo><img src=/images/beam_logo_circle.svg class=footer__logo alt="Beam logo"></div><div class=footer__cols__col__logo><img src=/images/apache_logo_circle.svg class=footer__logo alt="Apache logo"></div></div><div class="footer__cols__col footer__cols__col--md"><div class=foo [...]
diff --git a/website/generated-content/roadmap/apex-runner/index.html b/website/generated-content/roadmap/apex-runner/index.html
deleted file mode 100644
index a469917..0000000
--- a/website/generated-content/roadmap/apex-runner/index.html
+++ /dev/null
@@ -1,7 +0,0 @@
-<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Samza Runner Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain S [...]
-<span class=sr-only>Toggle navigation</span>
-<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
-<a href=http://www.apache.org>The Apache Software Foundation</a>
-| <a href=/privacy_policy>Privacy Policy</a>
-| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/roadmap/connectors-go-sdk/index.html b/website/generated-content/roadmap/connectors-go-sdk/index.html
index f99d5ca..028128b 100644
--- a/website/generated-content/roadmap/connectors-go-sdk/index.html
+++ b/website/generated-content/roadmap/connectors-go-sdk/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Connectors - Go SDK</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Sp [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
meantime bounded sources can be developed in the form of ParDos. See
<a href=https://beam.apache.org/documentation/io/authoring-overview/>Authoring I/O Transforms</a> for more details.</p></li><li><p>Go SDK plans to utilize currently available Java and Python connectors
through cross-language transforms feature.</p></li></ul></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=footer__cols__col><div class=footer__cols__col__logo><img src=/images/beam_logo_circle.svg class=footer__logo alt="Beam logo"></div><div class=footer__cols__col__logo><img src=/images/apache_logo_circle.svg class=footer__logo alt="Apache logo"></div></div><div class="footer__cols__col footer__cols__col--md"><div class=footer__cols__col__ [...]
diff --git a/website/generated-content/roadmap/connectors-java-sdk/index.html b/website/generated-content/roadmap/connectors-java-sdk/index.html
index 74f73d4..a2068f1 100644
--- a/website/generated-content/roadmap/connectors-java-sdk/index.html
+++ b/website/generated-content/roadmap/connectors-java-sdk/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Connectors - Java SDK</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<a href=https://issues.apache.org/jira/browse/BEAM-1893>BEAM-1893</a> for more details on the
planned Beam connector for Couchbase.</p><h1 id=influxdb>InfluxDB</h1><p>InfluxDB is a database for fast and highly available storage and retrieval
of time series data. See <a href=https://issues.apache.org/jira/browse/BEAM-2546>BEAM-2546</a> for
diff --git a/website/generated-content/roadmap/connectors-multi-sdk/index.html b/website/generated-content/roadmap/connectors-multi-sdk/index.html
index 6aebc21..7042e76 100644
--- a/website/generated-content/roadmap/connectors-multi-sdk/index.html
+++ b/website/generated-content/roadmap/connectors-multi-sdk/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Multi-SDK Connector Efforts</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and D [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
replace current frameworks for developing bounded and unbounded sources.
Splittable DoFn is being developed along side current Beam portability
efforts. See <a href=https://beam.apache.org/roadmap/portability/>Beam portability framework roadmap</a> for more details.</p><h1 id=cross-language-transforms>Cross-language transforms</h1><p><em>Last updated on May 2020.</em></p><p>As an added benefit of Beam portability effort, we are able to utilize Beam transforms across SDKs. This has many benefits.</p><ul><li>Connector sharing across SDKs. For example,<ul><li>Beam pipelines written using Python and Go SDKs will be able to utilize t [...]
diff --git a/website/generated-content/roadmap/connectors-python-sdk/index.html b/website/generated-content/roadmap/connectors-python-sdk/index.html
index ba08f4c..8d6632e 100644
--- a/website/generated-content/roadmap/connectors-python-sdk/index.html
+++ b/website/generated-content/roadmap/connectors-python-sdk/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Connectors - Python SDK</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domai [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
Splittable DoFn API is planned. This is partially blocked till
Splittable DoFn work related to portability framework is finalized.
See <a href=https://issues.apache.org/jira/browse/BEAM-3788>BEAM-3788</a> for more details.</p><h1 id=parquet>Parquet</h1><p>A Python connector for Parquet file format is currently in development.
diff --git a/website/generated-content/roadmap/dataflow-runner/index.html b/website/generated-content/roadmap/dataflow-runner/index.html
index 4543d97..ac286ef 100644
--- a/website/generated-content/roadmap/dataflow-runner/index.html
+++ b/website/generated-content/roadmap/dataflow-runner/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Flink Runner Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain S [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/roadmap/euphoria/index.html b/website/generated-content/roadmap/euphoria/index.html
index 7a69a26..d9f21b9 100644
--- a/website/generated-content/roadmap/euphoria/index.html
+++ b/website/generated-content/roadmap/euphoria/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Euphoria API Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain S [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
the large keys into multiple splits, using key distribution approximated by count min sketch data structure.</p><h2 id=pipeline-sampling>Pipeline sampling</h2><p>In order to pick the right translation for the operator without user interference, we can leverage knowledge from
previous pipeline runs. We want to provide a convenient and portable way to gather this knowledge.</p><h2 id=fluent-api>Fluent API</h2><p>Implementation of an easy to use Fluent API on top of Euphoria DSL.</p><h2 id=side-outputs>Side Outputs</h2><p>An convenient API for multiple outputs.</p><h2 id=table-stream-joins>Table-stream joins</h2><p>Introduce API for converting streams to tables (KStream <-> KTable approach) and various types of (windowed and unwindowed) joins on them.</p></d [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
diff --git a/website/generated-content/roadmap/flink-runner/index.html b/website/generated-content/roadmap/flink-runner/index.html
index 67a9118..2e3360a 100644
--- a/website/generated-content/roadmap/flink-runner/index.html
+++ b/website/generated-content/roadmap/flink-runner/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Flink Runner Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain S [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/roadmap/go-sdk/index.html b/website/generated-content/roadmap/go-sdk/index.html
index 91dc8f4..2763704 100644
--- a/website/generated-content/roadmap/go-sdk/index.html
+++ b/website/generated-content/roadmap/go-sdk/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Go SDK Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specifi [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
by the status of the <a href=https://beam.apache.org/roadmap/portability/>Beam Portability Framework</a> and the existence of
portable runners.</p><p><strong>April 2020 Update</strong>
This year we hope to move the SDK out of experimental at least for Batch usage.</p><p>To do so, there are a few blocking changes:</p><ul><li>Support Interoperability with other SDKs</li><li>Scalable IOs</li><li>Portability Proto stability</li><li>Go Ecosystem integration improvements</li></ul><p>Interoperability while not a blocking feature in itself, requires Beam Schema support, which
diff --git a/website/generated-content/roadmap/index.html b/website/generated-content/roadmap/index.html
index 2137447..abcb858 100644
--- a/website/generated-content/roadmap/index.html
+++ b/website/generated-content/roadmap/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Apache Beam Roadmap Highlights</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) an [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
Project Management Committee (PMC), so we do not have a roadmap in the sense of
a plan with a specific timeline.
Instead, we share our vision for the future and major initiatives receiving
diff --git a/website/generated-content/roadmap/index.xml b/website/generated-content/roadmap/index.xml
index 7b1f80e..78b5af3 100644
--- a/website/generated-content/roadmap/index.xml
+++ b/website/generated-content/roadmap/index.xml
@@ -514,24 +514,7 @@ limitations under the License.
<p>Contributions and feedback are welcome!</p>
<p>If you are interested in helping, you can select an unassigned issue on the Kanban board and assign it to yourself. If you cannot assign the issue to yourself, comment on the issue. When submitting a new PR, please tag <a href="https://github.com/aaltay">@aaltay</a>, and <a href="https://github.com/tvalentyn">@tvalentyn</a>.</p>
<p>To report a Python 3 related issue, create a subtask in <a href="https://issues.apache.org/jira/browse/BEAM-1251">BEAM-1251</a> and cc: [~altay] and [~tvalentyn] in a JIRA comment. The best way to help us identify and investigate the issue is with a minimal pipeline that reproduces the issue.</p>
-<p>You can also discuss encountered issues on user@ or dev@ mailing lists as appropriate.</p></description></item><item><title>Roadmap: Samza Runner Roadmap</title><link>/roadmap/apex-runner/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>/roadmap/apex-runner/</guid><description>
-<!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-http://www.apache.org/licenses/LICENSE-2.0
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-<h1 id="apache-apex-runner-roadmap">Apache Apex Runner Roadmap</h1>
-<p>This roadmap is in progress. In the meantime, here are available resources:</p>
-<ul>
-<li><a href="/documentation/runners/apex">Runner documentation</a></li>
-<li>JIRA: <a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20component%20%3D%20runner-apex">runner-apex</a></li>
-</ul></description></item><item><title>Roadmap: Samza Runner Roadmap</title><link>/roadmap/samza-runner/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>/roadmap/samza-runner/</guid><description>
+<p>You can also discuss encountered issues on user@ or dev@ mailing lists as appropriate.</p></description></item><item><title>Roadmap: Samza Runner Roadmap</title><link>/roadmap/samza-runner/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>/roadmap/samza-runner/</guid><description>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
diff --git a/website/generated-content/roadmap/java-sdk/index.html b/website/generated-content/roadmap/java-sdk/index.html
index 5a0fff9..428f725 100644
--- a/website/generated-content/roadmap/java-sdk/index.html
+++ b/website/generated-content/roadmap/java-sdk/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Java SDK Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Speci [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
about the scope and info on the various tasks please see the JIRA ticket.</p><ul><li>JIRA: <a href=https://issues.apache.org/jira/browse/BEAM-2530>BEAM-2530</a></li><li>Contact: <a href=mailto:mwalenia@apache.org>Michał Walenia</a></li></ul></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=footer__cols__col><div class=footer__cols__col__logo><img src=/images/beam_logo_circle.svg class=footer__logo alt="Beam logo"></div><div class=footer__col [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
diff --git a/website/generated-content/roadmap/nemo-runner/index.html b/website/generated-content/roadmap/nemo-runner/index.html
index 16820b1..fed3663 100644
--- a/website/generated-content/roadmap/nemo-runner/index.html
+++ b/website/generated-content/roadmap/nemo-runner/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Nemo Runner Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Sp [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/roadmap/portability/index.html b/website/generated-content/roadmap/portability/index.html
index f30eecb..ec8da91 100644
--- a/website/generated-content/roadmap/portability/index.html
+++ b/website/generated-content/roadmap/portability/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Portability Framework Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
Beam. So far, however, the reality is that most runners support the
Java SDK only, because each SDK-runner combination requires non-trivial
work on both sides. All runners are also currently written in Java,
diff --git a/website/generated-content/roadmap/python-sdk/index.html b/website/generated-content/roadmap/python-sdk/index.html
index fbac1cd..bcbb903 100644
--- a/website/generated-content/roadmap/python-sdk/index.html
+++ b/website/generated-content/roadmap/python-sdk/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Python SDK Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Spe [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/roadmap/samza-runner/index.html b/website/generated-content/roadmap/samza-runner/index.html
index 0217eff..dc2f971 100644
--- a/website/generated-content/roadmap/samza-runner/index.html
+++ b/website/generated-content/roadmap/samza-runner/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Samza Runner Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain S [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/roadmap/spark-runner/index.html b/website/generated-content/roadmap/spark-runner/index.html
index 1f05671..5dc1fe8 100644
--- a/website/generated-content/roadmap/spark-runner/index.html
+++ b/website/generated-content/roadmap/spark-runner/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Spark Runner Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain S [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
tasks please refer to the JIRA ticket.</p><ul><li>JIRA: <a href=https://issues.apache.org/jira/browse/BEAM-7093>BEAM-7093</a></li><li>Contact: <a href=mailto:iemejia@apache.org>Ismaël Mejía</a></li></ul></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=footer__cols__col><div class=footer__cols__col__logo><img src=/images/beam_logo_circle.svg class=footer__logo alt="Beam logo"></div><div class=footer__cols__col__logo><img src=/images/apache_l [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
diff --git a/website/generated-content/roadmap/sql/index.html b/website/generated-content/roadmap/sql/index.html
index 103d497..8e0d706 100644
--- a/website/generated-content/roadmap/sql/index.html
+++ b/website/generated-content/roadmap/sql/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Beam SQL Roadmap</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Speci [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index e64d435..e8f7963 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.22.0/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/b [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.22.0/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/b [...]
\ No newline at end of file