You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2019/10/07 19:59:30 UTC
[beam] branch asf-site updated: Publishing website 2019/10/07
19:59:17 at commit 1df2e6c
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 41e7717 Publishing website 2019/10/07 19:59:17 at commit 1df2e6c
41e7717 is described below
commit 41e77173f56823b262eb55faefc8bb0070b5ef33
Author: jenkins <bu...@apache.org>
AuthorDate: Mon Oct 7 19:59:17 2019 +0000
Publishing website 2019/10/07 19:59:17 at commit 1df2e6c
---
.../generated-content/blog/2019/09/04/gsoc-19.html | 2 +-
.../documentation/execution-model/index.html | 748 +--------------------
website/generated-content/documentation/index.html | 12 +-
.../io/built-in/google-bigquery/index.html | 10 +-
.../documentation/io/built-in/hadoop/index.html | 10 +-
.../documentation/io/built-in/hcatalog/index.html | 10 +-
.../documentation/io/built-in/index.html | 10 +-
.../documentation/io/built-in/parquet/index.html | 10 +-
.../documentation/io/developing-io-java/index.html | 10 +-
.../io/developing-io-overview/index.html | 10 +-
.../io/developing-io-python/index.html | 10 +-
.../documentation/io/testing/index.html | 10 +-
.../documentation/patterns/custom-io/index.html | 10 +-
.../patterns/custom-windows/index.html | 10 +-
.../patterns/file-processing/index.html | 10 +-
.../documentation/patterns/overview/index.html | 10 +-
.../patterns/pipeline-options/index.html | 10 +-
.../documentation/patterns/side-inputs/index.html | 10 +-
.../pipelines/create-your-pipeline/index.html | 10 +-
.../pipelines/design-your-pipeline/index.html | 10 +-
.../pipelines/test-your-pipeline/index.html | 10 +-
.../documentation/programming-guide/index.html | 10 +-
.../resources/learning-resources/index.html | 12 +-
.../resources/videos-and-podcasts/index.html | 10 +-
.../documentation/runners/jstorm/index.html | 10 +-
.../environments}/index.html | 332 ++++-----
.../{execution-model => runtime/model}/index.html | 18 +-
.../aggregation/approximatequantiles/index.html | 10 +-
.../java/aggregation/approximateunique/index.html | 10 +-
.../java/aggregation/cogroupbykey/index.html | 10 +-
.../transforms/java/aggregation/combine/index.html | 10 +-
.../java/aggregation/combinewithcontext/index.html | 10 +-
.../transforms/java/aggregation/count/index.html | 10 +-
.../java/aggregation/distinct/index.html | 10 +-
.../java/aggregation/groupbykey/index.html | 10 +-
.../java/aggregation/groupintobatches/index.html | 10 +-
.../transforms/java/aggregation/latest/index.html | 10 +-
.../transforms/java/aggregation/max/index.html | 10 +-
.../transforms/java/aggregation/mean/index.html | 10 +-
.../transforms/java/aggregation/min/index.html | 10 +-
.../transforms/java/aggregation/sample/index.html | 10 +-
.../transforms/java/aggregation/sum/index.html | 10 +-
.../transforms/java/aggregation/top/index.html | 10 +-
.../transforms/java/elementwise/filter/index.html | 10 +-
.../java/elementwise/flatmapelements/index.html | 10 +-
.../transforms/java/elementwise/keys/index.html | 10 +-
.../transforms/java/elementwise/kvswap/index.html | 10 +-
.../java/elementwise/mapelements/index.html | 10 +-
.../transforms/java/elementwise/pardo/index.html | 10 +-
.../java/elementwise/partition/index.html | 10 +-
.../transforms/java/elementwise/regex/index.html | 10 +-
.../transforms/java/elementwise/reify/index.html | 10 +-
.../java/elementwise/tostring/index.html | 10 +-
.../transforms/java/elementwise/values/index.html | 10 +-
.../java/elementwise/withkeys/index.html | 10 +-
.../java/elementwise/withtimestamps/index.html | 10 +-
.../transforms/java/other/create/index.html | 10 +-
.../transforms/java/other/flatten/index.html | 10 +-
.../transforms/java/other/passert/index.html | 10 +-
.../transforms/java/other/view/index.html | 10 +-
.../transforms/java/other/window/index.html | 10 +-
.../transforms/java/overview/index.html | 10 +-
.../aggregation/approximatequantiles/index.html | 10 +-
.../aggregation/approximateunique/index.html | 10 +-
.../python/aggregation/cogroupbykey/index.html | 10 +-
.../python/aggregation/combineglobally/index.html | 10 +-
.../aggregation/combinewithcontext/index.html | 10 +-
.../transforms/python/aggregation/count/index.html | 10 +-
.../python/aggregation/distinct/index.html | 10 +-
.../python/aggregation/groupbykey/index.html | 10 +-
.../python/aggregation/groupintobatches/index.html | 10 +-
.../python/aggregation/latest/index.html | 10 +-
.../transforms/python/aggregation/max/index.html | 10 +-
.../transforms/python/aggregation/mean/index.html | 10 +-
.../transforms/python/aggregation/min/index.html | 10 +-
.../python/aggregation/sample/index.html | 10 +-
.../transforms/python/aggregation/sum/index.html | 10 +-
.../transforms/python/aggregation/top/index.html | 10 +-
.../python/elementwise/filter/index.html | 10 +-
.../python/elementwise/flatmap/index.html | 10 +-
.../transforms/python/elementwise/keys/index.html | 10 +-
.../python/elementwise/kvswap/index.html | 10 +-
.../transforms/python/elementwise/map/index.html | 10 +-
.../transforms/python/elementwise/pardo/index.html | 12 +-
.../python/elementwise/partition/index.html | 10 +-
.../transforms/python/elementwise/regex/index.html | 10 +-
.../transforms/python/elementwise/reify/index.html | 10 +-
.../python/elementwise/tostring/index.html | 10 +-
.../python/elementwise/values/index.html | 10 +-
.../python/elementwise/withkeys/index.html | 10 +-
.../python/elementwise/withtimestamps/index.html | 10 +-
.../transforms/python/other/create/index.html | 10 +-
.../transforms/python/other/flatten/index.html | 10 +-
.../transforms/python/other/passert/index.html | 10 +-
.../transforms/python/other/reshuffle/index.html | 10 +-
.../transforms/python/other/view/index.html | 10 +-
.../transforms/python/other/windowinto/index.html | 10 +-
.../transforms/python/overview/index.html | 10 +-
website/generated-content/feed.xml | 2 +-
99 files changed, 1039 insertions(+), 1009 deletions(-)
diff --git a/website/generated-content/blog/2019/09/04/gsoc-19.html b/website/generated-content/blog/2019/09/04/gsoc-19.html
index f651080..5d40fbd 100644
--- a/website/generated-content/blog/2019/09/04/gsoc-19.html
+++ b/website/generated-content/blog/2019/09/04/gsoc-19.html
@@ -213,7 +213,7 @@ I wanted to explore Data Engineering, so for GSoC, I wanted to work on a project
Before actually submitting a proposal, I went through a bunch of resources to make sure I had a concrete understanding of Beam.
I read the <a href="https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101">Streaming 101</a> and <a href="https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102">Streaming 102</a> blogs by Tyler Akidau. They are the perfect introduction to Beam’s unified model for Batch and Streaming.
In addition, I watched all Beam talks on YouTube. You can find them on the <a href="https://beam.apache.org/documentation/resources/videos-and-podcasts/">Beam Website</a>.
-Beam has really good documentation. The <a href="https://beam.apache.org/documentation/programming-guide/">Programming Guide</a> lays out all of Beam’s concepts really well. <a href="https://beam.apache.org/documentation/execution-model/">Beam’s execution model</a> is also documented well and is a must-read to understand how Beam processes data.
+Beam has really good documentation. The <a href="https://beam.apache.org/documentation/programming-guide/">Programming Guide</a> lays out all of Beam’s concepts really well. <a href="https://beam.apache.org/documentation/runtime/model">Beam’s execution model</a> is also documented well and is a must-read to understand how Beam processes data.
<a href="https://www.waitingforcode.com/apache-beam">waitingforcode.com</a> also has good blog posts about Beam concepts.
To get a better sense of the Beam codebase, I played around with it and worked on some PRs to understand Beam better and got familiar with the test suite and workflows.</p>
diff --git a/website/generated-content/documentation/execution-model/index.html b/website/generated-content/documentation/execution-model/index.html
index 511fb90..84fcbdc 100644
--- a/website/generated-content/documentation/execution-model/index.html
+++ b/website/generated-content/documentation/execution-model/index.html
@@ -1,742 +1,10 @@
<!DOCTYPE html>
-<!--
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-<html lang="en">
- <!--
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-<head>
- <meta charset="utf-8">
- <meta http-equiv="X-UA-Compatible" content="IE=edge">
- <meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Beam Execution Model</title>
- <meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
-">
- <link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
- <link rel="stylesheet" href="/css/site.css">
- <script src="https://code.jquery.com/jquery-2.2.4.min.js"></script>
- <script src="/js/bootstrap.min.js"></script>
- <script src="/js/language-switch.js"></script>
- <script src="/js/fix-menu.js"></script>
- <script src="/js/section-nav.js"></script>
- <script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/execution-model/" data-proofer-ignore>
- <link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
- <link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
- <link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
- <script>
- (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
- (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
- m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
- })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
- ga('create', 'UA-73650088-1', 'auto');
- ga('send', 'pageview');
- </script>
-</head>
-
- <body class="body" data-spy="scroll" data-target=".page-nav" data-offset="0">
- <!--
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-<nav class="header navbar navbar-fixed-top">
- <div class="navbar-header">
- <button type="button" class="navbar-toggle" aria-expanded="false" aria-controls="navbar">
- <span class="sr-only">Toggle navigation</span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
-
- <a href="/" class="navbar-brand" >
- <img alt="Brand" style="height: 25px" src="/images/beam_logo_navbar.png">
- </a>
- </div>
-
- <div class="navbar-mask closed"></div>
-
- <div id="navbar" class="navbar-container closed">
- <ul class="nav navbar-nav">
- <li>
- <a href="/get-started/beam-overview/">Get Started</a>
- </li>
- <li>
- <a href="/documentation/">Documentation</a>
- </li>
- <li>
- <a href="/documentation/sdks/java/">Languages</a>
- </li>
- <li>
- <a href="/documentation/runners/capability-matrix/">RUNNERS</a>
- </li>
- <li>
- <a href="/roadmap/">Roadmap</a>
- </li>
- <li>
- <a href="/contribute/">Contribute</a>
- </li>
- <li>
- <a href="/community/contact-us/">Community</a>
- </li>
- <li><a href="/blog">Blog</a></li>
- </ul>
- <ul class="nav navbar-nav navbar-right">
- <li>
- <div style="width: 300px;">
- <script>
- (function() {
- var cx = '012923275103528129024:4emlchv9wzi';
- var gcse = document.createElement('script');
- gcse.type = 'text/javascript';
- gcse.async = true;
- gcse.src = 'https://cse.google.com/cse.js?cx=' + cx;
- var s = document.getElementsByTagName('script')[0];
- s.parentNode.insertBefore(gcse, s);
- })();
- </script>
- <gcse:search></gcse:search>
- </div>
- </li>
- <li class="dropdown">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false"><img src="https://www.apache.org/foundation/press/kit/feather_small.png" alt="Apache Logo" style="height:20px;"><span class="caret"></span></a>
- <ul class="dropdown-menu dropdown-menu-right">
- <li><a href="http://www.apache.org/">ASF Homepage</a></li>
- <li><a href="http://www.apache.org/licenses/">License</a></li>
- <li><a href="http://www.apache.org/security/">Security</a></li>
- <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
- <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
- <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct</a></li>
- </ul>
- </li>
- <li>
- <!--
- data-proofer-ignore disables link checking from website test automation.
- GitHub links will not resolve until the markdown source is available on the master branch.
- New pages would fail validation during development / PR test automation.
- -->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/execution-model.md" data-proofer-ignore>
- <i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
- </a>
- </li>
- </ul>
- </div>
-</nav>
-
- <div class="clearfix container-main-content">
- <div class="section-nav closed" data-offset-top="90" data-offset-bottom="500">
- <span class="section-nav-back glyphicon glyphicon-menu-left"></span>
- <nav>
- <ul class="section-nav-list" data-section-nav>
- <!--
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-<li><span class="section-nav-list-main-title">Documentation</span></li>
-<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
-<li>
- <span class="section-nav-list-title">Pipeline development lifecycle</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a></li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a></li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a></li>
- </ul>
-</li>
-<li>
- <span class="section-nav-list-title">Beam programming guide</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/">Overview</a></li>
- <li><a href="/documentation/programming-guide/#creating-a-pipeline">Pipelines</a></li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">PCollections</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#pcollections">Creating a PCollection</a></li>
- <li><a href="/documentation/programming-guide/#pcollection-characteristics">PCollection characteristics</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Transforms</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#applying-transforms">Applying transforms</a></li>
- <li>
- <span class="section-nav-list-title">Core Beam transforms</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#pardo">ParDo</a></li>
- <li><a href="/documentation/programming-guide/#groupbykey">GroupByKey</a></li>
- <li><a href="/documentation/programming-guide/#cogroupbykey">CoGroupByKey</a></li>
- <li><a href="/documentation/programming-guide/#combine">Combine</a></li>
- <li><a href="/documentation/programming-guide/#flatten">Flatten</a></li>
- <li><a href="/documentation/programming-guide/#partition">Partition</a></li>
- </ul>
- </li>
-
- <li><a href="/documentation/programming-guide/#requirements-for-writing-user-code-for-beam-transforms">Requirements for user code</a></li>
- <li><a href="/documentation/programming-guide/#side-inputs">Side inputs</a></li>
- <li><a href="/documentation/programming-guide/#additional-outputs">Additional outputs</a></li>
- <li><a href="/documentation/programming-guide/#composite-transforms">Composite transforms</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Pipeline I/O</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#pipeline-io">Using I/O transforms</a></li>
- <li><a href="/documentation/io/built-in/">Built-in I/O connectors</a></li>
-
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Developing new I/O connectors</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/io/developing-io-overview/">Overview: Developing connectors</a></li>
- <li><a href="/documentation/io/developing-io-java/">Developing connectors (Java)</a></li>
- <li><a href="/documentation/io/developing-io-python/">Developing connectors (Python)</a></li>
- </ul>
- </li>
-
- <li><a href="/documentation/io/testing/">Testing I/O transforms</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Data encoding and type safety</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#data-encoding-and-type-safety">Data encoding basics</a></li>
- <li><a href="/documentation/programming-guide/#specifying-coders">Specifying coders</a></li>
- <li><a href="/documentation/programming-guide/#default-coders-and-the-coderregistry">Default coders and the CoderRegistry</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Windowing</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#windowing">Windowing basics</a></li>
- <li><a href="/documentation/programming-guide/#provided-windowing-functions">Provided windowing functions</a></li>
- <li><a href="/documentation/programming-guide/#setting-your-pcollections-windowing-function">Setting your PCollection’s windowing function</a></li>
- <li><a href="/documentation/programming-guide/#watermarks-and-late-data">Watermarks and late data</a></li>
- <li><a href="/documentation/programming-guide/#adding-timestamps-to-a-pcollections-elements">Adding timestamps to a PCollection’s elements</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Triggers</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#triggers">Trigger basics</a></li>
- <li><a href="/documentation/programming-guide/#event-time-triggers">Event time triggers and the default trigger</a></li>
- <li><a href="/documentation/programming-guide/#processing-time-triggers">Processing time triggers</a></li>
- <li><a href="/documentation/programming-guide/#data-driven-triggers">Data-driven triggers</a></li>
- <li><a href="/documentation/programming-guide/#setting-a-trigger">Setting a trigger</a></li>
- <li><a href="/documentation/programming-guide/#composite-triggers">Composite triggers</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Metrics</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#metrics">Metrics basics</a></li>
- <li><a href="/documentation/programming-guide/#types-of-metrics">Types of metrics</a></li>
- <li><a href="/documentation/programming-guide/#querying-metrics">Querying metrics</a></li>
- <li><a href="/documentation/programming-guide/#using-metrics">Using metrics in pipeline</a></li>
- </ul>
- </li>
- </ul>
-</li>
-
-<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Transform catalog</span>
-
- <ul class="section-nav-list">
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Python</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Element-wise</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
- <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
- <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
- <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
- <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
- <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
- <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
- <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
- <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
- <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
- <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
- <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Aggregation</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
- <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
- <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
- <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
- <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
- <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
- <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
- <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Other</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
- <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
- <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
- <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
- </ul>
- </li>
- </ul>
- </li>
-
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Java</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/java/overview/">Overview</a></li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Element-wise</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/java/elementwise/filter/">Filter</a></li>
- <li><a href="/documentation/transforms/java/elementwise/flatmapelements/">FlatMapElements</a></li>
- <li><a href="/documentation/transforms/java/elementwise/keys/">Keys</a></li>
- <li><a href="/documentation/transforms/java/elementwise/kvswap/">KvSwap</a></li>
- <li><a href="/documentation/transforms/java/elementwise/mapelements/">MapElements</a></li>
- <li><a href="/documentation/transforms/java/elementwise/pardo/">ParDo</a></li>
- <li><a href="/documentation/transforms/java/elementwise/partition/">Partition</a></li>
- <li><a href="/documentation/transforms/java/elementwise/regex/">Regex</a></li>
- <li><a href="/documentation/transforms/java/elementwise/reify/">Reify</a></li>
- <li><a href="/documentation/transforms/java/elementwise/values/">Values</a></li>
- <li><a href="/documentation/transforms/java/elementwise/tostring/">ToString</a></li>
- <li><a href="/documentation/transforms/java/elementwise/withkeys/">WithKeys</a></li>
- <li><a href="/documentation/transforms/java/elementwise/withtimestamps/">WithTimestamps</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Aggregation</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/java/aggregation/approximatequantiles/">ApproximateQuantiles</a></li>
- <li><a href="/documentation/transforms/java/aggregation/approximateunique/">ApproximateUnique</a></li>
- <li><a href="/documentation/transforms/java/aggregation/cogroupbykey/">CoGroupByKey</a></li>
- <li><a href="/documentation/transforms/java/aggregation/combine/">Combine</a></li>
- <li><a href="/documentation/transforms/java/aggregation/combinewithcontext/">CombineWithContext</a></li>
- <li><a href="/documentation/transforms/java/aggregation/count/">Count</a></li>
- <li><a href="/documentation/transforms/java/aggregation/distinct/">Distinct</a></li>
- <li><a href="/documentation/transforms/java/aggregation/groupbykey/">GroupByKey</a></li>
- <li><a href="/documentation/transforms/java/aggregation/groupintobatches/">GroupIntoBatches</a></li>
- <li><a href="/documentation/transforms/java/aggregation/latest/">Latest</a></li>
- <li><a href="/documentation/transforms/java/aggregation/max/">Max</a></li>
- <li><a href="/documentation/transforms/java/aggregation/mean/">Mean</a></li>
- <li><a href="/documentation/transforms/java/aggregation/min/">Min</a></li>
- <li><a href="/documentation/transforms/java/aggregation/sample/">Sample</a></li>
- <li><a href="/documentation/transforms/java/aggregation/sum/">Sum</a></li>
- <li><a href="/documentation/transforms/java/aggregation/top/">Top</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Other</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/java/other/create/">Create</a></li>
- <li><a href="/documentation/transforms/java/other/flatten/">Flatten</a></li>
- <li><a href="/documentation/transforms/java/other/passert/">PAssert</a></li>
- <li><a href="/documentation/transforms/java/other/view/">View</a></li>
- <li><a href="/documentation/transforms/java/other/window/">Window</a></li>
- </ul>
- </li>
- </ul>
- </li>
- </ul>
-
-</li>
-
-<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Common pipeline patterns</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/patterns/overview/">Overview</a></li>
- <li><a href="/documentation/patterns/file-processing/">File processing</a></li>
- <li><a href="/documentation/patterns/side-inputs/">Side inputs</a></li>
- <li><a href="/documentation/patterns/pipeline-options/">Pipeline options</a></li>
- <li><a href="/documentation/patterns/custom-io/">Custom I/O</a></li>
- <li><a href="/documentation/patterns/custom-windows/">Custom windows</a></li>
- </ul>
-</li>
-
-<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Learning resources</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/resources/learning-resources/#getting-started">Getting Started</a></li>
- <li><a href="/documentation/resources/learning-resources/#articles">Articles</a></li>
- <li><a href="/documentation/resources/learning-resources/#interactive-labs">Interactive Labs</a></li>
- <li><a href="/documentation/resources/learning-resources/#beam-katas">Beam Katas</a></li>
- <li><a href="/documentation/resources/learning-resources/#code-examples">Code Examples</a></li>
- <li><a href="/documentation/resources/learning-resources/#api-reference">API Reference</a></li>
- <li><a href="/documentation/resources/learning-resources/#feedback-and-suggestions">Feedback and Suggestions</a></li>
- <li><a href="/documentation/resources/learning-resources/#how-to-contribute">How to Contribute</a></li>
- <li><a href="/documentation/resources/videos-and-podcasts">Videos and Podcasts</a></li>
- </ul>
-</li>
-<li><a href="https://cwiki.apache.org/confluence/display/BEAM/Apache+Beam">Beam Wiki</a></li>
-
- </ul>
- </nav>
- </div>
-
- <nav class="page-nav clearfix" data-offset-top="90" data-offset-bottom="500">
- <!--
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-
-
-<ul class="nav">
- <li><a href="#processing-of-elements">Processing of elements</a>
- <ul>
- <li><a href="#serialization-and-communication">Serialization and communication</a></li>
- <li><a href="#bundling-and-persistence">Bundling and persistence</a></li>
- </ul>
- </li>
- <li><a href="#parallelism">Failures and parallelism within and between transforms</a>
- <ul>
- <li><a href="#data-parallelism">Data-parallelism within one transform</a></li>
- <li><a href="#dependent-parallellism">Dependent-parallelism between transforms</a></li>
- <li><a href="#failures-within-one-transform">Failures within one transform</a></li>
- <li><a href="#coupled-failure">Coupled failure: Failures between transforms</a></li>
- </ul>
- </li>
-</ul>
-
-
- </nav>
-
- <div class="body__contained body__section-nav">
- <!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-
-<h1 id="apache-beam-execution-model">Apache Beam Execution Model</h1>
-
-<p>The Beam model allows runners to execute your pipeline in different ways. You
-may observe various effects as a result of the runner’s choices. This page
-describes these effects so you can better understand how Beam pipelines execute.</p>
-
-<h2 id="processing-of-elements">Processing of elements</h2>
-
-<p>The serialization and communication of elements between machines is one of the
-most expensive operations in a distributed execution of your pipeline. Avoiding
-this serialization may require re-processing elements after failures or may
-limit the distribution of output to other machines.</p>
-
-<h3 id="serialization-and-communication">Serialization and communication</h3>
-
-<p>The runner might serialize elements between machines for communication purposes
-and for other reasons such as persistence.</p>
-
-<p>A runner may decide to transfer elements between transforms in a variety of
-ways, such as:</p>
-
-<ul>
- <li>Routing elements to a worker for processing as part of a grouping operation.
-This may involve serializing elements and grouping or sorting them by their
-key.</li>
- <li>Redistributing elements between workers to adjust parallelism. This may
-involve serializing elements and communicating them to other workers.</li>
- <li>Using the elements in a side input to a <code class="highlighter-rouge">ParDo</code>. This may require
-serializing the elements and broadcasting them to all the workers executing
-the <code class="highlighter-rouge">ParDo</code>.</li>
- <li>Passing elements between transforms that are running on the same worker.
-This may allow the runner to avoid serializing elements; instead, the runner
-can just pass the elements in memory.</li>
-</ul>
-
-<p>Some situations where the runner may serialize and persist elements are:</p>
-
-<ol>
- <li>When used as part of a stateful <code class="highlighter-rouge">DoFn</code>, the runner may persist values to some
-state mechanism.</li>
- <li>When committing the results of processing, the runner may persist the outputs
-as a checkpoint.</li>
-</ol>
-
-<h3 id="bundling-and-persistence">Bundling and persistence</h3>
-
-<p>Beam pipelines often focus on “<a href="https://en.wikipedia.org/wiki/embarrassingly_parallel">embarassingly parallel</a>”
-problems. Because of this, the APIs emphasize processing elements in parallel,
-which makes it difficult to express actions like “assign a sequence number to
-each element in a PCollection”. This is intentional as such algorithms are much
-more likely to suffer from scalability problems.</p>
-
-<p>Processing all elements in parallel also has some drawbacks. Specifically, it
-makes it impossible to batch any operations, such as writing elements to a sink
-or checkpointing progress during processing.</p>
-
-<p>Instead of processing all elements simultaneously, the elements in a
-<code class="highlighter-rouge">PCollection</code> are processed in <em>bundles</em>. The division of the collection into
-bundles is arbitrary and selected by the runner. This allows the runner to
-choose an appropriate middle-ground between persisting results after every
-element, and having to retry everything if there is a failure. For example, a
-streaming runner may prefer to process and commit small bundles, and a batch
-runner may prefer to process larger bundles.</p>
-
-<h2 id="parallelism">Failures and parallelism within and between transforms</h2>
-
-<p>In this section, we discuss how elements in the input collection are processed
-in parallel, and how transforms are retried when failures occur.</p>
-
-<h3 id="data-parallelism">Data-parallelism within one transform</h3>
-
-<p>When executing a single <code class="highlighter-rouge">ParDo</code>, a runner might divide an example input
-collection of nine elements into two bundles as shown in figure 1.</p>
-
-<p><img src="/images/execution_model_bundling.svg" alt="Bundle A contains five elements. Bundle B contains four elements." /></p>
-
-<p><em>Figure 1: A runner divides an input collection into two bundles.</em></p>
-
-<p>When the <code class="highlighter-rouge">ParDo</code> executes, workers may process the two bundles in parallel as
-shown in figure 2.</p>
-
-<p><img src="/images/execution_model_bundling_gantt.svg" alt="Two workers process the two bundles in parallel. Worker one processes bundle
- A. Worker two processes bundle B." /></p>
-
-<p><em>Figure 2: Two workers process the two bundles in parallel.</em></p>
-
-<p>Since elements cannot be split, the maximum parallelism for a transform depends
-on the number of elements in the collection. In figure 3, the input collection
-has nine elements, so the maximum parallelism is nine.</p>
-
-<p><img src="/images/execution_model_bundling_gantt_max.svg" alt="Nine workers process a nine element input collection in parallel." /></p>
-
-<p><em>Figure 3: Nine workers process a nine element input collection in parallel.</em></p>
-
-<p>Note: Splittable ParDo allows splitting the processing of a single input across
-multiple bundles. This feature is a work in progress.</p>
-
-<h3 id="dependent-parallellism">Dependent-parallelism between transforms</h3>
-
-<p><code class="highlighter-rouge">ParDo</code> transforms that are in sequence may be <em>dependently parallel</em> if the
-runner chooses to execute the consuming transform on the producing transform’s
-output elements without altering the bundling. In figure 4, <code class="highlighter-rouge">ParDo1</code> and
-<code class="highlighter-rouge">ParDo2</code> are <em>dependently parallel</em> if the output of <code class="highlighter-rouge">ParDo1</code> for a given
-element must be processed on the same worker.</p>
-
-<p><img src="/images/execution_model_bundling_multi.svg" alt="ParDo1 processes an input collection that contains bundles A and B. ParDo2 then
- processes the output collection from ParDo1, which contains bundles C and D." /></p>
-
-<p><em>Figure 4: Two transforms in sequence and their corresponding input collections.</em></p>
-
-<p>Figure 5 shows how these dependently parallel transforms might execute. The
-first worker executes <code class="highlighter-rouge">ParDo1</code> on the elements in bundle A (which results in
-bundle C), and then executes <code class="highlighter-rouge">ParDo2</code> on the elements in bundle C. Similarly,
-the second worker executes <code class="highlighter-rouge">ParDo1</code> on the elements in bundle B (which results
-in bundle D), and then executes <code class="highlighter-rouge">ParDo2</code> on the elements in bundle D.</p>
-
-<p><img src="/images/execution_model_bundling_multi_gantt.svg" alt="Worker one executes ParDo1 on bundle A and Pardo2 on bundle C. Worker two
- executes ParDo1 on bundle B and ParDo2 on bundle D." /></p>
-
-<p><em>Figure 5: Two workers execute dependently parallel ParDo transforms.</em></p>
-
-<p>Executing transforms this way allows a runner to avoid redistributing elements
-between workers, which saves on communication costs. However, the maximum parallelism
-now depends on the maximum parallelism of the first of the dependently parallel
-steps.</p>
-
-<h3 id="failures-within-one-transform">Failures within one transform</h3>
-
-<p>If processing of an element within a bundle fails, the entire bundle fails. The
-elements in the bundle must be retried (otherwise the entire pipeline fails),
-although they do not need to be retried with the same bundling.</p>
-
-<p>For this example, we will use the <code class="highlighter-rouge">ParDo</code> from figure 1 that has an input
-collection with nine elements and is divided into two bundles.</p>
-
-<p>In figure 6, the first worker successfully processes all five elements in bundle
-A. The second worker processes the four elements in bundle B: the first two
-elements were successfully processed, the third element’s processing failed, and
-there is one element still awaiting processing.</p>
-
-<p>We see that the runner retries all elements in bundle B and the processing
-completes successfully the second time. Note that the retry does not necessarily
-happen on the same worker as the original processing attempt, as shown in the
-figure.</p>
-
-<p><img src="/images/execution_model_failure_retry.svg" alt="Worker two fails to process an element in bundle B. Worker one finishes
- processing bundle A and then successfully retries to execute bundle B." /></p>
-
-<p><em>Figure 6: The processing of an element within bundle B fails, and another worker
-retries the entire bundle.</em></p>
-
-<p>Because we encountered a failure while processing an element in the input
-bundle, we had to reprocess <em>all</em> of the elements in the input bundle. This means
-the runner must throw away the entire output bundle since all of the results it
-contains will be recomputed.</p>
-
-<p>Note that if the failed transform is a <code class="highlighter-rouge">ParDo</code>, then the <code class="highlighter-rouge">DoFn</code> instance is torn
-down and abandoned.</p>
-
-<h3 id="coupled-failure">Coupled failure: Failures between transforms</h3>
-
-<p>If a failure to process an element in <code class="highlighter-rouge">ParDo2</code> causes <code class="highlighter-rouge">ParDo1</code> to re-execute,
-these two steps are said to be <em>co-failing</em>.</p>
-
-<p>For this example, we will use the two <code class="highlighter-rouge">ParDo</code>s from figure 4.</p>
-
-<p>In figure 7, worker two successfully executes <code class="highlighter-rouge">ParDo1</code> on all elements in bundle
-B. However, the worker fails to process an element in bundle D, so <code class="highlighter-rouge">ParDo2</code>
-fails (shown as the red X). As a result, the runner must discard and recompute
-the output of <code class="highlighter-rouge">ParDo2</code>. Because the runner was executing <code class="highlighter-rouge">ParDo1</code> and <code class="highlighter-rouge">ParDo2</code>
-together, the output bundle from <code class="highlighter-rouge">ParDo1</code> must also be thrown away, and all
-elements in the input bundle must be retried. These two <code class="highlighter-rouge">ParDo</code>s are co-failing.</p>
-
-<p><img src="/images/execution_model_bundling_coupled_failure.svg" alt="Worker two fails to process en element in bundle D, so all elements in both
- bundle B and bundle D must be retried." /></p>
-
-<p><em>Figure 7: Processing of an element within bundle D fails, so all elements in
-the input bundle are retried.</em></p>
-
-<p>Note that the retry does not necessarily have the same processing time as the
-original attempt, as shown in the diagram.</p>
-
-<p>All <code class="highlighter-rouge">DoFns</code> that experience coupled failures are terminated and must be torn
-down since they aren’t following the normal <code class="highlighter-rouge">DoFn</code> lifecycle .</p>
-
-<p>Executing transforms this way allows a runner to avoid persisting elements
-between transforms, saving on persistence costs.</p>
-
- </div>
- </div>
- <!--
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-<footer class="footer">
- <div class="footer__contained">
- <div class="footer__cols">
- <div class="footer__cols__col">
- <div class="footer__cols__col__logo">
- <img src="/images/beam_logo_circle.svg" class="footer__logo" alt="Beam logo">
- </div>
- <div class="footer__cols__col__logo">
- <img src="/images/apache_logo_circle.svg" class="footer__logo" alt="Apache logo">
- </div>
- </div>
- <div class="footer__cols__col footer__cols__col--md">
- <div class="footer__cols__col__title">Start</div>
- <div class="footer__cols__col__link"><a href="/get-started/beam-overview/">Overview</a></div>
- <div class="footer__cols__col__link"><a href="/get-started/quickstart-java/">Quickstart (Java)</a></div>
- <div class="footer__cols__col__link"><a href="/get-started/quickstart-py/">Quickstart (Python)</a></div>
- <div class="footer__cols__col__link"><a href="/get-started/quickstart-go/">Quickstart (Go)</a></div>
- <div class="footer__cols__col__link"><a href="/get-started/downloads/">Downloads</a></div>
- </div>
- <div class="footer__cols__col footer__cols__col--md">
- <div class="footer__cols__col__title">Docs</div>
- <div class="footer__cols__col__link"><a href="/documentation/programming-guide/">Concepts</a></div>
- <div class="footer__cols__col__link"><a href="/documentation/pipelines/design-your-pipeline/">Pipelines</a></div>
- <div class="footer__cols__col__link"><a href="/documentation/runners/capability-matrix/">Runners</a></div>
- </div>
- <div class="footer__cols__col footer__cols__col--md">
- <div class="footer__cols__col__title">Community</div>
- <div class="footer__cols__col__link"><a href="/contribute/">Contribute</a></div>
- <div class="footer__cols__col__link"><a href="https://projects.apache.org/committee.html?beam" target="_blank">Team<img src="/images/external-link-icon.png"
- width="14" height="14"
- alt="External link."></a></div>
- <div class="footer__cols__col__link"><a href="/contribute/presentation-materials/">Media</a></div>
- </div>
- <div class="footer__cols__col footer__cols__col--md">
- <div class="footer__cols__col__title">Resources</div>
- <div class="footer__cols__col__link"><a href="/blog/">Blog</a></div>
- <div class="footer__cols__col__link"><a href="/get-started/support/">Support</a></div>
- <div class="footer__cols__col__link"><a href="https://github.com/apache/beam">GitHub</a></div>
- </div>
- </div>
- </div>
- <div class="footer__bottom">
- ©
- <a href="http://www.apache.org">The Apache Software Foundation</a>
- | <a href="/privacy_policy">Privacy Policy</a>
- | <a href="/feed.xml">RSS Feed</a>
- <br><br>
- Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are
- either registered trademarks or trademarks of The Apache Software
- Foundation. All other products or name brands are trademarks of their
- respective holders, including The Apache Software Foundation.
- </div>
-</footer>
-
- </body>
+<html lang="en-US">
+<meta charset="utf-8">
+<title>Redirecting…</title>
+<link rel="canonical" href="https://beam.apache.org/documentation/runtime/model/">
+<meta http-equiv="refresh" content="0; url=https://beam.apache.org/documentation/runtime/model/">
+<h1>Redirecting…</h1>
+<a href="https://beam.apache.org/documentation/runtime/model/">Click here if you are not redirected.</a>
+<script>location="https://beam.apache.org/documentation/runtime/model/"</script>
</html>
diff --git a/website/generated-content/documentation/index.html b/website/generated-content/documentation/index.html
index 5f66eb5..7eca26c 100644
--- a/website/generated-content/documentation/index.html
+++ b/website/generated-content/documentation/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
@@ -487,7 +495,7 @@ limitations under the License.
<ul>
<li>Read the <a href="/documentation/programming-guide/">Programming Guide</a>, which introduces all the key Beam concepts.</li>
- <li>Learn about Beam’s <a href="/documentation/execution-model/">execution model</a> to better understand how pipelines execute.</li>
+ <li>Learn about Beam’s <a href="/documentation/runtime/model">execution model</a> to better understand how pipelines execute.</li>
<li>Visit <a href="/documentation/resources/learning-resources">Learning Resources</a> for some of our favorite articles and talks about Beam.</li>
</ul>
diff --git a/website/generated-content/documentation/io/built-in/google-bigquery/index.html b/website/generated-content/documentation/io/built-in/google-bigquery/index.html
index 95b4b46..6cafde6 100644
--- a/website/generated-content/documentation/io/built-in/google-bigquery/index.html
+++ b/website/generated-content/documentation/io/built-in/google-bigquery/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/io/built-in/hadoop/index.html b/website/generated-content/documentation/io/built-in/hadoop/index.html
index a386fc6..263b97c 100644
--- a/website/generated-content/documentation/io/built-in/hadoop/index.html
+++ b/website/generated-content/documentation/io/built-in/hadoop/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/io/built-in/hcatalog/index.html b/website/generated-content/documentation/io/built-in/hcatalog/index.html
index 69ccf15..09c38c7 100644
--- a/website/generated-content/documentation/io/built-in/hcatalog/index.html
+++ b/website/generated-content/documentation/io/built-in/hcatalog/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/io/built-in/index.html b/website/generated-content/documentation/io/built-in/index.html
index 4d1e8c8..1cdae9e 100644
--- a/website/generated-content/documentation/io/built-in/index.html
+++ b/website/generated-content/documentation/io/built-in/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/io/built-in/parquet/index.html b/website/generated-content/documentation/io/built-in/parquet/index.html
index e358d5d..b75add8 100644
--- a/website/generated-content/documentation/io/built-in/parquet/index.html
+++ b/website/generated-content/documentation/io/built-in/parquet/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/io/developing-io-java/index.html b/website/generated-content/documentation/io/developing-io-java/index.html
index 1993590..5c2a867 100644
--- a/website/generated-content/documentation/io/developing-io-java/index.html
+++ b/website/generated-content/documentation/io/developing-io-java/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/io/developing-io-overview/index.html b/website/generated-content/documentation/io/developing-io-overview/index.html
index 3cf146b..d3eb36a 100644
--- a/website/generated-content/documentation/io/developing-io-overview/index.html
+++ b/website/generated-content/documentation/io/developing-io-overview/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/io/developing-io-python/index.html b/website/generated-content/documentation/io/developing-io-python/index.html
index d3ae740..8025db4 100644
--- a/website/generated-content/documentation/io/developing-io-python/index.html
+++ b/website/generated-content/documentation/io/developing-io-python/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/io/testing/index.html b/website/generated-content/documentation/io/testing/index.html
index 26dcf1e..f9a4217 100644
--- a/website/generated-content/documentation/io/testing/index.html
+++ b/website/generated-content/documentation/io/testing/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/patterns/custom-io/index.html b/website/generated-content/documentation/patterns/custom-io/index.html
index e436ea2..5c1ec9b 100644
--- a/website/generated-content/documentation/patterns/custom-io/index.html
+++ b/website/generated-content/documentation/patterns/custom-io/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/patterns/custom-windows/index.html b/website/generated-content/documentation/patterns/custom-windows/index.html
index d4037b4..42fac1d 100644
--- a/website/generated-content/documentation/patterns/custom-windows/index.html
+++ b/website/generated-content/documentation/patterns/custom-windows/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/patterns/file-processing/index.html b/website/generated-content/documentation/patterns/file-processing/index.html
index 58329da..18a5586 100644
--- a/website/generated-content/documentation/patterns/file-processing/index.html
+++ b/website/generated-content/documentation/patterns/file-processing/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/patterns/overview/index.html b/website/generated-content/documentation/patterns/overview/index.html
index 130d70b..c306c47 100644
--- a/website/generated-content/documentation/patterns/overview/index.html
+++ b/website/generated-content/documentation/patterns/overview/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/patterns/pipeline-options/index.html b/website/generated-content/documentation/patterns/pipeline-options/index.html
index 4d55cf2..836e7a4 100644
--- a/website/generated-content/documentation/patterns/pipeline-options/index.html
+++ b/website/generated-content/documentation/patterns/pipeline-options/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/patterns/side-inputs/index.html b/website/generated-content/documentation/patterns/side-inputs/index.html
index f10c20f..af04023 100644
--- a/website/generated-content/documentation/patterns/side-inputs/index.html
+++ b/website/generated-content/documentation/patterns/side-inputs/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/pipelines/create-your-pipeline/index.html b/website/generated-content/documentation/pipelines/create-your-pipeline/index.html
index 3a06e37..aab1b07 100644
--- a/website/generated-content/documentation/pipelines/create-your-pipeline/index.html
+++ b/website/generated-content/documentation/pipelines/create-your-pipeline/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/pipelines/design-your-pipeline/index.html b/website/generated-content/documentation/pipelines/design-your-pipeline/index.html
index 38b5c00..fdf5ef9 100644
--- a/website/generated-content/documentation/pipelines/design-your-pipeline/index.html
+++ b/website/generated-content/documentation/pipelines/design-your-pipeline/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/pipelines/test-your-pipeline/index.html b/website/generated-content/documentation/pipelines/test-your-pipeline/index.html
index 3af7221..127ca82 100644
--- a/website/generated-content/documentation/pipelines/test-your-pipeline/index.html
+++ b/website/generated-content/documentation/pipelines/test-your-pipeline/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/programming-guide/index.html b/website/generated-content/documentation/programming-guide/index.html
index cee3367..55f620c 100644
--- a/website/generated-content/documentation/programming-guide/index.html
+++ b/website/generated-content/documentation/programming-guide/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/resources/learning-resources/index.html b/website/generated-content/documentation/resources/learning-resources/index.html
index f1801ad..45bd8d9 100644
--- a/website/generated-content/documentation/resources/learning-resources/index.html
+++ b/website/generated-content/documentation/resources/learning-resources/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
@@ -571,7 +579,7 @@ limitations under the License.
<li><strong><a href="https://beam.apache.org/documentation/programming-guide/">Programming Guide</a></strong> - The Programming Guide contains more in-depth information on most topics in the Apache Beam SDK. These include descriptions on how everything works as well as code snippets to see how to use every part. This can be used as a reference guidebook.</li>
<li><strong><a href="https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101">The world beyond batch: Streaming 101</a></strong> - Covers some basic background information, terminology, time domains, batch processing, and streaming.</li>
<li><strong><a href="https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102">The world beyond batch: Streaming 102</a></strong> - Tour of the unified batch and streaming programming model in Beam, alongside with an example to explain many of the concepts.</li>
- <li><strong><a href="https://beam.apache.org/documentation/execution-model/">Apache Beam Execution Model</a></strong> - Explanation on how runners execute an Apache Beam pipeline. This includes why serialization is important, and how a runner might distribute the work in parallel to multiple machines.</li>
+ <li><strong><a href="https://beam.apache.org/documentation/runtime/model">Apache Beam Execution Model</a></strong> - Explanation on how runners execute an Apache Beam pipeline. This includes why serialization is important, and how a runner might distribute the work in parallel to multiple machines.</li>
</ul>
<h3 id="common-patterns">Common Patterns</h3>
diff --git a/website/generated-content/documentation/resources/videos-and-podcasts/index.html b/website/generated-content/documentation/resources/videos-and-podcasts/index.html
index fbc456e..ea2b248 100644
--- a/website/generated-content/documentation/resources/videos-and-podcasts/index.html
+++ b/website/generated-content/documentation/resources/videos-and-podcasts/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/runners/jstorm/index.html b/website/generated-content/documentation/runners/jstorm/index.html
index 63cef60..22200bc 100644
--- a/website/generated-content/documentation/runners/jstorm/index.html
+++ b/website/generated-content/documentation/runners/jstorm/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/io/developing-io-overview/index.html b/website/generated-content/documentation/runtime/environments/index.html
similarity index 72%
copy from website/generated-content/documentation/io/developing-io-overview/index.html
copy to website/generated-content/documentation/runtime/environments/index.html
index 3cf146b..c1245ae 100644
--- a/website/generated-content/documentation/io/developing-io-overview/index.html
+++ b/website/generated-content/documentation/runtime/environments/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Overview: Developing a new I/O connector</title>
+ <title>Runtime environments</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/io/developing-io-overview/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/runtime/environments/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -141,7 +141,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/io/developing-io-overview.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/runtime/environments.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
@@ -448,13 +456,19 @@
<ul class="nav">
- <li><a href="#sources">Sources</a>
+ <li><a href="#customizing-container-images">Customizing container images</a>
+ <ul>
+ <li><a href="#writing-new-dockerfiles">Writing new Dockerfiles on top of the original</a></li>
+ <li><a href="#modifying-dockerfiles">Modifying the original Dockerfile</a></li>
+ <li><a href="#testing-customized-images">Testing customized images</a></li>
+ </ul>
+ </li>
+ <li><a href="#building-container-images">Building container images</a>
<ul>
- <li><a href="#when-to-use-source">When to use the Source interface</a></li>
- <li><a href="#using-pardo-and-groupbykey">Using ParDo and GroupByKey</a></li>
+ <li><a href="#overriding-default-docker-targets">Overriding default Docker targets</a></li>
</ul>
</li>
- <li><a href="#sinks">Sinks</a></li>
+ <li><a href="#pushing-container-images">Pushing container images</a></li>
</ul>
@@ -475,191 +489,179 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="overview-developing-a-new-io-connector">Overview: Developing a new I/O connector</h1>
-
-<p><em>A guide for users who need to connect to a data store that isn’t supported by
-the <a href="/documentation/io/built-in/">Built-in I/O connectors</a></em></p>
-
-<p>To connect to a data store that isn’t supported by Beam’s existing I/O
-connectors, you must create a custom I/O connector. A connector usually consists
-of a source and a sink. All Beam sources and sinks are composite transforms;
-however, the implementation of your custom I/O depends on your use case. Here
-are the recommended steps to get started:</p>
-
-<ol>
- <li>
- <p>Read this overview and choose your implementation. You can email the
-<a href="/get-started/support">Beam dev mailing list</a> with any
-questions you might have. In addition, you can check if anyone else is
-working on the same I/O connector.</p>
- </li>
- <li>
- <p>If you plan to contribute your I/O connector to the Beam community, see the
-<a href="/contribute/contribution-guide/">Apache Beam contribution guide</a>.</p>
- </li>
- <li>
- <p>Read the <a href="/contribute/ptransform-style-guide/">PTransform style guide</a>
-for additional style guide recommendations.</p>
- </li>
-</ol>
-
-<h2 id="sources">Sources</h2>
+<h1 id="runtime-environments">Runtime environments</h1>
-<p>For <strong>bounded (batch) sources</strong>, there are currently two options for creating a
-Beam source:</p>
+<p>The Beam SDK runtime environment is isolated from other runtime systems because the SDK runtime environment is <a href="https://s.apache.org/beam-fn-api-container-contract">containerized</a> with <a href="https://www.docker.com/">Docker</a>. This means that any execution engine can run the Beam SDK.</p>
-<ol>
- <li>
- <p>Use <code class="highlighter-rouge">ParDo</code> and <code class="highlighter-rouge">GroupByKey</code>.</p>
- </li>
- <li>
- <p>Use the <code class="highlighter-rouge">Source</code> interface and extend the <code class="highlighter-rouge">BoundedSource</code> abstract subclass.</p>
- </li>
-</ol>
+<p>This page describes how to customize, build, and push Beam SDK container images.</p>
-<p><code class="highlighter-rouge">ParDo</code> is the recommended option, as implementing a <code class="highlighter-rouge">Source</code> can be tricky. See
-<a href="#when-to-use-source">When to use the Source interface</a> for a list of some use
-cases where you might want to use a <code class="highlighter-rouge">Source</code> (such as
-<a href="/blog/2016/05/18/splitAtFraction-method.html">dynamic work rebalancing</a>).</p>
+<p>Before you begin, install <a href="https://www.docker.com/">Docker</a> on your workstation.</p>
-<p>(Java only) For <strong>unbounded (streaming) sources</strong>, you must use the <code class="highlighter-rouge">Source</code>
-interface and extend the <code class="highlighter-rouge">UnboundedSource</code> abstract subclass. <code class="highlighter-rouge">UnboundedSource</code>
-supports features that are useful for streaming pipelines, such as
-checkpointing.</p>
+<h2 id="customizing-container-images">Customizing container images</h2>
-<p>Splittable DoFn is a new sources framework that is under development and will
-replace the other options for developing bounded and unbounded sources. For more
-information, see the
-<a href="/roadmap/connectors-multi-sdk/">roadmap for multi-SDK connector efforts</a>.</p>
+<p>You can add extra dependencies to container images so that you don’t have to supply the dependencies to execution engines.</p>
-<h3 id="when-to-use-source">When to use the Source interface</h3>
+<p>To customize a container image, either:</p>
+<ul>
+ <li><a href="#writing-new-dockerfiles">Write a new</a> <a href="https://docs.docker.com/engine/reference/builder/">Dockerfile</a> on top of the original.</li>
+ <li><a href="#modifying-dockerfiles">Modify</a> the <a href="https://github.com/apache/beam/blob/master/sdks/python/container/Dockerfile">original Dockerfile</a> and reimage the container.</li>
+</ul>
-<p>If you are not sure whether to use <code class="highlighter-rouge">Source</code>, feel free to email the <a href="/get-started/support">Beam dev
-mailing list</a> and we can discuss the
-specific pros and cons of your case.</p>
+<p>It’s often easier to write a new Dockerfile. However, by modifying the original Dockerfile, you can customize anything (including the base OS).</p>
-<p>In some cases, implementing a <code class="highlighter-rouge">Source</code> might be necessary or result in better
-performance:</p>
+<h3 id="writing-new-dockerfiles">Writing new Dockerfiles on top of the original</h3>
-<ul>
- <li>
- <p><strong>Unbounded sources:</strong> <code class="highlighter-rouge">ParDo</code> does not work for reading from unbounded
-sources. <code class="highlighter-rouge">ParDo</code> does not support checkpointing or mechanisms like de-duping
-that are useful for streaming data sources.</p>
- </li>
- <li>
- <p><strong>Progress and size estimation:</strong> <code class="highlighter-rouge">ParDo</code> can’t provide hints to runners about
-progress or the size of data they are reading. Without size estimation of the
-data or progress on your read, the runner doesn’t have any way to guess how
-large your read will be. Therefore, if the runner attempts to dynamically
-allocate workers, it does not have any clues as to how many workers you might
-need for your pipeline.</p>
- </li>
- <li>
- <p><strong>Dynamic work rebalancing:</strong> <code class="highlighter-rouge">ParDo</code> does not support dynamic work
-rebalancing, which is used by some readers to improve the processing speed of
-jobs. Depending on your data source, dynamic work rebalancing might not be
-possible.</p>
- </li>
- <li>
- <p><strong>Splitting into parts of particular size recommended by the runner:</strong> <code class="highlighter-rouge">ParDo</code>
-does not receive <code class="highlighter-rouge">desired_bundle_size</code> as a hint from runners when performing
-initial splitting.</p>
+<ol>
+ <li>Pull a <a href="https://hub.docker.com/u/apachebeam">prebuilt SDK container image</a> for your <a href="https://docs.docker.com/docker-hub/repos/#searching-for-repositories">target</a> language and version. The following example pulls the latest Python SDK:
+ <div class="highlighter-rouge"><pre class="highlight"><code>docker pull apachebeam/python3.7_sdk
+</code></pre>
+ </div>
</li>
-</ul>
+ <li><a href="https://docs.docker.com/develop/develop-images/dockerfile_best-practices/">Write a new Dockerfile</a> that <a href="https://docs.docker.com/engine/reference/builder/#from">designates</a> the original as its <a href="https://docs.docker.com/glossary/?term=parent%20image">parent</a>.</li>
+ <li><a href="#building-container-images">Build</a> a child image.</li>
+</ol>
-<p>For example, if you’d like to read from a new file format that contains many
-records per file, or if you’d like to read from a key-value store that supports
-read operations in sorted key order.</p>
+<h3 id="modifying-dockerfiles">Modifying the original Dockerfile</h3>
-<h3 id="using-pardo-and-groupbykey">Using ParDo and GroupByKey</h3>
+<ol>
+ <li>Clone the <code class="highlighter-rouge">beam</code> repository:
+ <div class="highlighter-rouge"><pre class="highlight"><code>git clone https://github.com/apache/beam.git
+</code></pre>
+ </div>
+ </li>
+ <li>Customize the <a href="https://github.com/apache/beam/blob/master/sdks/python/container/Dockerfile">Dockerfile</a>. If you’re adding dependencies from <a href="https://pypi.org/">PyPI</a>, use <a href="https://github.com/apache/beam/blob/master/sdks/python/container/base_image_requirements.txt"><code class="highlighter-rouge">base_image_requirements.txt</code></a> instead.</li>
+ <li><a href="#building-container-images">Reimage</a> the container.</li>
+</ol>
-<p>For data stores or file types where the data can be read in parallel, you can
-think of the process as a mini-pipeline. This often consists of two steps:</p>
+<h3 id="testing-customized-images">Testing customized images</h3>
+
+<p>To test a customized image locally, run a pipeline with PortableRunner and set the <code class="highlighter-rouge">--environment_config</code> flag to the image path:</p>
+
+<div class="runner-direct highlighter-rouge"><pre class="highlight"><code>python -m apache_beam.examples.wordcount \
+--input=/path/to/inputfile \
+--output /path/to/write/counts \
+--runner=PortableRunner \
+--job_endpoint=embed \
+--environment_config=path/to/container/image
+</code></pre>
+</div>
+
+<div class="runner-flink-local highlighter-rouge"><pre class="highlight"><code># Start a Flink job server on localhost:8099
+./gradlew :runners:flink:1.5:job-server:runShadow
+
+# Run a pipeline on the Flink job server
+python -m apache_beam.examples.wordcount \
+--input=/path/to/inputfile \
+--output=/path/to/write/counts \
+--runner=PortableRunner \
+--job_endpoint=localhost:8099 \
+--environment_config=path/to/container/image
+</code></pre>
+</div>
+
+<div class="runner-spark-local highlighter-rouge"><pre class="highlight"><code># Start a Spark job server on localhost:8099
+./gradlew :runners:spark:job-server:runShadow
+
+# Run a pipeline on the Spark job server
+python -m apache_beam.examples.wordcount \
+--input=/path/to/inputfile \
+--output=path/to/write/counts \
+--runner=PortableRunner \
+--job_endpoint=localhost:8099 \
+--environment_config=path/to/container/image
+</code></pre>
+</div>
+
+<p>To test a customized image on the Google Cloud Dataflow runner, use the <code class="highlighter-rouge">DataflowRunner</code> option and the <code class="highlighter-rouge">worker_harness_container_image</code> flag:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>python -m apache_beam.examples.wordcount \
+--input=path/to/inputfile \
+--output=/path/to/write/counts \
+--runner=DataflowRunner \
+--project={gcp_project_id} \
+--temp_location={gcs_location} \ \
+--experiment=beam_fn_api \
+--sdk_location=[…]/beam/sdks/python/container/py{version}/build/target/apache-beam.tar.gz \
+--worker_harness_container_image=path/to/container/image
+
+# The sdk_location option accepts four Python version variables: 2, 35, 36, and 37
+</code></pre>
+</div>
+
+<h2 id="building-container-images">Building container images</h2>
+
+<p>To build Beam SDK container images:</p>
<ol>
- <li>
- <p>Splitting the data into parts to be read in parallel</p>
- </li>
- <li>
- <p>Reading from each of those parts</p>
- </li>
+ <li>Navigate to the local copy of your <a href="#customizing-container-images">customized container image</a>.</li>
+ <li>Run Gradle with the <code class="highlighter-rouge">docker</code> target. If you’re <a href="#writing-new-dockerfiles">building a child image</a>, set the optional <code class="highlighter-rouge">--file</code> flag to the new Dockerfile. If you’re <a href="#modifying-dockerfiles">building an image from an original Dockerfile</a>, ignore the <code class="highlighter-rouge">--file</code> flag and use a default repository:</li>
</ol>
-<p>Each of those steps will be a <code class="highlighter-rouge">ParDo</code>, with a <code class="highlighter-rouge">GroupByKey</code> in between. The
-<code class="highlighter-rouge">GroupByKey</code> is an implementation detail, but for most runners <code class="highlighter-rouge">GroupByKey</code>
-allows the runner to use different numbers of workers in some situations:</p>
+<div class="highlighter-rouge"><pre class="highlight"><code># The default repository of each SDK
+./gradlew [--file=path/to/new/Dockerfile] :sdks:java:container:docker
+./gradlew [--file=path/to/new/Dockerfile] :sdks:go:container:docker
+./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container:py2:docker
+./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container:py35:docker
+./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container:py36:docker
+./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container:py37:docker
-<ul>
- <li>
- <p>Determining how to split up the data to be read into chunks</p>
- </li>
- <li>
- <p>Reading data, which often benefits from more workers</p>
- </li>
-</ul>
+# Shortcut for building all four Python SDKs
+./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container buildAll
+</code></pre>
+</div>
-<p>In addition, <code class="highlighter-rouge">GroupByKey</code> also allows dynamic work rebalancing to happen on
-runners that support the feature.</p>
+<p>To examine the containers that you built, run <code class="highlighter-rouge">docker images</code> from anywhere in the command line. If you successfully built all of the container images, the command prints a table like the following:</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>REPOSITORY TAG IMAGE ID CREATED SIZE
+apachebeam/java_sdk latest 16ca619d489e 2 weeks ago 550MB
+apachebeam/python2.7_sdk latest b6fb40539c29 2 weeks ago 1.78GB
+apachebeam/python3.5_sdk latest bae309000d09 2 weeks ago 1.85GB
+apachebeam/python3.6_sdk latest 42faad307d1a 2 weeks ago 1.86GB
+apachebeam/python3.7_sdk latest 18267df54139 2 weeks ago 1.86GB
+apachebeam/go_sdk latest 30cf602e9763 2 weeks ago 124MB
+</code></pre>
+</div>
-<p>Here are some examples of read transform implementations that use the “reading
-as a mini-pipeline” model when data can be read in parallel:</p>
+<h3 id="overriding-default-docker-targets">Overriding default Docker targets</h3>
-<ul>
- <li><strong>Reading from a file glob</strong>: For example, reading all files in “~/data/**”.
- <ul>
- <li>Get File Paths <code class="highlighter-rouge">ParDo</code>: As input, take in a file glob. Produce a
-<code class="highlighter-rouge">PCollection</code> of strings, each of which is a file path.</li>
- <li>Reading <code class="highlighter-rouge">ParDo</code>: Given the <code class="highlighter-rouge">PCollection</code> of file paths, read each one,
-producing a <code class="highlighter-rouge">PCollection</code> of records.</li>
- </ul>
- </li>
- <li><strong>Reading from a NoSQL database</strong> (such as Apache HBase): These databases
-often allow reading from ranges in parallel.
- <ul>
- <li>Determine Key Ranges <code class="highlighter-rouge">ParDo</code>: As input, receive connection information for
-the database and the key range to read from. Produce a <code class="highlighter-rouge">PCollection</code> of key
-ranges that can be read in parallel efficiently.</li>
- <li>Read Key Range <code class="highlighter-rouge">ParDo</code>: Given the <code class="highlighter-rouge">PCollection</code> of key ranges, read the key
-range, producing a <code class="highlighter-rouge">PCollection</code> of records.</li>
- </ul>
- </li>
-</ul>
+<p>The default <a href="https://docs.docker.com/engine/reference/commandline/tag/">tag</a> is <code class="highlighter-rouge">latest</code> and the default repositories are in the Docker Hub <code class="highlighter-rouge">apachebeam</code> namespace. The <code class="highlighter-rouge">docker</code> command-line tool implicitly <a href="#pushing-container-images">pushes container images</a> to this location.</p>
-<p>For data stores or files where reading cannot occur in parallel, reading is a
-simple task that can be accomplished with a single <code class="highlighter-rouge">ParDo</code>+<code class="highlighter-rouge">GroupByKey</code>. For
-example:</p>
+<p>To tag a local image, set the <code class="highlighter-rouge">docker-tag</code> option when building the container. The following command tags a Python SDK image with a date.</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>./gradlew :sdks:python:container:py2:docker -Pdocker-tag=2019-10-04
+</code></pre>
+</div>
-<ul>
- <li>
- <p><strong>Reading from a database query</strong>: Traditional SQL database queries often
-can only be read in sequence. In this case, the <code class="highlighter-rouge">ParDo</code> would establish a
-connection to the database and read batches of records, producing a
-<code class="highlighter-rouge">PCollection</code> of those records.</p>
- </li>
- <li>
- <p><strong>Reading from a gzip file</strong>: A gzip file must be read in order, so the read
-cannot be parallelized. In this case, the <code class="highlighter-rouge">ParDo</code> would open the file and
-read in sequence, producing a <code class="highlighter-rouge">PCollection</code> of records from the file.</p>
- </li>
-</ul>
+<p>To change the repository, set the <code class="highlighter-rouge">docker-repository-root</code> option to a new location. The following command sets the <code class="highlighter-rouge">docker-repository-root</code> to a Bintray repository named <code class="highlighter-rouge">apache</code>.</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>./gradlew :sdks:python:container:py2:docker -Pdocker-repository-root=$USER-docker-apache.bintray.io/beam/python
+</code></pre>
+</div>
-<h2 id="sinks">Sinks</h2>
+<h2 id="pushing-container-images">Pushing container images</h2>
-<p>To create a Beam sink, we recommend that you use a <code class="highlighter-rouge">ParDo</code> that writes the
-received records to the data store. To develop more complex sinks (for example,
-to support data de-duplication when failures are retried by a runner), use
-<code class="highlighter-rouge">ParDo</code>, <code class="highlighter-rouge">GroupByKey</code>, and other available Beam transforms.</p>
+<p>After <a href="#building-container-images">building a container image</a>, you can store it in a remote Docker repository.</p>
-<p>For <strong>file-based sinks</strong>, you can use the <code class="highlighter-rouge">FileBasedSink</code> abstraction that is
-provided by both the Java and Python SDKs. See our language specific
-implementation guides for more details:</p>
+<p>The following steps push a Python SDK image to the <a href="#overriding-default-docker-targets"><code class="highlighter-rouge">docker-root-repository</code> value</a>.</p>
-<ul>
- <li><a href="/documentation/io/developing-io-java/">Developing I/O connectors for Java</a></li>
- <li><a href="/documentation/io/developing-io-python/">Developing I/O connectors for Python</a></li>
-</ul>
+<ol>
+ <li>Sign in to your Docker registry:
+ <div class="highlighter-rouge"><pre class="highlight"><code>docker login
+</code></pre>
+ </div>
+ </li>
+ <li>Navigate to the local copy of your container image and upload it to the remote repository:
+ <div class="highlighter-rouge"><pre class="highlight"><code>docker push apachebeam/python2.7_sdk
+</code></pre>
+ </div>
+ </li>
+</ol>
+
+<p>To download the image again, run <code class="highlighter-rouge">docker pull</code>:</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>docker pull apachebeam/python2.7_sdk
+</code></pre>
+</div>
+<blockquote>
+ <p><strong>Note</strong>: After pushing a container image, the remote image ID and digest match the local image ID and digest.</p>
+</blockquote>
</div>
</div>
diff --git a/website/generated-content/documentation/execution-model/index.html b/website/generated-content/documentation/runtime/model/index.html
similarity index 98%
copy from website/generated-content/documentation/execution-model/index.html
copy to website/generated-content/documentation/runtime/model/index.html
index 511fb90..77b64e8 100644
--- a/website/generated-content/documentation/execution-model/index.html
+++ b/website/generated-content/documentation/runtime/model/index.html
@@ -28,7 +28,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Beam Execution Model</title>
+ <title>Execution model</title>
<meta name="description" content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400" rel="stylesheet">
@@ -39,7 +39,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/documentation/execution-model/" data-proofer-ignore>
+ <link rel="canonical" href="https://beam.apache.org/documentation/runtime/model/" data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam" href="https://beam.apache.org/feed.xml">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css" integrity="sha384-5sAR7xN1Nv6T6+dT2mhtzEpVJvfS3NScPQTrOxhwjIuvcA67KV2R5Jz6kr4abQsz" crossorigin="anonymous">
@@ -141,7 +141,7 @@
GitHub links will not resolve until the markdown source is available on the master branch.
New pages would fail validation during development / PR test automation.
-->
- <a href="https://github.com/apache/beam/edit/master/website/src/documentation/execution-model.md" data-proofer-ignore>
+ <a href="https://github.com/apache/beam/edit/master/website/src/documentation/runtime/model.md" data-proofer-ignore>
<i class="far fa-edit fa-lg" alt="Edit on GitHub" title="Edit on GitHub"></i>
</a>
</li>
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
@@ -482,7 +490,7 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-execution-model">Apache Beam Execution Model</h1>
+<h1 id="execution-model">Execution model</h1>
<p>The Beam model allows runners to execute your pipeline in different ways. You
may observe various effects as a result of the runner’s choices. This page
diff --git a/website/generated-content/documentation/transforms/java/aggregation/approximatequantiles/index.html b/website/generated-content/documentation/transforms/java/aggregation/approximatequantiles/index.html
index 75d68ec..0b3c6da 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/approximatequantiles/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/approximatequantiles/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/approximateunique/index.html b/website/generated-content/documentation/transforms/java/aggregation/approximateunique/index.html
index d4b56ea..9bff142 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/approximateunique/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/approximateunique/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/cogroupbykey/index.html b/website/generated-content/documentation/transforms/java/aggregation/cogroupbykey/index.html
index 7b6a414..3ff8b47 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/cogroupbykey/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/cogroupbykey/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/combine/index.html b/website/generated-content/documentation/transforms/java/aggregation/combine/index.html
index 297e30a..aedfdfa 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/combine/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/combine/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/combinewithcontext/index.html b/website/generated-content/documentation/transforms/java/aggregation/combinewithcontext/index.html
index f295302..a587b3c 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/combinewithcontext/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/combinewithcontext/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/count/index.html b/website/generated-content/documentation/transforms/java/aggregation/count/index.html
index 842dce7..6fba48e 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/count/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/count/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/distinct/index.html b/website/generated-content/documentation/transforms/java/aggregation/distinct/index.html
index f652d2d..82d781f 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/distinct/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/distinct/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/groupbykey/index.html b/website/generated-content/documentation/transforms/java/aggregation/groupbykey/index.html
index b793383..5b2a4c5 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/groupbykey/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/groupbykey/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/groupintobatches/index.html b/website/generated-content/documentation/transforms/java/aggregation/groupintobatches/index.html
index 3438301..9764db9 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/groupintobatches/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/groupintobatches/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/latest/index.html b/website/generated-content/documentation/transforms/java/aggregation/latest/index.html
index 56e202c..78358bc 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/latest/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/latest/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/max/index.html b/website/generated-content/documentation/transforms/java/aggregation/max/index.html
index 455365c..e1efc43 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/max/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/max/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/mean/index.html b/website/generated-content/documentation/transforms/java/aggregation/mean/index.html
index 36d3fff..3746376 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/mean/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/mean/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/min/index.html b/website/generated-content/documentation/transforms/java/aggregation/min/index.html
index 542f30a..e8c4a23 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/min/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/min/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/sample/index.html b/website/generated-content/documentation/transforms/java/aggregation/sample/index.html
index 124199a..0316730 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/sample/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/sample/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/sum/index.html b/website/generated-content/documentation/transforms/java/aggregation/sum/index.html
index 6dffaaf..975fc3a 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/sum/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/sum/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/aggregation/top/index.html b/website/generated-content/documentation/transforms/java/aggregation/top/index.html
index 39e2c0c..bb4ecd1 100644
--- a/website/generated-content/documentation/transforms/java/aggregation/top/index.html
+++ b/website/generated-content/documentation/transforms/java/aggregation/top/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/filter/index.html b/website/generated-content/documentation/transforms/java/elementwise/filter/index.html
index 25da329..787641f 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/filter/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/filter/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/flatmapelements/index.html b/website/generated-content/documentation/transforms/java/elementwise/flatmapelements/index.html
index 6a99247..c234c05 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/flatmapelements/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/flatmapelements/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/keys/index.html b/website/generated-content/documentation/transforms/java/elementwise/keys/index.html
index 2e1db5b..3949154 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/keys/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/keys/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/kvswap/index.html b/website/generated-content/documentation/transforms/java/elementwise/kvswap/index.html
index ad6d6f6..f139033 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/kvswap/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/kvswap/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/mapelements/index.html b/website/generated-content/documentation/transforms/java/elementwise/mapelements/index.html
index f571de9..c6b1ea4 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/mapelements/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/mapelements/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/pardo/index.html b/website/generated-content/documentation/transforms/java/elementwise/pardo/index.html
index ca73a26..7e80c95 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/pardo/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/pardo/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/partition/index.html b/website/generated-content/documentation/transforms/java/elementwise/partition/index.html
index fd15650..302d056 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/partition/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/partition/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/regex/index.html b/website/generated-content/documentation/transforms/java/elementwise/regex/index.html
index 11ce423..1f1c1d7 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/regex/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/regex/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/reify/index.html b/website/generated-content/documentation/transforms/java/elementwise/reify/index.html
index 486d7c6..68ab28b 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/reify/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/reify/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/tostring/index.html b/website/generated-content/documentation/transforms/java/elementwise/tostring/index.html
index 2effef1..8bb3c26 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/tostring/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/tostring/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/values/index.html b/website/generated-content/documentation/transforms/java/elementwise/values/index.html
index daf51a3..3e70f04 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/values/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/values/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/withkeys/index.html b/website/generated-content/documentation/transforms/java/elementwise/withkeys/index.html
index 208b9a1..7714b07 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/withkeys/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/withkeys/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/elementwise/withtimestamps/index.html b/website/generated-content/documentation/transforms/java/elementwise/withtimestamps/index.html
index 265aa88..dc9be19 100644
--- a/website/generated-content/documentation/transforms/java/elementwise/withtimestamps/index.html
+++ b/website/generated-content/documentation/transforms/java/elementwise/withtimestamps/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/other/create/index.html b/website/generated-content/documentation/transforms/java/other/create/index.html
index 815a093..f2d42ab 100644
--- a/website/generated-content/documentation/transforms/java/other/create/index.html
+++ b/website/generated-content/documentation/transforms/java/other/create/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/other/flatten/index.html b/website/generated-content/documentation/transforms/java/other/flatten/index.html
index 13f7aea..ffbb766 100644
--- a/website/generated-content/documentation/transforms/java/other/flatten/index.html
+++ b/website/generated-content/documentation/transforms/java/other/flatten/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/other/passert/index.html b/website/generated-content/documentation/transforms/java/other/passert/index.html
index 84d3973..8f7fa18 100644
--- a/website/generated-content/documentation/transforms/java/other/passert/index.html
+++ b/website/generated-content/documentation/transforms/java/other/passert/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/other/view/index.html b/website/generated-content/documentation/transforms/java/other/view/index.html
index 4b59dda..67c5f69 100644
--- a/website/generated-content/documentation/transforms/java/other/view/index.html
+++ b/website/generated-content/documentation/transforms/java/other/view/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/other/window/index.html b/website/generated-content/documentation/transforms/java/other/window/index.html
index 92d4c64..1d5503f 100644
--- a/website/generated-content/documentation/transforms/java/other/window/index.html
+++ b/website/generated-content/documentation/transforms/java/other/window/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/java/overview/index.html b/website/generated-content/documentation/transforms/java/overview/index.html
index 95a815c..4488adb 100644
--- a/website/generated-content/documentation/transforms/java/overview/index.html
+++ b/website/generated-content/documentation/transforms/java/overview/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/approximatequantiles/index.html b/website/generated-content/documentation/transforms/python/aggregation/approximatequantiles/index.html
index 6fb21b8..3530b75 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/approximatequantiles/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/approximatequantiles/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/approximateunique/index.html b/website/generated-content/documentation/transforms/python/aggregation/approximateunique/index.html
index 9ee5019..b3840e4 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/approximateunique/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/approximateunique/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/cogroupbykey/index.html b/website/generated-content/documentation/transforms/python/aggregation/cogroupbykey/index.html
index 9ae0656..2e3879f 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/cogroupbykey/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/cogroupbykey/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/combineglobally/index.html b/website/generated-content/documentation/transforms/python/aggregation/combineglobally/index.html
index 0d8c11a..aee7e2e 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/combineglobally/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/combineglobally/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/combinewithcontext/index.html b/website/generated-content/documentation/transforms/python/aggregation/combinewithcontext/index.html
index e180f15..440bd96 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/combinewithcontext/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/combinewithcontext/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/count/index.html b/website/generated-content/documentation/transforms/python/aggregation/count/index.html
index 53db1dd..d662538 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/count/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/count/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/distinct/index.html b/website/generated-content/documentation/transforms/python/aggregation/distinct/index.html
index 96c90fe..04894be 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/distinct/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/distinct/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/groupbykey/index.html b/website/generated-content/documentation/transforms/python/aggregation/groupbykey/index.html
index 38617de..0a1d484 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/groupbykey/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/groupbykey/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/groupintobatches/index.html b/website/generated-content/documentation/transforms/python/aggregation/groupintobatches/index.html
index f77e11d..ec08457 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/groupintobatches/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/groupintobatches/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/latest/index.html b/website/generated-content/documentation/transforms/python/aggregation/latest/index.html
index f7d30d0..3eaf116 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/latest/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/latest/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/max/index.html b/website/generated-content/documentation/transforms/python/aggregation/max/index.html
index 55f2707..1fecf13 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/max/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/max/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/mean/index.html b/website/generated-content/documentation/transforms/python/aggregation/mean/index.html
index 1842ec1..cefbe47 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/mean/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/mean/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/min/index.html b/website/generated-content/documentation/transforms/python/aggregation/min/index.html
index dbc4415..db50845 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/min/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/min/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/sample/index.html b/website/generated-content/documentation/transforms/python/aggregation/sample/index.html
index 8aa3abe..a766ab5 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/sample/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/sample/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/sum/index.html b/website/generated-content/documentation/transforms/python/aggregation/sum/index.html
index 25f3534..d613e08 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/sum/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/sum/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/aggregation/top/index.html b/website/generated-content/documentation/transforms/python/aggregation/top/index.html
index fde64e2..0b1a1f5 100644
--- a/website/generated-content/documentation/transforms/python/aggregation/top/index.html
+++ b/website/generated-content/documentation/transforms/python/aggregation/top/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/filter/index.html b/website/generated-content/documentation/transforms/python/elementwise/filter/index.html
index 3b5eb48..428e908 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/filter/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/filter/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/flatmap/index.html b/website/generated-content/documentation/transforms/python/elementwise/flatmap/index.html
index 4bb1ea0..be0cde6 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/flatmap/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/flatmap/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/keys/index.html b/website/generated-content/documentation/transforms/python/elementwise/keys/index.html
index 37a45a7..3f3ef94 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/keys/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/keys/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/kvswap/index.html b/website/generated-content/documentation/transforms/python/elementwise/kvswap/index.html
index 0ad37bd..3c18c49 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/kvswap/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/kvswap/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/map/index.html b/website/generated-content/documentation/transforms/python/elementwise/map/index.html
index a0fa286..02cff59 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/map/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/map/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/pardo/index.html b/website/generated-content/documentation/transforms/python/elementwise/pardo/index.html
index 0047178..c284e01 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/pardo/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/pardo/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
@@ -635,7 +643,7 @@ window.max_timestamp() -> Timestamp(1584675689.999999) (2020-03-20 03:41:29.9
can be customized with a number of methods that can help create more complex behaviors.
You can customize what a worker does when it starts and shuts down with <code class="highlighter-rouge">setup</code> and <code class="highlighter-rouge">teardown</code>.
You can also customize what to do when a
-<a href="https://beam.apache.org/documentation/execution-model/#bundling-and-persistence"><em>bundle of elements</em></a>
+<a href="https://beam.apache.org/documentation/runtime/model/#bundling-and-persistence"><em>bundle of elements</em></a>
starts and finishes with <code class="highlighter-rouge">start_bundle</code> and <code class="highlighter-rouge">finish_bundle</code>.</p>
<ul>
diff --git a/website/generated-content/documentation/transforms/python/elementwise/partition/index.html b/website/generated-content/documentation/transforms/python/elementwise/partition/index.html
index af824b9..25aa77a 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/partition/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/partition/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/regex/index.html b/website/generated-content/documentation/transforms/python/elementwise/regex/index.html
index 66e6932..a0d0852 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/regex/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/regex/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/reify/index.html b/website/generated-content/documentation/transforms/python/elementwise/reify/index.html
index b74e892..42a874e 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/reify/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/reify/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/tostring/index.html b/website/generated-content/documentation/transforms/python/elementwise/tostring/index.html
index d371c06..69a153c 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/tostring/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/tostring/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/values/index.html b/website/generated-content/documentation/transforms/python/elementwise/values/index.html
index 0439e95..91228fc 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/values/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/values/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/withkeys/index.html b/website/generated-content/documentation/transforms/python/elementwise/withkeys/index.html
index c2a5067..07af500 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/withkeys/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/withkeys/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/elementwise/withtimestamps/index.html b/website/generated-content/documentation/transforms/python/elementwise/withtimestamps/index.html
index 5c53875..36ed26b 100644
--- a/website/generated-content/documentation/transforms/python/elementwise/withtimestamps/index.html
+++ b/website/generated-content/documentation/transforms/python/elementwise/withtimestamps/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/other/create/index.html b/website/generated-content/documentation/transforms/python/other/create/index.html
index 1cfd0b0..7b6afb6 100644
--- a/website/generated-content/documentation/transforms/python/other/create/index.html
+++ b/website/generated-content/documentation/transforms/python/other/create/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/other/flatten/index.html b/website/generated-content/documentation/transforms/python/other/flatten/index.html
index d7ce49d..2156207 100644
--- a/website/generated-content/documentation/transforms/python/other/flatten/index.html
+++ b/website/generated-content/documentation/transforms/python/other/flatten/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/other/passert/index.html b/website/generated-content/documentation/transforms/python/other/passert/index.html
index 8aa74fd..5cdaa08 100644
--- a/website/generated-content/documentation/transforms/python/other/passert/index.html
+++ b/website/generated-content/documentation/transforms/python/other/passert/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/other/reshuffle/index.html b/website/generated-content/documentation/transforms/python/other/reshuffle/index.html
index 8a064e8..8284c24 100644
--- a/website/generated-content/documentation/transforms/python/other/reshuffle/index.html
+++ b/website/generated-content/documentation/transforms/python/other/reshuffle/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/other/view/index.html b/website/generated-content/documentation/transforms/python/other/view/index.html
index f86b01f..04e0b8e 100644
--- a/website/generated-content/documentation/transforms/python/other/view/index.html
+++ b/website/generated-content/documentation/transforms/python/other/view/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/other/windowinto/index.html b/website/generated-content/documentation/transforms/python/other/windowinto/index.html
index d1a0eca..841bd2d 100644
--- a/website/generated-content/documentation/transforms/python/other/windowinto/index.html
+++ b/website/generated-content/documentation/transforms/python/other/windowinto/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/documentation/transforms/python/overview/index.html b/website/generated-content/documentation/transforms/python/overview/index.html
index 743c762..a6586ae 100644
--- a/website/generated-content/documentation/transforms/python/overview/index.html
+++ b/website/generated-content/documentation/transforms/python/overview/index.html
@@ -168,7 +168,6 @@
<li><span class="section-nav-list-main-title">Documentation</span></li>
<li><a href="/documentation">Using the Documentation</a></li>
-<li><a href="/documentation/execution-model">Beam Execution Model</a></li>
<li>
<span class="section-nav-list-title">Pipeline development lifecycle</span>
@@ -412,6 +411,15 @@
</li>
<li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Runtime systems</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/runtime/model/">Execution model</a></li>
+ <li><a href="/documentation/runtime/environments/">Runtime environments</a></li>
+ </ul>
+</li>
+
+<li class="section-nav-item--collapsible">
<span class="section-nav-list-title">Learning resources</span>
<ul class="section-nav-list">
diff --git a/website/generated-content/feed.xml b/website/generated-content/feed.xml
index 9879203..901993e 100644
--- a/website/generated-content/feed.xml
+++ b/website/generated-content/feed.xml
@@ -60,7 +60,7 @@ I wanted to explore Data Engineering, so for GSoC, I wanted to work on a project
Before actually submitting a proposal, I went through a bunch of resources to make sure I had a concrete understanding of Beam.
I read the <a href="https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101">Streaming 101</a> and <a href="https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102">Streaming 102</a> blogs by Tyler Akidau. They are the perfect introduction to Beam’s unified model for Batch and Streaming.
In addition, I watched all Beam talks on YouTube. You can find them on the <a href="https://beam.apache.org/documentation/resources/videos-and-podcasts/">Beam Website</a>.
-Beam has really good documentation. The <a href="https://beam.apache.org/documentation/programming-guide/">Programming Guide</a> lays out all of Beam’s concepts really well. <a href="https://beam.apache.org/documentation/execution-model/">Beam’s execution model</a> is also documented well and is a must-read to understand how Beam processes data.
+Beam has really good documentation. The <a href="https://beam.apache.org/documentation/programming-guide/">Programming Guide</a> lays out all of Beam’s concepts really well. <a href="https://beam.apache.org/documentation/runtime/model">Beam’s execution model</a> is also documented well and is a must-read to understand how Beam processes data.
<a href="https://www.waitingforcode.com/apache-beam">waitingforcode.com</a> also has good blog posts about Beam concepts.
To get a better sense of the Beam codebase, I played around with it and worked on some PRs to understand Beam better and got familiar with the test suite and workflows.</p>