You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2021/06/11 06:02:19 UTC
[beam] branch asf-site updated: Publishing website 2021/06/11
06:01:30 at commit f4430e6
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 4c9c273 Publishing website 2021/06/11 06:01:30 at commit f4430e6
4c9c273 is described below
commit 4c9c273b76ff908987f78bf5d90a638e49ba0afd
Author: jenkins <bu...@apache.org>
AuthorDate: Fri Jun 11 06:01:31 2021 +0000
Publishing website 2021/06/11 06:01:30 at commit f4430e6
---
.../documentation/basics/index.html | 18 +++++++---------
website/generated-content/documentation/index.xml | 24 ++++++----------------
website/generated-content/sitemap.xml | 2 +-
3 files changed, 14 insertions(+), 30 deletions(-)
diff --git a/website/generated-content/documentation/basics/index.html b/website/generated-content/documentation/basics/index.html
index ee848cb..a1d78b7 100644
--- a/website/generated-content/documentation/basics/index.html
+++ b/website/generated-content/documentation/basics/index.html
@@ -31,15 +31,11 @@ processing engine.</li></ul><p>These concepts may be very similar to your proces
Beam’s design is for cross-language operation and reusable libraries of
transforms, there are some special features worth highlighting.</p><h3 id=pipeline>Pipeline</h3><p>A pipeline in Beam is a graph of PTransforms operating on PCollections. A
pipeline is constructed by a user in their SDK of choice, and makes its way to
-your runner either via the SDK directly or via the Runner API’s (forthcoming)
-RPC interfaces.</p><h3 id=ptransforms>PTransforms</h3><p>In Beam, a PTransform can be one of the five primitives or it can be a
-composite transform encapsulating a subgraph. The primitives are:</p><ul><li><a href=#implementing-the-read-primitive><em>Read</em></a> - parallel connectors to external
-systems</li><li><a href=#implementing-the-pardo-primitive><em>ParDo</em></a> - per element processing</li><li><a href=#implementing-the-groupbykey-and-window-primitive><em>GroupByKey</em></a> -
-aggregating elements per key and window</li><li><a href=#implementing-the-flatten-primitive><em>Flatten</em></a> - union of PCollections</li><li><a href=#implementing-the-window-primitive><em>Window</em></a> - set the windowing strategy
-for a PCollection</li></ul><p>When implementing a runner, these are the operations you need to implement.
-Composite transforms may or may not be important to your runner. If you expose
-a UI, maintaining some of the composite structure will make the pipeline easier
-for a user to understand. But the result of processing is not changed.</p><h3 id=pcollections>PCollections</h3><p>A PCollection is an unordered bag of elements. Your runner will be responsible
+your runner either via the SDK directly or via the Runner API’s
+RPC interfaces.</p><h3 id=ptransforms>PTransforms</h3><p>A <code>PTransform</code> represents a data processing operation, or a step,
+in your pipeline. A <code>PTransform</code> can be applied to one or more
+<code>PCollection</code> objects as input which performs some processing on the elements of that
+<code>PCollection</code> and produces zero or more output <code>PCollection</code> objects.</p><h3 id=pcollections>PCollections</h3><p>A PCollection is an unordered bag of elements. Your runner will be responsible
for storing these elements. There are some major aspects of a PCollection to
note:</p><h4 id=bounded-vs-unbounded>Bounded vs Unbounded</h4><p>A PCollection may be bounded or unbounded.</p><ul><li><em>Bounded</em> - it is finite and you know it, as in batch use cases</li><li><em>Unbounded</em> - it may be never end, you don’t know, as in streaming use cases</li></ul><p>These derive from the intuitions of batch and stream processing, but the two
are unified in Beam and bounded and unbounded PCollections can coexist in the
@@ -87,7 +83,7 @@ in side inputs)</li><li><em>WindowMappingFn</em> - maps one element’s wind
bounds on how far in the past the result window will be (used in side
inputs)</li><li><em>CombineFn</em> - associative and commutative aggregation (used in Combine and
state)</li><li><em>Coder</em> - encodes user data; some coders have standard formats and are not really UDFs</li></ul><p>The various types of user-defined functions will be described further alongside
-the primitives that use them.</p><h3 id=runner>Runner</h3><p>The term “runner” is used for a couple of things. It generally refers to the
+the <a href=#ptransforms><em>PTransforms</em></a> that use them.</p><h3 id=runner>Runner</h3><p>The term “runner” is used for a couple of things. It generally refers to the
software that takes a Beam pipeline and executes it somehow. Often, this is the
translation code that you write. It usually also includes some customized
operators for your data processing engine, and is sometimes used to refer to
@@ -96,7 +92,7 @@ use code font for proper nouns in our APIs, whether or not the identifiers
match across all SDKs.</p><p>The <code>run(Pipeline)</code> method should be asynchronous and results in a
PipelineResult which generally will be a job descriptor for your data
processing engine, providing methods for checking its status, canceling it, and
-waiting for it to terminate.</p><div class=feedback><p class=update>Last updated on 2021/02/05</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:dev@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=" [...]
+waiting for it to terminate.</p><div class=feedback><p class=update>Last updated on 2021/06/03</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:dev@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class=" [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div></div></div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml
index 03f52d5..8c869ad 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -3204,25 +3204,13 @@ transforms, there are some special features worth highlighting.</p>
<h3 id="pipeline">Pipeline</h3>
<p>A pipeline in Beam is a graph of PTransforms operating on PCollections. A
pipeline is constructed by a user in their SDK of choice, and makes its way to
-your runner either via the SDK directly or via the Runner API&rsquo;s (forthcoming)
+your runner either via the SDK directly or via the Runner API&rsquo;s
RPC interfaces.</p>
<h3 id="ptransforms">PTransforms</h3>
-<p>In Beam, a PTransform can be one of the five primitives or it can be a
-composite transform encapsulating a subgraph. The primitives are:</p>
-<ul>
-<li><a href="#implementing-the-read-primitive"><em>Read</em></a> - parallel connectors to external
-systems</li>
-<li><a href="#implementing-the-pardo-primitive"><em>ParDo</em></a> - per element processing</li>
-<li><a href="#implementing-the-groupbykey-and-window-primitive"><em>GroupByKey</em></a> -
-aggregating elements per key and window</li>
-<li><a href="#implementing-the-flatten-primitive"><em>Flatten</em></a> - union of PCollections</li>
-<li><a href="#implementing-the-window-primitive"><em>Window</em></a> - set the windowing strategy
-for a PCollection</li>
-</ul>
-<p>When implementing a runner, these are the operations you need to implement.
-Composite transforms may or may not be important to your runner. If you expose
-a UI, maintaining some of the composite structure will make the pipeline easier
-for a user to understand. But the result of processing is not changed.</p>
+<p>A <code>PTransform</code> represents a data processing operation, or a step,
+in your pipeline. A <code>PTransform</code> can be applied to one or more
+<code>PCollection</code> objects as input which performs some processing on the elements of that
+<code>PCollection</code> and produces zero or more output <code>PCollection</code> objects.</p>
<h3 id="pcollections">PCollections</h3>
<p>A PCollection is an unordered bag of elements. Your runner will be responsible
for storing these elements. There are some major aspects of a PCollection to
@@ -3309,7 +3297,7 @@ state)</li>
<li><em>Coder</em> - encodes user data; some coders have standard formats and are not really UDFs</li>
</ul>
<p>The various types of user-defined functions will be described further alongside
-the primitives that use them.</p>
+the <a href="#ptransforms"><em>PTransforms</em></a> that use them.</p>
<h3 id="runner">Runner</h3>
<p>The term &ldquo;runner&rdquo; is used for a couple of things. It generally refers to the
software that takes a Beam pipeline and executes it somehow. Often, this is the
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index 0131fdd..6d3818b 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.30.0/</loc><lastmod>2021-05-26T19:50:36-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/blog/v [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.30.0/</loc><lastmod>2021-05-26T19:50:36-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-06-09T10:56:20-07:00</lastmod></url><url><loc>/blog/v [...]
\ No newline at end of file