You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2021/12/07 06:04:17 UTC
[beam] branch asf-site updated: Publishing website 2021/12/07 06:03:36 at commit 6d63a70
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new a89b07d Publishing website 2021/12/07 06:03:36 at commit 6d63a70
a89b07d is described below
commit a89b07df119671d4dcb216913f33723d82798d96
Author: jenkins <bu...@apache.org>
AuthorDate: Tue Dec 7 06:03:37 2021 +0000
Publishing website 2021/12/07 06:03:36 at commit 6d63a70
---
.../documentation/basics/index.html | 71 +++++++++++---
website/generated-content/documentation/index.xml | 107 ++++++++++++++++++---
.../documentation/programming-guide/index.html | 11 ++-
website/generated-content/sitemap.xml | 2 +-
4 files changed, 157 insertions(+), 34 deletions(-)
diff --git a/website/generated-content/documentation/basics/index.html b/website/generated-content/documentation/basics/index.html
index 89478b1..d74214d 100644
--- a/website/generated-content/documentation/basics/index.html
+++ b/website/generated-content/documentation/basics/index.html
@@ -18,7 +18,7 @@
function addPlaceholder(){$('input:text').attr('placeholder',"What are you looking for?");}
function endSearch(){var search=document.querySelector(".searchBar");search.classList.add("disappear");var icons=document.querySelector("#iconsBar");icons.classList.remove("disappear");}
function blockScroll(){$("body").toggleClass("fixedPosition");}
-function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Documentation</span></li><li><a href=/documentation>Using the Documentation</a></li><li class=section-nav-item--collapsible><span class=section-nav-lis [...]
+function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Documentation</span></li><li><a href=/documentation>Using the Documentation</a></li><li class=section-nav-item--collapsible><span class=section-nav-lis [...]
data-parallel processing pipelines. To get started with Beam, you’ll need to
understand an important set of core concepts:</p><ul><li><a href=#pipeline><em>Pipeline</em></a> - A pipeline is a user-constructed graph of
transformations that defines the desired data processing operations.</li><li><a href=#pcollection><em>PCollection</em></a> - A <code>PCollection</code> is a data set or data
@@ -33,7 +33,13 @@ a <code>PCollection</code>. The schema for a <code>PCollection</code> defines el
<code>PCollection</code> as an ordered list of named fields.</li><li><a href=/documentation/sdks/java/><em>SDK</em></a> - A language-specific library that lets
pipeline authors build transforms, construct their pipelines, and submit
them to a runner.</li><li><a href=#runner><em>Runner</em></a> - A runner runs a Beam pipeline using the capabilities of
-your chosen data processing engine.</li><li><a href=#trigger><em>Trigger</em></a> - A trigger determines when to aggregate the results of
+your chosen data processing engine.</li><li><a href=#window><em>Window</em></a> - A <code>PCollection</code> can be subdivided into windows based on
+the timestamps of the individual elements. Windows enable grouping operations
+over collections that grow over time by dividing the collection into windows
+of finite collections.</li><li><a href=#watermark><em>Watermark</em></a> - A watermark is a guess as to when all data in a
+certain window is expected to have arrived. This is needed because data isn’t
+always guaranteed to arrive in a pipeline in time order, or to always arrive
+at predictable intervals.</li><li><a href=#trigger><em>Trigger</em></a> - A trigger determines when to aggregate the results of
each window.</li><li><a href=#state-and-timers><em>State and timers</em></a> - Per-key state and timer callbacks
are lower level primitives that give you full control over aggregating input
collections that grow over time.</li><li><a href=#splittable-dofn><em>Splittable DoFn</em></a> - Splittable DoFns let you process
@@ -95,18 +101,17 @@ responsible for providing initial timestamps. The runner must propagate and
aggregate timestamps. If the timestamp is not important, such as with certain
batch processing jobs where elements do not denote events, the timestamp will be
the minimum representable timestamp, often referred to colloquially as “negative
-infinity”.</p><h4 id=watermarks>Watermarks</h4><p>Every <code>PCollection</code> must have a watermark that estimates how complete the
-<code>PCollection</code> is.</p><p>The watermark is a guess that “we’ll never see an element with an earlier
+infinity”.</p><h4 id=watermarks>Watermarks</h4><p>Every <code>PCollection</code> must have a <a href=#watermark>watermark</a> that estimates how
+complete the <code>PCollection</code> is.</p><p>The watermark is a guess that “we’ll never see an element with an earlier
timestamp”. Data sources are responsible for producing a watermark. The runner
must implement watermark propagation as PCollections are processed, merged, and
partitioned.</p><p>The contents of a <code>PCollection</code> are complete when a watermark advances to
“infinity”. In this manner, you can discover that an unbounded PCollection is
-finite.</p><h4 id=windowed-elements>Windowed elements</h4><p>Every element in a <code>PCollection</code> resides in a window. No element resides in
-multiple windows; two elements can be equal except for their window, but they
-are not the same.</p><p>When elements are read from the outside world, they arrive in the global window.
-When they are written to the outside world, they are effectively placed back
+finite.</p><h4 id=windowed-elements>Windowed elements</h4><p>Every element in a <code>PCollection</code> resides in a <a href=#window>window</a>. No element
+resides in multiple windows; two elements can be equal except for their window,
+but they are not the same.</p><p>When elements are written to the outside world, they are effectively placed back
into the global window. Transforms that write data and don’t take this
-perspective probably risks data loss.</p><p>A window has a maximum timestamp. When the watermark exceeds the maximum
+perspective risk data loss.</p><p>A window has a maximum timestamp. When the watermark exceeds the maximum
timestamp plus the user-specified allowed lateness, the window is expired. All
data related to an expired window might be discarded at any time.</p><h4 id=coder>Coder</h4><p>Every <code>PCollection</code> has a coder, which is a specification of the binary format
of the elements.</p><p>In Beam, the user’s pipeline can be written in a language other than the
@@ -150,8 +155,8 @@ the transform. For example, when using <code>ParDo</code>, user-defined code spe
operation to apply to every element. For <code>Combine</code>, it specifies how values
should be combined. By using <a href=/documentation/patterns/cross-language/>cross-language transforms</a>,
a Beam pipeline can contain UDFs written in a different language, or even
-multiple languages in the same pipeline.</p><p>Beam has several varieties of UDFs:</p><ul><li><a href=/programming-guide/#pardo><em>DoFn</em></a> - per-element processing function (used
-in <code>ParDo</code>)</li><li><a href=/programming-guide/#setting-your-pcollections-windowing-function><em>WindowFn</em></a> -
+multiple languages in the same pipeline.</p><p>Beam has several varieties of UDFs:</p><ul><li><a href=/documentation/programming-guide/#pardo><em>DoFn</em></a> - per-element processing
+function (used in <code>ParDo</code>)</li><li><a href=/documentation/programming-guide/#setting-your-pcollections-windowing-function><em>WindowFn</em></a> -
places elements in windows and merges windows (used in <code>Window</code> and
<code>GroupByKey</code>)</li><li><a href=/documentation/programming-guide/#side-inputs><em>ViewFn</em></a> - adapts a
materialized <code>PCollection</code> to a particular interface (used in side inputs)</li><li><a href=/documentation/programming-guide/#side-inputs-windowing><em>WindowMappingFn</em></a> -
@@ -167,7 +172,7 @@ without communicating or sharing state with any of the other copies. Each copy
of your user code function might be retried or run multiple times, depending on
the pipeline runner and the processing backend that you choose for your
pipeline. Beam also supports stateful processing through the
-<a href=/blog/stateful-processing/>stateful processing API</a>.</p><p>For more information about user-defined functions, see the following pages:</p><ul><li><a href=/documentation/programming-guide/#requirements-for-writing-user-code-for-beam-transforms>Requirements for writing user code for Beam transforms</a></li><li><a href=/documentation/programming-guide/#pardo>Beam Programming Guide: ParDo</a></li><li><a href=/programming-guide/#setting-your-pcollections-windowing-function>Beam Pro [...]
+<a href=/blog/stateful-processing/>stateful processing API</a>.</p><p>For more information about user-defined functions, see the following pages:</p><ul><li><a href=/documentation/programming-guide/#requirements-for-writing-user-code-for-beam-transforms>Requirements for writing user code for Beam transforms</a></li><li><a href=/documentation/programming-guide/#pardo>Beam Programming Guide: ParDo</a></li><li><a href=/documentation/programming-guide/#setting-your-pcollections-windowing-fun [...]
schema for a <code>PCollection</code> defines elements of that <code>PCollection</code> as an ordered
list of named fields. Each field has a name, a type, and possibly a set of user
options.</p><p>In many cases, the element type in a <code>PCollection</code> has a structure that can be
@@ -188,7 +193,45 @@ Flink runner translates a Beam pipeline into a Flink job. The Direct Runner runs
pipelines locally so you can test, debug, and validate that your pipeline
adheres to the Apache Beam model as closely as possible.</p><p>For an up-to-date list of Beam runners and which features of the Apache Beam
model they support, see the runner
-<a href=/documentation/runners/capability-matrix/>capability matrix</a>.</p><p>For more information about runners, see the following pages:</p><ul><li><a href=/documentation/#choosing-a-runner>Choosing a Runner</a></li><li><a href=/documentation/runners/capability-matrix/>Beam Capability Matrix</a></li></ul><h3 id=trigger>Trigger</h3><p>When collecting and grouping data into windows, Beam uses <em>triggers</em> to
+<a href=/documentation/runners/capability-matrix/>capability matrix</a>.</p><p>For more information about runners, see the following pages:</p><ul><li><a href=/documentation/#choosing-a-runner>Choosing a Runner</a></li><li><a href=/documentation/runners/capability-matrix/>Beam Capability Matrix</a></li></ul><h3 id=window>Window</h3><p>Windowing subdivides a <code>PCollection</code> into <em>windows</em> according to the timestamps
+of its individual elements. Windows enable grouping operations over unbounded
+collections by dividing the collection into windows of finite collections.</p><p>A <em>windowing function</em> tells the runner how to assign elements to one or more
+initial windows, and how to merge windows of grouped elements. Each element in a
+<code>PCollection</code> can only be in one window, so if a windowing function specifies
+multiple windows for an element, the element is conceptually duplicated into
+each of the windows and each element is identical except for its window.</p><p>Transforms that aggregate multiple elements, such as <code>GroupByKey</code> and <code>Combine</code>,
+work implicitly on a per-window basis; they process each <code>PCollection</code> as a
+succession of multiple, finite windows, though the entire collection itself may
+be of unbounded size.</p><p>Beam provides several windowing functions:</p><ul><li><strong>Fixed time windows</strong> (also known as “tumbling windows”) represent a consistent
+duration, non-overlapping time interval in the data stream.</li><li><strong>Sliding time windows</strong> (also known as “hopping windows”) also represent time
+intervals in the data stream; however, sliding time windows can overlap.</li><li><strong>Per-session windows</strong> define windows that contain elements that are within a
+certain gap duration of another element.</li><li><strong>Single global window</strong>: by default, all data in a <code>PCollection</code> is assigned to
+the single global window, and late data is discarded.</li><li><strong>Calendar-based windows</strong> (not supported by the Beam SDK for Python)</li></ul><p>You can also define your own windowing function if you have more complex
+requirements.</p><p>For example, let’s say we have a <code>PCollection</code> that uses fixed-time windowing,
+with windows that are five minutes long. For each window, Beam must collect all
+the data with an event time timestamp in the given window range (between 0:00
+and 4:59 in the first window, for instance). Data with timestamps outside that
+range (data from 5:00 or later) belongs to a different window.</p><p>Two concepts are closely related to windowing and covered in the following
+sections: <a href=#watermark>watermarks</a> and <a href=#trigger>triggers</a>.</p><p>For more information about windows, see the following page:</p><ul><li><a href=/documentation/programming-guide/#windowing>Beam Programming Guide: Windowing</a></li><li><a href=/documentation/programming-guide/#setting-your-pcollections-windowing-function>Beam Programming Guide: WindowFn</a></li></ul><h3 id=watermark>Watermark</h3><p>In any data processing system, there is a certain amount of lag between [...]
+a data event occurs (the “event time”, determined by the timestamp on the data
+element itself) and the time the actual data element gets processed at any stage
+in your pipeline (the “processing time”, determined by the clock on the system
+processing the element). In addition, data isn’t always guaranteed to arrive in
+a pipeline in time order, or to always arrive at predictable intervals. For
+example, you might have intermediate systems that don’t preserve order, or you
+might have two servers that timestamp data but one has a better network
+connection.</p><p>To address this potential unpredictability, Beam tracks a <em>watermark</em>. A
+watermark is a guess as to when all data in a certain window is expected to have
+arrived in the pipeline. You can also think of this as “we’ll never see an
+element with an earlier timestamp”.</p><p>Data sources are responsible for producing a watermark, and every <code>PCollection</code>
+must have a watermark that estimates how complete the <code>PCollection</code> is. The
+contents of a <code>PCollection</code> are complete when a watermark advances to
+“infinity”. In this manner, you might discover that an unbounded <code>PCollection</code>
+is finite. After the watermark progresses past the end of a window, any further
+element that arrives with a timestamp in that window is considered <em>late data</em>.</p><p>Triggers are a related concept that allow you to modify and refine the windowing
+strategy for a <code>PCollection</code>. You can use triggers to decide when each
+individual window aggregates and reports its results, including how the window
+emits late elements.</p><p>For more information about watermarks, see the following page:</p><ul><li><a href=/documentation/programming-guide/#watermarks-and-late-data>Beam Programming Guide: Watermarks and late data</a></li></ul><h3 id=trigger>Trigger</h3><p>When collecting and grouping data into windows, Beam uses <em>triggers</em> to
determine when to emit the aggregated results of each window (referred to as a
<em>pane</em>). If you use Beam’s default windowing configuration and default trigger,
Beam outputs the aggregated result when it estimates all data has arrived, and
@@ -259,7 +302,7 @@ checkpoint the sub-element and the runner repeats step 2.</li></ol><p>You can al
processing. For example, if you write a splittable <code>DoFn</code> to watch a set of
directories and output filenames as they arrive, you can split to subdivide the
work of different directories. This allows the runner to split off a hot
-directory and give it additional resources.</p><p>For more information about Splittable <code>DoFn</code>, see the following pages:</p><ul><li><a href=/documentation/programming-guide/#splittable-dofns>Splittable DoFns</a></li><li><a href=/blog/splittable-do-fn-is-available/>Splittable DoFn in Apache Beam is Ready to Use</a></li></ul><div class=feedback><p class=update>Last updated on 2021/10/21</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all us [...]
+directory and give it additional resources.</p><p>For more information about Splittable <code>DoFn</code>, see the following pages:</p><ul><li><a href=/documentation/programming-guide/#splittable-dofns>Splittable DoFns</a></li><li><a href=/blog/splittable-do-fn-is-available/>Splittable DoFn in Apache Beam is Ready to Use</a></li></ul><div class=feedback><p class=update>Last updated on 2021/12/06</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all us [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div></div></div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml
index fc279a0..53c27a4 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -3205,6 +3205,14 @@ pipeline authors build transforms, construct their pipelines, and submit
them to a runner.</li>
<li><a href="#runner"><em>Runner</em></a> - A runner runs a Beam pipeline using the capabilities of
your chosen data processing engine.</li>
+<li><a href="#window"><em>Window</em></a> - A <code>PCollection</code> can be subdivided into windows based on
+the timestamps of the individual elements. Windows enable grouping operations
+over collections that grow over time by dividing the collection into windows
+of finite collections.</li>
+<li><a href="#watermark"><em>Watermark</em></a> - A watermark is a guess as to when all data in a
+certain window is expected to have arrived. This is needed because data isn’t
+always guaranteed to arrive in a pipeline in time order, or to always arrive
+at predictable intervals.</li>
<li><a href="#trigger"><em>Trigger</em></a> - A trigger determines when to aggregate the results of
each window.</li>
<li><a href="#state-and-timers"><em>State and timers</em></a> - Per-key state and timer callbacks
@@ -3316,8 +3324,8 @@ batch processing jobs where elements do not denote events, the timestamp will be
the minimum representable timestamp, often referred to colloquially as &ldquo;negative
infinity&rdquo;.</p>
<h4 id="watermarks">Watermarks</h4>
-<p>Every <code>PCollection</code> must have a watermark that estimates how complete the
-<code>PCollection</code> is.</p>
+<p>Every <code>PCollection</code> must have a <a href="#watermark">watermark</a> that estimates how
+complete the <code>PCollection</code> is.</p>
<p>The watermark is a guess that &ldquo;we&rsquo;ll never see an element with an earlier
timestamp&rdquo;. Data sources are responsible for producing a watermark. The runner
must implement watermark propagation as PCollections are processed, merged, and
@@ -3326,13 +3334,12 @@ partitioned.</p>
&ldquo;infinity&rdquo;. In this manner, you can discover that an unbounded PCollection is
finite.</p>
<h4 id="windowed-elements">Windowed elements</h4>
-<p>Every element in a <code>PCollection</code> resides in a window. No element resides in
-multiple windows; two elements can be equal except for their window, but they
-are not the same.</p>
-<p>When elements are read from the outside world, they arrive in the global window.
-When they are written to the outside world, they are effectively placed back
+<p>Every element in a <code>PCollection</code> resides in a <a href="#window">window</a>. No element
+resides in multiple windows; two elements can be equal except for their window,
+but they are not the same.</p>
+<p>When elements are written to the outside world, they are effectively placed back
into the global window. Transforms that write data and don&rsquo;t take this
-perspective probably risks data loss.</p>
+perspective risk data loss.</p>
<p>A window has a maximum timestamp. When the watermark exceeds the maximum
timestamp plus the user-specified allowed lateness, the window is expired. All
data related to an expired window might be discarded at any time.</p>
@@ -3410,9 +3417,9 @@ a Beam pipeline can contain UDFs written in a different language, or even
multiple languages in the same pipeline.</p>
<p>Beam has several varieties of UDFs:</p>
<ul>
-<li><a href="/programming-guide/#pardo"><em>DoFn</em></a> - per-element processing function (used
-in <code>ParDo</code>)</li>
-<li><a href="/programming-guide/#setting-your-pcollections-windowing-function"><em>WindowFn</em></a> -
+<li><a href="/documentation/programming-guide/#pardo"><em>DoFn</em></a> - per-element processing
+function (used in <code>ParDo</code>)</li>
+<li><a href="/documentation/programming-guide/#setting-your-pcollections-windowing-function"><em>WindowFn</em></a> -
places elements in windows and merges windows (used in <code>Window</code> and
<code>GroupByKey</code>)</li>
<li><a href="/documentation/programming-guide/#side-inputs"><em>ViewFn</em></a> - adapts a
@@ -3439,7 +3446,7 @@ pipeline. Beam also supports stateful processing through the
<ul>
<li><a href="/documentation/programming-guide/#requirements-for-writing-user-code-for-beam-transforms">Requirements for writing user code for Beam transforms</a></li>
<li><a href="/documentation/programming-guide/#pardo">Beam Programming Guide: ParDo</a></li>
-<li><a href="/programming-guide/#setting-your-pcollections-windowing-function">Beam Programming Guide: WindowFn</a></li>
+<li><a href="/documentation/programming-guide/#setting-your-pcollections-windowing-function">Beam Programming Guide: WindowFn</a></li>
<li><a href="/documentation/programming-guide/#combine">Beam Programming Guide: CombineFn</a></li>
<li><a href="/documentation/programming-guide/#data-encoding-and-type-safety">Beam Programming Guide: Coder</a></li>
<li><a href="/documentation/programming-guide/#side-inputs">Beam Programming Guide: Side inputs</a></li>
@@ -3482,6 +3489,73 @@ model they support, see the runner
<li><a href="/documentation/#choosing-a-runner">Choosing a Runner</a></li>
<li><a href="/documentation/runners/capability-matrix/">Beam Capability Matrix</a></li>
</ul>
+<h3 id="window">Window</h3>
+<p>Windowing subdivides a <code>PCollection</code> into <em>windows</em> according to the timestamps
+of its individual elements. Windows enable grouping operations over unbounded
+collections by dividing the collection into windows of finite collections.</p>
+<p>A <em>windowing function</em> tells the runner how to assign elements to one or more
+initial windows, and how to merge windows of grouped elements. Each element in a
+<code>PCollection</code> can only be in one window, so if a windowing function specifies
+multiple windows for an element, the element is conceptually duplicated into
+each of the windows and each element is identical except for its window.</p>
+<p>Transforms that aggregate multiple elements, such as <code>GroupByKey</code> and <code>Combine</code>,
+work implicitly on a per-window basis; they process each <code>PCollection</code> as a
+succession of multiple, finite windows, though the entire collection itself may
+be of unbounded size.</p>
+<p>Beam provides several windowing functions:</p>
+<ul>
+<li><strong>Fixed time windows</strong> (also known as &ldquo;tumbling windows&rdquo;) represent a consistent
+duration, non-overlapping time interval in the data stream.</li>
+<li><strong>Sliding time windows</strong> (also known as &ldquo;hopping windows&rdquo;) also represent time
+intervals in the data stream; however, sliding time windows can overlap.</li>
+<li><strong>Per-session windows</strong> define windows that contain elements that are within a
+certain gap duration of another element.</li>
+<li><strong>Single global window</strong>: by default, all data in a <code>PCollection</code> is assigned to
+the single global window, and late data is discarded.</li>
+<li><strong>Calendar-based windows</strong> (not supported by the Beam SDK for Python)</li>
+</ul>
+<p>You can also define your own windowing function if you have more complex
+requirements.</p>
+<p>For example, let&rsquo;s say we have a <code>PCollection</code> that uses fixed-time windowing,
+with windows that are five minutes long. For each window, Beam must collect all
+the data with an event time timestamp in the given window range (between 0:00
+and 4:59 in the first window, for instance). Data with timestamps outside that
+range (data from 5:00 or later) belongs to a different window.</p>
+<p>Two concepts are closely related to windowing and covered in the following
+sections: <a href="#watermark">watermarks</a> and <a href="#trigger">triggers</a>.</p>
+<p>For more information about windows, see the following page:</p>
+<ul>
+<li><a href="/documentation/programming-guide/#windowing">Beam Programming Guide: Windowing</a></li>
+<li><a href="/documentation/programming-guide/#setting-your-pcollections-windowing-function">Beam Programming Guide: WindowFn</a></li>
+</ul>
+<h3 id="watermark">Watermark</h3>
+<p>In any data processing system, there is a certain amount of lag between the time
+a data event occurs (the “event time”, determined by the timestamp on the data
+element itself) and the time the actual data element gets processed at any stage
+in your pipeline (the “processing time”, determined by the clock on the system
+processing the element). In addition, data isn’t always guaranteed to arrive in
+a pipeline in time order, or to always arrive at predictable intervals. For
+example, you might have intermediate systems that don&rsquo;t preserve order, or you
+might have two servers that timestamp data but one has a better network
+connection.</p>
+<p>To address this potential unpredictability, Beam tracks a <em>watermark</em>. A
+watermark is a guess as to when all data in a certain window is expected to have
+arrived in the pipeline. You can also think of this as “we’ll never see an
+element with an earlier timestamp”.</p>
+<p>Data sources are responsible for producing a watermark, and every <code>PCollection</code>
+must have a watermark that estimates how complete the <code>PCollection</code> is. The
+contents of a <code>PCollection</code> are complete when a watermark advances to
+“infinity”. In this manner, you might discover that an unbounded <code>PCollection</code>
+is finite. After the watermark progresses past the end of a window, any further
+element that arrives with a timestamp in that window is considered <em>late data</em>.</p>
+<p>Triggers are a related concept that allow you to modify and refine the windowing
+strategy for a <code>PCollection</code>. You can use triggers to decide when each
+individual window aggregates and reports its results, including how the window
+emits late elements.</p>
+<p>For more information about watermarks, see the following page:</p>
+<ul>
+<li><a href="/documentation/programming-guide/#watermarks-and-late-data">Beam Programming Guide: Watermarks and late data</a></li>
+</ul>
<h3 id="trigger">Trigger</h3>
<p>When collecting and grouping data into windows, Beam uses <em>triggers</em> to
determine when to emit the aggregated results of each window (referred to as a
@@ -8893,9 +8967,12 @@ window.</p>
</ul>
<p>You can also define your own <code>WindowFn</code> if you have a more complex need.</p>
<p>Note that each element can logically belong to more than one window, depending
-on the windowing function you use. Sliding time windowing, for example, creates
-overlapping windows wherein a single element can be assigned to multiple
-windows.</p>
+on the windowing function you use. Sliding time windowing, for example, can
+create overlapping windows wherein a single element can be assigned to multiple
+windows. However, each element in a <code>PCollection</code> can only be in one window, so
+if an element is assigned to multiple windows, the element is conceptually
+duplicated into each of the windows and each element is identical except for its
+window.</p>
<h4 id="fixed-time-windows">8.2.1. Fixed time windows</h4>
<p>The simplest form of windowing is using <strong>fixed time windows</strong>: given a
timestamped <code>PCollection</code> which might be continuously updating, each window
diff --git a/website/generated-content/documentation/programming-guide/index.html b/website/generated-content/documentation/programming-guide/index.html
index 1ef8286..82acce2 100644
--- a/website/generated-content/documentation/programming-guide/index.html
+++ b/website/generated-content/documentation/programming-guide/index.html
@@ -2558,9 +2558,12 @@ for that <code>PCollection</code>. The <code>GroupByKey</code> transform groups
subsequent <code>ParDo</code> transform gets applied multiple times per key, once for each
window.</p><h3 id=provided-windowing-functions>8.2. Provided windowing functions</h3><p>You can define different kinds of windows to divide the elements of your
<code>PCollection</code>. Beam provides several windowing functions, including:</p><ul><li>Fixed Time Windows</li><li>Sliding Time Windows</li><li>Per-Session Windows</li><li>Single Global Window</li><li>Calendar-based Windows (not supported by the Beam SDK for Python or Go)</li></ul><p>You can also define your own <code>WindowFn</code> if you have a more complex need.</p><p>Note that each element can logically belong to more than one window, depending
-on the windowing function you use. Sliding time windowing, for example, creates
-overlapping windows wherein a single element can be assigned to multiple
-windows.</p><h4 id=fixed-time-windows>8.2.1. Fixed time windows</h4><p>The simplest form of windowing is using <strong>fixed time windows</strong>: given a
+on the windowing function you use. Sliding time windowing, for example, can
+create overlapping windows wherein a single element can be assigned to multiple
+windows. However, each element in a <code>PCollection</code> can only be in one window, so
+if an element is assigned to multiple windows, the element is conceptually
+duplicated into each of the windows and each element is identical except for its
+window.</p><h4 id=fixed-time-windows>8.2.1. Fixed time windows</h4><p>The simplest form of windowing is using <strong>fixed time windows</strong>: given a
timestamped <code>PCollection</code> which might be continuously updating, each window
might capture (for example) all elements with timestamps that fall into a 30
second interval.</p><p>A fixed time window represents a consistent duration, non overlapping time
@@ -4307,7 +4310,7 @@ expansionAddr := "localhost:8097"
outT := beam.UnnamedOutput(typex.New(reflectx.String))
res := beam.CrossLanguage(s, urn, payload, expansionAddr, beam.UnnamedInput(inputPCol), outT)
</code></pre></div></div></li><li><p>After the job has been submitted to the Beam runner, shutdown the expansion service by
-terminating the expansion service process.</p></li></ol><h3 id=x-lang-transform-runner-support>13.3. Runner Support</h3><p>Currently, portable runners such as Flink, Spark, and the Direct runner can be used with multi-language pipelines.</p><p>Google Cloud Dataflow supports multi-language pipelines through the Dataflow Runner v2 backend architecture.</p><div class=feedback><p class=update>Last updated on 2021/11/18</p><h3>Have you found everything you were looking for?</h3><p class=descr [...]
+terminating the expansion service process.</p></li></ol><h3 id=x-lang-transform-runner-support>13.3. Runner Support</h3><p>Currently, portable runners such as Flink, Spark, and the Direct runner can be used with multi-language pipelines.</p><p>Google Cloud Dataflow supports multi-language pipelines through the Dataflow Runner v2 backend architecture.</p><div class=feedback><p class=update>Last updated on 2021/12/06</p><h3>Have you found everything you were looking for?</h3><p class=descr [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div></div></div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index 78bbfff..5047cde 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.34.0/</loc><lastmod>2021-11-11T11:07:06-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-11-11T11:07:06-08:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-11-11T11:07:06-08:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-12-01T21:32:04+03:00</lastmod></url><url><loc>/blog/g [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.34.0/</loc><lastmod>2021-11-11T11:07:06-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-11-11T11:07:06-08:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-11-11T11:07:06-08:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-12-01T21:32:04+03:00</lastmod></url><url><loc>/blog/g [...]
\ No newline at end of file