You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2021/04/02 00:01:59 UTC
[beam] branch asf-site updated: Publishing website 2021/04/02
00:01:30 at commit 5ddb69f
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new c73b953 Publishing website 2021/04/02 00:01:30 at commit 5ddb69f
c73b953 is described below
commit c73b953bdd71804ae24e2563f1864558a3e6bcd3
Author: jenkins <bu...@apache.org>
AuthorDate: Fri Apr 2 00:01:31 2021 +0000
Publishing website 2021/04/02 00:01:30 at commit 5ddb69f
---
website/generated-content/blog/test-stream/index.html | 12 ++++++------
website/generated-content/sitemap.xml | 2 +-
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/website/generated-content/blog/test-stream/index.html b/website/generated-content/blog/test-stream/index.html
index 47b8e27..0c715a7 100644
--- a/website/generated-content/blog/test-stream/index.html
+++ b/website/generated-content/blog/test-stream/index.html
@@ -86,7 +86,7 @@ receives them progresses as the graph goes upwards. The watermark is represented
by the squiggly red line, and each starburst is the firing of a trigger and the
associated pane.</p><img class=center-block src=/images/blog/test-stream/elements-all-on-time.png alt="Elements on the Event and Processing time axes, with the Watermark and produced panes" width=442><h3 id=everything-arrives-on-time>Everything arrives on-time</h3><p>For example, if we create a TestStream where all the data arrives before the
watermark and provide the result PCollection as input to the CalculateTeamScores
-PTransform:</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>TestStream</span><span class=o><</span><span class=n>GameActionInfo</span><span class=o>></span> <span class=n>infos</span> <span class=o>=</span> <span class=n>TestSt [...]
+PTransform:</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>TestStream</span><span class=o><</span><span class=n>GameActionInfo</span><span class=o>></span> <span class=n>createEvents</span> <span class=o>=</span> <span class=n [...]
<span class=o>.</span><span class=na>addElements</span><span class=o>(</span><span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"sky"</span><span class=o>,</span> <span class=s>"blue"</span><span class=o>,</span> <span class=n>12</span><span class=o>,</span> <span class=k>new</span> <span class=n>Instant</span><span class=o>(</span><span class=n>0L</span><span class=o>)),</span>
<span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"navy"</span><span class=o>,</span> <span class=s>"blue"</span><span class=o>,</span> <span class=n>3</span><span class=o>,</span> <span class=k>new</span> <span class=n>Instant</span><span class=o>(</span><span class=n>0L</span><span class=o>)),</span>
<span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"navy"</span><span class=o>,</span> <span class=s>"blue"</span><span class=o>,</span> <span class=n>3</span><span class=o>,</span> <span class=k>new</span> <span class=n>Instant</span><span class=o>(</span><span class=n>0L</span><span class=o>).</span><span class=na>plus</span><span class=o>(</span><span class=n>Duration</span><span class=o>.</span><span class= [...]
@@ -104,7 +104,7 @@ PTransform:</p><div class="language-java snippet"><div class="notebook-skip code
of the window (shown below to the left of the red watermark), which demonstrates
“unobservably late” data - that is, data that arrives late, but is promoted by
the system to be on time, as it arrives before the watermark passes the end of
-the window</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>TestStream</span><span class=o><</span><span class=n>GameActionInfo</span><span class=o>></span> <span class=n>infos</span> <span class=o>=</span> <span class=n>TestStr [...]
+the window</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>TestStream</span><span class=o><</span><span class=n>GameActionInfo</span><span class=o>></span> <span class=n>createEvents</span> <span class=o>=</span> <span class=n> [...]
<span class=o>.</span><span class=na>addElements</span><span class=o>(</span><span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"sky"</span><span class=o>,</span> <span class=s>"blue"</span><span class=o>,</span> <span class=n>3</span><span class=o>,</span> <span class=k>new</span> <span class=n>Instant</span><span class=o>(</span><span class=n>0L</span><span class=o>)),</span>
<span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"navy"</span><span class=o>,</span> <span class=s>"blue"</span><span class=o>,</span> <span class=n>3</span><span class=o>,</span> <span class=k>new</span> <span class=n>Instant</span><span class=o>(</span><span class=n>0L</span><span class=o>).</span><span class=na>plus</span><span class=o>(</span><span class=n>Duration</span><span class=o>.</span><span class= [...]
<span class=c1>// Move the watermark up to "near" the end of the window
@@ -120,7 +120,7 @@ the window</p><div class="language-java snippet"><div class="notebook-skip code-
<span class=o>.</span><span class=na>containsInAnyOrder</span><span class=o>(</span><span class=n>KV</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=s>"blue"</span><span class=o>,</span> <span class=n>18</span><span class=o>));</span>
<span class=n>p</span><span class=o>.</span><span class=na>run</span><span class=o>();</span></code></pre></div></div></div><h3 id=elements-are-late-and-arrive-after-the-end-of-the-window>Elements are late, and arrive after the end of the window</h3><p>By advancing the watermark farther in time before adding the late data, we can
demonstrate the triggering behavior that causes the system to emit an on-time
-pane, and then after the late data arrives, a pane that refines the result.</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>TestStream</span><span class=o><</span><span class=n>GameActionInfo</span><span class=o>></span> <span [...]
+pane, and then after the late data arrives, a pane that refines the result.</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>TestStream</span><span class=o><</span><span class=n>GameActionInfo</span><span class=o>></span> <span [...]
<span class=o>.</span><span class=na>addElements</span><span class=o>(</span><span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"sky"</span><span class=o>,</span> <span class=s>"blue"</span><span class=o>,</span> <span class=n>3</span><span class=o>,</span> <span class=k>new</span> <span class=n>Instant</span><span class=o>(</span><span class=n>0L</span><span class=o>)),</span>
<span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"navy"</span><span class=o>,</span> <span class=s>"blue"</span><span class=o>,</span> <span class=n>3</span><span class=o>,</span> <span class=k>new</span> <span class=n>Instant</span><span class=o>(</span><span class=n>0L</span><span class=o>).</span><span class=na>plus</span><span class=o>(</span><span class=n>Duration</span><span class=o>.</span><span class= [...]
<span class=c1>// Move the watermark up to "near" the end of the window
@@ -140,7 +140,7 @@ pane, and then after the late data arrives, a pane that refines the result.</p><
<span class=o>.</span><span class=na>containsInAnyOrder</span><span class=o>(</span><span class=n>KV</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=s>"blue"</span><span class=o>,</span> <span class=n>18</span><span class=o>));</span>
<span class=n>p</span><span class=o>.</span><span class=na>run</span><span class=o>();</span></code></pre></div></div></div><h3 id=elements-are-late-and-after-the-end-of-the-window-plus-the-allowed-lateness>Elements are late, and after the end of the window plus the allowed lateness</h3><p>If we push the watermark even further into the future, beyond the maximum
configured allowed lateness, we can demonstrate that the late element is dropped
-by the system.</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>TestStream</span><span class=o><</span><span class=n>GameActionInfo</span><span class=o>></span> <span class=n>infos</span> <span class=o>=</span> <span class=n>Tes [...]
+by the system.</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>TestStream</span><span class=o><</span><span class=n>GameActionInfo</span><span class=o>></span> <span class=n>createEvents</span> <span class=o>=</span> <span clas [...]
<span class=o>.</span><span class=na>addElements</span><span class=o>(</span><span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"sky"</span><span class=o>,</span> <span class=s>"blue"</span><span class=o>,</span> <span class=n>3</span><span class=o>,</span> <span class=n>Duration</span><span class=o>.</span><span class=na>ZERO</span><span class=o>),</span>
<span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"navy"</span><span class=o>,</span> <span class=s>"blue"</span><span class=o>,</span> <span class=n>3</span><span class=o>,</span> <span class=n>Duration</span><span class=o>.</span><span class=na>standardMinutes</span><span class=o>(</span><span class=n>3</span><span class=o>)))</span>
<span class=c1>// Move the watermark up to "near" the end of the window
@@ -163,7 +163,7 @@ by the system.</p><div class="language-java snippet"><div class="notebook-skip c
<span class=n>p</span><span class=o>.</span><span class=na>run</span><span class=o>();</span></code></pre></div></div></div><h3 id=elements-arrive-before-the-end-of-the-window-and-some-processing-time-passes>Elements arrive before the end of the window, and some processing time passes</h3><p>Using additional methods, we can demonstrate the behavior of speculative
triggers by advancing the processing time of the TestStream. If we add elements
to an input PCollection, occasionally advancing the processing time clock, and
-apply <code>CalculateUserScores</code></p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>TestStream</span><span class=o>.</span><span class=na>create</span><span class=o>(</span><span class=n>AvroCoder</span><span class=o>.</span><span [...]
+apply <code>CalculateUserScores</code></p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=n>TestStream</span><span class=o><</span><span class=n>GameActionInfo</span><span class=o>></span> <span class=n>createEvents</span> <span cla [...]
<span class=o>.</span><span class=na>addElements</span><span class=o>(</span><span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"scarlet"</span><span class=o>,</span> <span class=s>"red"</span><span class=o>,</span> <span class=n>3</span><span class=o>,</span> <span class=k>new</span> <span class=n>Instant</span><span class=o>(</span><span class=n>0L</span><span class=o>)),</span>
<span class=k>new</span> <span class=n>GameActionInfo</span><span class=o>(</span><span class=s>"scarlet"</span><span class=o>,</span> <span class=s>"red"</span><span class=o>,</span> <span class=n>2</span><span class=o>,</span> <span class=k>new</span> <span class=n>Instant</span><span class=o>(</span><span class=n>0L</span><span class=o>).</span><span class=na>plus</span><span class=o>(</span><span class=n>Duration</span><span class=o>.</span><span clas [...]
<span class=o>.</span><span class=na>advanceProcessingTime</span><span class=o>(</span><span class=n>Duration</span><span class=o>.</span><span class=na>standardMinutes</span><span class=o>(</span><span class=n>12</span><span class=o>))</span>
@@ -173,7 +173,7 @@ apply <code>CalculateUserScores</code></p><div class="language-java snippet"><di
<span class=o>.</span><span class=na>advanceWatermarkToInfinity</span><span class=o>();</span>
<span class=n>PCollection</span><span class=o><</span><span class=n>KV</span><span class=o><</span><span class=n>String</span><span class=o>,</span> <span class=n>Integer</span><span class=o>>></span> <span class=n>userScores</span> <span class=o>=</span>
- <span class=n>p</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>infos</span><span class=o>).</span><span class=na>apply</span><span class=o>(</span><span class=k>new</span> <span class=n>CalculateUserScores</span><span class=o>(</span><span class=n>ALLOWED_LATENESS</span><span class=o>));</span></code></pre></div></div></div><img class=center-block src=/images/blog/test-stream/elements-processing-speculative.png alt="Elements all arrive befo [...]
+ <span class=n>p</span><span class=o>.</span><span class=na>apply</span><span class=o>(</span><span class=n>createEvents</span><span class=o>).</span><span class=na>apply</span><span class=o>(</span><span class=k>new</span> <span class=n>CalculateUserScores</span><span class=o>(</span><span class=n>ALLOWED_LATENESS</span><span class=o>));</span></code></pre></div></div></div><img class=center-block src=/images/blog/test-stream/elements-processing-speculative.png alt="Elements all arri [...]
<span class=o>.</span><span class=na>inEarlyGlobalWindowPanes</span><span class=o>()</span>
<span class=o>.</span><span class=na>containsInAnyOrder</span><span class=o>(</span><span class=n>KV</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=s>"scarlet"</span><span class=o>,</span> <span class=n>5</span><span class=o>),</span>
<span class=n>KV</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=s>"scarlet"</span><span class=o>,</span> <span class=n>9</span><span class=o>),</span>
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index b1e7f05..b532506 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.28.0/</loc><lastmod>2021-02-22T11:40:20-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-02-22T11:40:20-08:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-02-22T11:40:20-08:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-03-16T13:53:01+05:00</lastmod></url><url><loc>/blog/k [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.28.0/</loc><lastmod>2021-02-22T11:40:20-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/blog/k [...]
\ No newline at end of file