You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by me...@apache.org on 2018/01/30 17:59:29 UTC
[beam-site] branch asf-site updated (f0e45db -> 21477ff)
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam-site.git.
from f0e45db Prepare repository for deployment.
add 985fefd Fix broken links due to code directory moves
add d910f9b This closes #380
new 21477ff Prepare repository for deployment.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../2016/10/11/strata-hadoop-world-and-beam.html | 2 +-
content/blog/2016/10/20/test-stream.html | 4 +-
content/documentation/io/testing/index.html | 2 +-
content/documentation/programming-guide/index.html | 7 +-
content/feed.xml | 4 +-
.../get-started/mobile-gaming-example/index.html | 193 +++++++++++----------
.../2016-10-12-strata-hadoop-world-and-beam.md | 2 +-
src/_posts/2016-10-20-test-stream.md | 4 +-
src/documentation/io/testing.md | 2 +-
src/documentation/programming-guide.md | 8 +-
src/get-started/mobile-gaming-example.md | 30 ++--
11 files changed, 136 insertions(+), 122 deletions(-)
--
To stop receiving notification emails like this one, please contact
mergebot-role@apache.org.
[beam-site] 01/01: Prepare repository for deployment.
Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit 21477ff9ba5ffff06f84da4dc3e6155046098503
Author: Mergebot <me...@apache.org>
AuthorDate: Tue Jan 30 09:59:27 2018 -0800
Prepare repository for deployment.
---
.../2016/10/11/strata-hadoop-world-and-beam.html | 2 +-
content/blog/2016/10/20/test-stream.html | 4 +-
content/documentation/io/testing/index.html | 2 +-
content/documentation/programming-guide/index.html | 7 +-
content/feed.xml | 4 +-
.../get-started/mobile-gaming-example/index.html | 193 +++++++++++----------
6 files changed, 113 insertions(+), 99 deletions(-)
diff --git a/content/beam/update/2016/10/11/strata-hadoop-world-and-beam.html b/content/beam/update/2016/10/11/strata-hadoop-world-and-beam.html
index 9c939a8..f42292e 100644
--- a/content/beam/update/2016/10/11/strata-hadoop-world-and-beam.html
+++ b/content/beam/update/2016/10/11/strata-hadoop-world-and-beam.html
@@ -95,7 +95,7 @@
<p><img src="/images/blog/IMG_20160927_170956.jpg" alt="Exercise time" /></p>
-<p>If you want to take a look at the tutorial materials, we’ve put them up <a href="https://github.com/eljefe6a/beamexample">on GitHub</a>. This includes the <a href="https://github.com/eljefe6a/beamexample/blob/master/BeamTutorial/slides.pdf">actual slides</a> as well as the <a href="https://github.com/eljefe6a/beamexample/tree/master/BeamTutorial/src/main/java/org/apache/beam/examples/tutorial/game">exercises</a> that we covered. If you’re looking to learn a little about Beam, this is [...]
+<p>If you want to take a look at the tutorial materials, we’ve put them up <a href="https://github.com/eljefe6a/beamexample">on GitHub</a>. This includes the <a href="https://github.com/eljefe6a/beamexample/blob/master/BeamTutorial/slides.pdf">actual slides</a> as well as the <a href="https://github.com/eljefe6a/beamexample/tree/master/BeamTutorial/src/main/java/org/apache/beam/examples/tutorial/game">exercises</a> that we covered. If you’re looking to learn a little about Beam, this is [...]
<p>I want to share some of takeaways I had about Beam during the conference.</p>
diff --git a/content/blog/2016/10/20/test-stream.html b/content/blog/2016/10/20/test-stream.html
index f98d239..f2cc8bb 100644
--- a/content/blog/2016/10/20/test-stream.html
+++ b/content/blog/2016/10/20/test-stream.html
@@ -128,8 +128,8 @@ from the Mobile Gaming example series.</p>
<h2 id="leaderboard-and-the-mobile-gaming-example">LeaderBoard and the Mobile Gaming Example</h2>
-<p><a href="https://github.com/apache/beam/blob/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java#L177">LeaderBoard</a>
-is part of the <a href="https://github.com/apache/beam/tree/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game">Beam mobile gaming examples</a>
+<p><a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java#L177">LeaderBoard</a>
+is part of the <a href="https://github.com/apache/beam/tree/master/examples/java/src/main/java/org/apache/beam/examples/complete/game">Beam mobile gaming examples</a>
(and <a href="/get-started/mobile-gaming-example/">walkthroughs</a>)
which produces a continuous accounting of user and team scores. User scores are
calculated over the lifetime of the program, while team scores are calculated
diff --git a/content/documentation/io/testing/index.html b/content/documentation/io/testing/index.html
index bb0b1b5..4a3eb11 100644
--- a/content/documentation/io/testing/index.html
+++ b/content/documentation/io/testing/index.html
@@ -832,7 +832,7 @@ dynamic_pipeline_options:
<li>Having your test take a pipeline option that decides whether to generate a small or large amount of test data (where small and large are sizes appropriate to your data store)</li>
</ol>
-<p>An example of this is <a href="https://github.com/apache/beam/tree/master/sdks/java/io/hadoop/input-format">HadoopInputFormatIO</a>’s tests.</p>
+<p>An example of this is <a href="https://github.com/apache/beam/tree/master/sdks/java/io/hadoop-input-format">HadoopInputFormatIO</a>’s tests.</p>
<!--
# Next steps
diff --git a/content/documentation/programming-guide/index.html b/content/documentation/programming-guide/index.html
index 1794b85..6db8a56 100644
--- a/content/documentation/programming-guide/index.html
+++ b/content/documentation/programming-guide/index.html
@@ -1391,10 +1391,9 @@ followed by a <code class="highlighter-rouge">ParDo</code> to consume the result
and format data from each collection.</p>
<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">PCollection</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">CoGbkResult</span><span class="o">>></span> <span class="n">results</span> <span class="o">=</span>
- <span class="n">KeyedPCollectionTuple</span>
- <span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">emailsTag</span><span class="o">,</span> <span class="n">emails</span><span class="o">)</span>
- <span class="o">.</span><span class="na">and</span><span class="o">(</span><span class="n">phonesTag</span><span class="o">,</span> <span class="n">phones</span><span class="o">)</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">CoGroupByKey</span><span class="o">.<</span><span class="n">String</span><span class="o">></span><span class="n">create</span><span class="o">());</span>
+ <span class="n">KeyedPCollectionTuple</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">emailsTag</span><span class="o">,</span> <span class="n">emails</span><span class="o">)</span>
+ <span class="o">.</span><span class="na">and</span><span class="o">(</span><span class="n">phonesTag</span><span class="o">,</span> <span class="n">phones</span><span class="o">)</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">CoGroupByKey</span><span class="o">.</span><span class="na">create</span><span class="o">());</span>
<span class="n">PCollection</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">contactLines</span> <span class="o">=</span> <span class="n">results</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">ParDo</span><span class="o">.</span><span class="na">of</span><span class="o">(</span>
<span class="k">new</span> <span class="n">DoFn</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">CoGbkResult</span><span class="o">>,</span> <span class="n">String</span><span class="o">>()</span> <span class="o">{</span>
diff --git a/content/feed.xml b/content/feed.xml
index 8e0966a..2c48dfa 100644
--- a/content/feed.xml
+++ b/content/feed.xml
@@ -2192,8 +2192,8 @@ from the Mobile Gaming example series.</p>
<h2 id="leaderboard-and-the-mobile-gaming-example">LeaderBoard and the Mobile Gaming Example</h2>
-<p><a href="https://github.com/apache/beam/blob/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java#L177">LeaderBoard</a>
-is part of the <a href="https://github.com/apache/beam/tree/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game">Beam mobile gaming examples</a>
+<p><a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java#L177">LeaderBoard</a>
+is part of the <a href="https://github.com/apache/beam/tree/master/examples/java/src/main/java/org/apache/beam/examples/complete/game">Beam mobile gaming examples</a>
(and <a href="/get-started/mobile-gaming-example/">walkthroughs</a>)
which produces a continuous accounting of user and team scores. User scores are
calculated over the lifetime of the program, while team scores are calculated
diff --git a/content/get-started/mobile-gaming-example/index.html b/content/get-started/mobile-gaming-example/index.html
index 91c260f..9714bbb 100644
--- a/content/get-started/mobile-gaming-example/index.html
+++ b/content/get-started/mobile-gaming-example/index.html
@@ -230,7 +230,7 @@ looks more like what is depicted by the red squiggly line above the ideal line.<
<p>The <code class="highlighter-rouge">UserScore</code> pipeline is the simplest example for processing mobile game data. <code class="highlighter-rouge">UserScore</code> determines the total score per user over a finite data set (for example, one day’s worth of scores stored on the game server). Pipelines like <code class="highlighter-rouge">UserScore</code> are best run periodically after all relevant data has been gathered. For example, <code class="highlighter-rouge">UserScore</code> [...]
<blockquote class="language-java">
- <p><strong>Note:</strong> See <a href="https://github.com/apache/beam/blob/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game/UserScore.java">UserScore on GitHub</a> for the complete example pipeline program.</p>
+ <p><strong>Note:</strong> See <a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete/game/UserScore.java">UserScore on GitHub</a> for the complete example pipeline program.</p>
</blockquote>
<blockquote class="language-py">
@@ -281,10 +281,11 @@ looks more like what is depicted by the red squiggly line above the ideal line.<
<span class="n">PCollection</span><span class="o"><</span><span class="n">GameActionInfo</span><span class="o">></span> <span class="n">gameInfo</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">gameInfo</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">MapElements</span>
- <span class="o">.</span><span class="na">into</span><span class="o">(</span><span class="n">TypeDescriptors</span><span class="o">.</span><span class="na">kvs</span><span class="o">(</span><span class="n">TypeDescriptors</span><span class="o">.</span><span class="na">strings</span><span class="o">(),</span> <span class="n">TypeDescriptors</span><span class="o">.</span><span class="na">integers</span><span class="o">()))</span>
- <span class="o">.</span><span class="na">via</span><span class="o">((</span><span class="n">GameActionInfo</span> <span class="n">gInfo</span><span class="o">)</span> <span class="o">-></span> <span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">gInfo</span><span class="o">.</span><span class="na">getKey</span><span class="o">(</span><span class="n">field</span><span class="o">),</span> <span class="n">gInfo</span [...]
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Sum</span><span class="o">.<</span><span class="n">String</span><span class="o">></span><span class="n">integersPerKey</span><span class="o">());</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="n">MapElements</span><span class="o">.</span><span class="na">into</span><span class="o">(</span>
+ <span class="n">TypeDescriptors</span><span class="o">.</span><span class="na">kvs</span><span class="o">(</span><span class="n">TypeDescriptors</span><span class="o">.</span><span class="na">strings</span><span class="o">(),</span> <span class="n">TypeDescriptors</span><span class="o">.</span><span class="na">integers</span><span class="o">()))</span>
+ <span class="o">.</span><span class="na">via</span><span class="o">((</span><span class="n">GameActionInfo</span> <span class="n">gInfo</span><span class="o">)</span> <span class="o">-></span> <span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">gInfo</span><span class="o">.</span><span class="na">getKey</span><span class="o">(</span><span class="n">field</span><span class="o">),</span> <span class="n">gInfo [...]
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Sum</span><span class="o">.</span><span class="na">integersPerKey</span><span class="o">());</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre>
@@ -321,11 +322,7 @@ looks more like what is depicted by the red squiggly line above the ideal line.<
<span class="c1">// Extract and sum username/score pairs from the event data.</span>
<span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"ExtractUserScore"</span><span class="o">,</span> <span class="k">new</span> <span class="n">ExtractAndSumScore</span><span class="o">(</span><span class="s">"user"</span><span class="o">))</span>
<span class="o">.</span><span class="na">apply</span><span class="o">(</span>
- <span class="s">"WriteUserScoreSums"</span><span class="o">,</span>
- <span class="k">new</span> <span class="n">WriteToText</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>>(</span>
- <span class="n">options</span><span class="o">.</span><span class="na">getOutput</span><span class="o">(),</span>
- <span class="n">configureOutput</span><span class="o">(),</span>
- <span class="kc">false</span><span class="o">));</span>
+ <span class="s">"WriteUserScoreSums"</span><span class="o">,</span> <span class="k">new</span> <span class="n">WriteToText</span><span class="o"><>(</span><span class="n">options</span><span class="o">.</span><span class="na">getOutput</span><span class="o">(),</span> <span class="n">configureOutput</span><span class="o">(),</span> <span class="kc">false</span><span class="o">));</span>
<span class="c1">// Run the batch pipeline.</span>
<span class="n">pipeline</span><span class="o">.</span><span class="na">run</span><span class="o">().</span><span class="na">waitUntilFinish</span><span class="o">();</span>
@@ -396,7 +393,7 @@ looks more like what is depicted by the red squiggly line above the ideal line.<
<p>Like <code class="highlighter-rouge">UserScore</code>, <code class="highlighter-rouge">HourlyTeamScore</code> is best thought of as a job to be run periodically after all the relevant data has been gathered (such as once per day). The pipeline reads a fixed data set from a file, and writes the results to a Google Cloud BigQuery table.</p>
<blockquote class="language-java">
- <p><strong>Note:</strong> See <a href="https://github.com/apache/beam/blob/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game/HourlyTeamScore.java">HourlyTeamScore on GitHub</a> for the complete example pipeline program.</p>
+ <p><strong>Note:</strong> See <a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete/game/HourlyTeamScore.java">HourlyTeamScore on GitHub</a> for the complete example pipeline program.</p>
</blockquote>
<blockquote class="language-py">
@@ -444,10 +441,12 @@ logical windows based on when those scores occurred in event time.</em></p>
<p>The following code shows this:</p>
<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="c1">// Add an element timestamp based on the event log, and apply fixed windowing.</span>
-<span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"AddEventTimestamps"</span><span class="o">,</span>
- <span class="n">WithTimestamps</span><span class="o">.</span><span class="na">of</span><span class="o">((</span><span class="n">GameActionInfo</span> <span class="n">i</span><span class="o">)</span> <span class="o">-></span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="n">i</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">())))</span>
-<span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"FixedWindowsTeam"</span><span class="o">,</span> <span class="n">Window</span><span class="o">.<</span><span class="n">GameActionInfo</span><span class="o">></span><span class="n">into</span><span class="o">(</span>
- <span class="n">FixedWindows</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getWindowDuration</span><span class="o">()))))</span>
+<span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"AddEventTimestamps"</span><span class="o">,</span>
+ <span class="n">WithTimestamps</span><span class="o">.</span><span class="na">of</span><span class="o">((</span><span class="n">GameActionInfo</span> <span class="n">i</span><span class="o">)</span> <span class="o">-></span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="n">i</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">())))</span>
+<span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"FixedWindowsTeam"</span><span class="o">,</span>
+ <span class="n">Window</span><span class="o">.</span><span class="na">into</span><span class="o">(</span><span class="n">FixedWindows</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getWindowDuration</span><span class="o">()))))</span>
</code></pre>
</div>
<div class="language-py highlighter-rouge"><pre class="highlight"><code><span class="c"># Add an element timestamp based on the event log, and apply fixed</span>
@@ -471,12 +470,14 @@ logical windows based on when those scores occurred in event time.</em></p>
<p>The following code shows how <code class="highlighter-rouge">HourlyTeamScore</code> uses the <code class="highlighter-rouge">Filter</code> transform to filter events that occur either before or after the relevant analysis period:</p>
-<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"FilterStartTime"</span><span class="o">,</span> <span class="n">Filter</span><span class="o">.</span><span class="na">by</span><span class="o">(</span>
- <span class="o">(</span><span class="n">GameActionInfo</span> <span class="n">gInfo</span><span class="o">)</span>
- <span class="o">-></span> <span class="n">gInfo</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">()</span> <span class="o">></span> <span class="n">startMinTimestamp</span><span class="o">.</span><span class="na">getMillis</span><span class="o">()))</span>
-<span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"FilterEndTime"</span><span class="o">,</span> <span class="n">Filter</span><span class="o">.</span><span class="na">by</span><span class="o">(</span>
- <span class="o">(</span><span class="n">GameActionInfo</span> <span class="n">gInfo</span><span class="o">)</span>
- <span class="o">-></span> <span class="n">gInfo</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">()</span> <span class="o"><</span> <span class="n">stopMinTimestamp</span><span class="o">.</span><span class="na">getMillis</span><span class="o">()))</span>
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"FilterStartTime"</span><span class="o">,</span>
+ <span class="n">Filter</span><span class="o">.</span><span class="na">by</span><span class="o">(</span>
+ <span class="o">(</span><span class="n">GameActionInfo</span> <span class="n">gInfo</span><span class="o">)</span> <span class="o">-></span> <span class="n">gInfo</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">()</span> <span class="o">></span> <span class="n">startMinTimestamp</span><span class="o">.</span><span class="na">getMillis</span><span class="o">()))</span>
+<span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"FilterEndTime"</span><span class="o">,</span>
+ <span class="n">Filter</span><span class="o">.</span><span class="na">by</span><span class="o">(</span>
+ <span class="o">(</span><span class="n">GameActionInfo</span> <span class="n">gInfo</span><span class="o">)</span> <span class="o">-></span> <span class="n">gInfo</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">()</span> <span class="o"><</span> <span class="n">stopMinTimestamp</span><span class="o">.</span><span class="na">getMillis</span><span class="o">()))</span>
</code></pre>
</div>
<div class="language-py highlighter-rouge"><pre class="highlight"><code><span class="o">|</span> <span class="s">'FilterStartTime'</span> <span class="o">>></span> <span class="n">beam</span><span class="o">.</span><span class="n">Filter</span><span class="p">(</span>
@@ -499,40 +500,43 @@ logical windows based on when those scores occurred in event time.</em></p>
<span class="kd">final</span> <span class="n">Instant</span> <span class="n">startMinTimestamp</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="n">minFmt</span><span class="o">.</span><span class="na">parseMillis</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getStartMin</span><span class="o">()));</span>
<span class="c1">// Read 'gaming' events from a text file.</span>
- <span class="n">pipeline</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">TextIO</span><span class="o">.</span><span class="na">read</span><span class="o">().</span><span class="na">from</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getInput</span><span class="o">()))</span>
- <span class="c1">// Parse the incoming data.</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"ParseGameEvent"</span><span class="o">,</span> <span class="n">ParDo</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="k">new</span> <span class="n">ParseEventFn</span><span class="o">()))</span>
-
- <span class="c1">// Filter out data before and after the given times so that it is not included</span>
- <span class="c1">// in the calculations. As we collect data in batches (say, by day), the batch for the day</span>
- <span class="c1">// that we want to analyze could potentially include some late-arriving data from the previous</span>
- <span class="c1">// day. If so, we want to weed it out. Similarly, if we include data from the following day</span>
- <span class="c1">// (to scoop up late-arriving events from the day we're analyzing), we need to weed out events</span>
- <span class="c1">// that fall after the time period we want to analyze.</span>
- <span class="c1">// [START DocInclude_HTSFilters]</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"FilterStartTime"</span><span class="o">,</span> <span class="n">Filter</span><span class="o">.</span><span class="na">by</span><span class="o">(</span>
- <span class="o">(</span><span class="n">GameActionInfo</span> <span class="n">gInfo</span><span class="o">)</span>
- <span class="o">-></span> <span class="n">gInfo</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">()</span> <span class="o">></span> <span class="n">startMinTimestamp</span><span class="o">.</span><span class="na">getMillis</span><span class="o">()))</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"FilterEndTime"</span><span class="o">,</span> <span class="n">Filter</span><span class="o">.</span><span class="na">by</span><span class="o">(</span>
- <span class="o">(</span><span class="n">GameActionInfo</span> <span class="n">gInfo</span><span class="o">)</span>
- <span class="o">-></span> <span class="n">gInfo</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">()</span> <span class="o"><</span> <span class="n">stopMinTimestamp</span><span class="o">.</span><span class="na">getMillis</span><span class="o">()))</span>
- <span class="c1">// [END DocInclude_HTSFilters]</span>
-
- <span class="c1">// [START DocInclude_HTSAddTsAndWindow]</span>
- <span class="c1">// Add an element timestamp based on the event log, and apply fixed windowing.</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"AddEventTimestamps"</span><span class="o">,</span>
- <span class="n">WithTimestamps</span><span class="o">.</span><span class="na">of</span><span class="o">((</span><span class="n">GameActionInfo</span> <span class="n">i</span><span class="o">)</span> <span class="o">-></span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="n">i</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">())))</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"FixedWindowsTeam"</span><span class="o">,</span> <span class="n">Window</span><span class="o">.<</span><span class="n">GameActionInfo</span><span class="o">></span><span class="n">into</span><span class="o">(</span>
- <span class="n">FixedWindows</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getWindowDuration</span><span class="o">()))))</span>
- <span class="c1">// [END DocInclude_HTSAddTsAndWindow]</span>
+ <span class="n">pipeline</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">TextIO</span><span class="o">.</span><span class="na">read</span><span class="o">().</span><span class="na">from</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getInput</span><span class="o">()))</span>
+ <span class="c1">// Parse the incoming data.</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"ParseGameEvent"</span><span class="o">,</span> <span class="n">ParDo</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="k">new</span> <span class="n">ParseEventFn</span><span class="o">()))</span>
- <span class="c1">// Extract and sum teamname/score pairs from the event data.</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"ExtractTeamScore"</span><span class="o">,</span> <span class="k">new</span> <span class="n">ExtractAndSumScore</span><span class="o">(</span><span class="s">"team"</span><span class="o">))</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"WriteTeamScoreSums"</span><span class="o">,</span>
- <span class="k">new</span> <span class="n">WriteToText</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>>(</span>
- <span class="n">options</span><span class="o">.</span><span class="na">getOutput</span><span class="o">(),</span>
- <span class="n">configureOutput</span><span class="o">(),</span>
- <span class="kc">true</span><span class="o">));</span>
+ <span class="c1">// Filter out data before and after the given times so that it is not included</span>
+ <span class="c1">// in the calculations. As we collect data in batches (say, by day), the batch for the day</span>
+ <span class="c1">// that we want to analyze could potentially include some late-arriving data from the</span>
+ <span class="c1">// previous day.</span>
+ <span class="c1">// If so, we want to weed it out. Similarly, if we include data from the following day</span>
+ <span class="c1">// (to scoop up late-arriving events from the day we're analyzing), we need to weed out</span>
+ <span class="c1">// events that fall after the time period we want to analyze.</span>
+ <span class="c1">// [START DocInclude_HTSFilters]</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"FilterStartTime"</span><span class="o">,</span>
+ <span class="n">Filter</span><span class="o">.</span><span class="na">by</span><span class="o">(</span>
+ <span class="o">(</span><span class="n">GameActionInfo</span> <span class="n">gInfo</span><span class="o">)</span> <span class="o">-></span> <span class="n">gInfo</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">()</span> <span class="o">></span> <span class="n">startMinTimestamp</span><span class="o">.</span><span class="na">getMillis</span><span class="o">()))</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"FilterEndTime"</span><span class="o">,</span>
+ <span class="n">Filter</span><span class="o">.</span><span class="na">by</span><span class="o">(</span>
+ <span class="o">(</span><span class="n">GameActionInfo</span> <span class="n">gInfo</span><span class="o">)</span> <span class="o">-></span> <span class="n">gInfo</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">()</span> <span class="o"><</span> <span class="n">stopMinTimestamp</span><span class="o">.</span><span class="na">getMillis</span><span class="o">()))</span>
+ <span class="c1">// [END DocInclude_HTSFilters]</span>
+
+ <span class="c1">// [START DocInclude_HTSAddTsAndWindow]</span>
+ <span class="c1">// Add an element timestamp based on the event log, and apply fixed windowing.</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"AddEventTimestamps"</span><span class="o">,</span>
+ <span class="n">WithTimestamps</span><span class="o">.</span><span class="na">of</span><span class="o">((</span><span class="n">GameActionInfo</span> <span class="n">i</span><span class="o">)</span> <span class="o">-></span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="n">i</span><span class="o">.</span><span class="na">getTimestamp</span><span class="o">())))</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"FixedWindowsTeam"</span><span class="o">,</span>
+ <span class="n">Window</span><span class="o">.</span><span class="na">into</span><span class="o">(</span><span class="n">FixedWindows</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getWindowDuration</span><span class="o">()))))</span>
+ <span class="c1">// [END DocInclude_HTSAddTsAndWindow]</span>
+
+ <span class="c1">// Extract and sum teamname/score pairs from the event data.</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"ExtractTeamScore"</span><span class="o">,</span> <span class="k">new</span> <span class="n">ExtractAndSumScore</span><span class="o">(</span><span class="s">"team"</span><span class="o">))</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"WriteTeamScoreSums"</span><span class="o">,</span> <span class="k">new</span> <span class="n">WriteToText</span><span class="o"><>(</span><span class="n">options</span><span class="o">.</span><span class="na">getOutput</span><span class="o">(),</span> <span class="n">configureOutput</span><span class="o">(),</span> <span class="kc">true</span><span class="o">));</span>
<span class="n">pipeline</span><span class="o">.</span><span class="na">run</span><span class="o">().</span><span class="na">waitUntilFinish</span><span class="o">();</span>
<span class="o">}</span>
@@ -663,7 +667,7 @@ logical windows based on when those scores occurred in event time.</em></p>
<p>Because the <code class="highlighter-rouge">LeaderBoard</code> pipeline reads the game data from an unbounded source as that data is generated, you can think of the pipeline as an ongoing job running concurrently with the game process. <code class="highlighter-rouge">LeaderBoard</code> can thus provide low-latency insights into how users are playing the game at any given moment — useful if, for example, we want to provide a live web-based scoreboard so that users can track their progr [...]
<blockquote class="language-java">
- <p><strong>Note:</strong> See <a href="https://github.com/apache/beam/blob/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java">LeaderBoard on GitHub</a> for the complete example pipeline program.</p>
+ <p><strong>Note:</strong> See <a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete/game/LeaderBoard.java">LeaderBoard on GitHub</a> for the complete example pipeline program.</p>
</blockquote>
<blockquote class="language-py">
@@ -852,7 +856,7 @@ late results.</em></p>
<p>Like <code class="highlighter-rouge">LeaderBoard</code>, <code class="highlighter-rouge">GameStats</code> reads data from an unbounded source. It is best thought of as an ongoing job that provides insight into the game as users play.</p>
<blockquote class="language-java">
- <p><strong>Note:</strong> See <a href="https://github.com/apache/beam/blob/master/examples/java8/src/main/java/org/apache/beam/examples/complete/game/GameStats.java">GameStats on GitHub</a> for the complete example pipeline program.</p>
+ <p><strong>Note:</strong> See <a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete/game/GameStats.java">GameStats on GitHub</a> for the complete example pipeline program.</p>
</blockquote>
<blockquote class="language-py">
@@ -889,12 +893,12 @@ late results.</em></p>
<span class="kd">public</span> <span class="n">PCollection</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="nf">expand</span><span class="o">(</span><span class="n">PCollection</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span cla [...]
<span class="c1">// Get the sum of scores for each user.</span>
- <span class="n">PCollection</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">sumScores</span> <span class="o">=</span> <span class="n">userScores</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"UserSum"</span><span class="o">,</span> <span class="n">Sum</span><span class="o">.<</span><span class="n">String</span><span class="o">></span><span class="n">integersPerKey</span><span class="o">());</span>
+ <span class="n">PCollection</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">sumScores</span> <span class="o">=</span>
+ <span class="n">userScores</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"UserSum"</span><span class="o">,</span> <span class="n">Sum</span><span class="o">.</span><span class="na">integersPerKey</span><span class="o">());</span>
<span class="c1">// Extract the score from each element, and use it to find the global mean.</span>
- <span class="kd">final</span> <span class="n">PCollectionView</span><span class="o"><</span><span class="n">Double</span><span class="o">></span> <span class="n">globalMeanScore</span> <span class="o">=</span> <span class="n">sumScores</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Values</span><span class="o">.<</span><span class="n">Integer</span><span class="o">></span><span class="n">create</span><span class="o">())</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Mean</span><span class="o">.<</span><span class="n">Integer</span><span class="o">></span><span class="n">globally</span><span class="o">().</span><span class="na">asSingletonView</span><span class="o">());</span>
+ <span class="kd">final</span> <span class="n">PCollectionView</span><span class="o"><</span><span class="n">Double</span><span class="o">></span> <span class="n">globalMeanScore</span> <span class="o">=</span>
+ <span class="n">sumScores</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Values</span><span class="o">.</span><span class="na">create</span><span class="o">()).</span><span class="na">apply</span><span class="o">(</span><span class="n">Mean</span><span class="o">.<</span><span class="n">Integer</span><span class="o">></span><span class="n">globally</span><span class="o">().</span><span class="na">asSingletonView</span><spa [...]
<span class="c1">// Filter the user sums using the global mean.</span>
<span class="n">PCollection</span><span class="o"><</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span> <span class="n">filtered</span> <span class="o">=</span> <span class="n">sumScores</span>
@@ -961,21 +965,26 @@ late results.</em></p>
<span class="c1">// suspected robots-- to filter out scores from those users from the sum.</span>
<span class="c1">// Write the results to BigQuery.</span>
<span class="n">rawEvents</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"WindowIntoFixedWindows"</span><span class="o">,</span> <span class="n">Window</span><span class="o">.<</span><span class="n">GameActionInfo</span><span class="o">></span><span class="n">into</span><span class="o">(</span>
- <span class="n">FixedWindows</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getFixedWindowDuration</span><span class="o">()))))</span>
- <span class="c1">// Filter out the detected spammer users, using the side input derived above.</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"FilterOutSpammers"</span><span class="o">,</span> <span class="n">ParDo</span>
- <span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="k">new</span> <span class="n">DoFn</span><span class="o"><</span><span class="n">GameActionInfo</span><span class="o">,</span> <span class="n">GameActionInfo</span><span class="o">>()</span> <span class="o">{</span>
- <span class="nd">@ProcessElement</span>
- <span class="kd">public</span> <span class="kt">void</span> <span class="nf">processElement</span><span class="o">(</span><span class="n">ProcessContext</span> <span class="n">c</span><span class="o">)</span> <span class="o">{</span>
- <span class="c1">// If the user is not in the spammers Map, output the data element.</span>
- <span class="k">if</span> <span class="o">(</span><span class="n">c</span><span class="o">.</span><span class="na">sideInput</span><span class="o">(</span><span class="n">spammersView</span><span class="o">).</span><span class="na">get</span><span class="o">(</span><span class="n">c</span><span class="o">.</span><span class="na">element</span><span class="o">().</span><span class="na">getUser</span><span class="o">().</span><span class="na">trim</span><span class="o">())</s [...]
- <span class="n">c</span><span class="o">.</span><span class="na">output</span><span class="o">(</span><span class="n">c</span><span class="o">.</span><span class="na">element</span><span class="o">());</span>
- <span class="o">}</span>
- <span class="o">}</span>
- <span class="o">}).</span><span class="na">withSideInputs</span><span class="o">(</span><span class="n">spammersView</span><span class="o">))</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"WindowIntoFixedWindows"</span><span class="o">,</span>
+ <span class="n">Window</span><span class="o">.</span><span class="na">into</span><span class="o">(</span>
+ <span class="n">FixedWindows</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getFixedWindowDuration</span><span class="o">()))))</span>
+ <span class="c1">// Filter out the detected spammer users, using the side input derived above.</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"FilterOutSpammers"</span><span class="o">,</span>
+ <span class="n">ParDo</span><span class="o">.</span><span class="na">of</span><span class="o">(</span>
+ <span class="k">new</span> <span class="n">DoFn</span><span class="o"><</span><span class="n">GameActionInfo</span><span class="o">,</span> <span class="n">GameActionInfo</span><span class="o">>()</span> <span class="o">{</span>
+ <span class="nd">@ProcessElement</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">processElement</span><span class="o">(</span><span class="n">ProcessContext</span> <span class="n">c</span><span class="o">)</span> <span class="o">{</span>
+ <span class="c1">// If the user is not in the spammers Map, output the data element.</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">c</span><span class="o">.</span><span class="na">sideInput</span><span class="o">(</span><span class="n">spammersView</span><span class="o">).</span><span class="na">get</span><span class="o">(</span><span class="n">c</span><span class="o">.</span><span class="na">element</span><span class="o">().</span><span class="na">getUser</span><span class="o">().</span><span class="na">trim</span><span class="o"> [...]
+ <span class="n">c</span><span class="o">.</span><span class="na">output</span><span class="o">(</span><span class="n">c</span><span class="o">.</span><span class="na">element</span><span class="o">());</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="o">})</span>
+ <span class="o">.</span><span class="na">withSideInputs</span><span class="o">(</span><span class="n">spammersView</span><span class="o">))</span>
<span class="c1">// Extract and sum teamname/score pairs from the event data.</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"ExtractTeamScore"</span><span class="o">,</span> <span class="k">new</span> <span class="n">ExtractAndSumScore</span><span class="o">(</span><span class="s">"team"</span><span class="o">))</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"ExtractTeamScore"</span><span class="o">,</span> <span class="k">new</span> <span class="n">ExtractAndSumScore</span><span class="o">(</span><span class="s">"team"</span><span class="o">))</span>
</code></pre>
</div>
<div class="language-py highlighter-rouge"><pre class="highlight"><code><span class="c"># Calculate the total score per team over fixed windows, and emit cumulative</span>
@@ -1016,14 +1025,16 @@ between instances are.</em></p>
<span class="c1">// This information could help the game designers track the changing user engagement</span>
<span class="c1">// as their set of games changes.</span>
<span class="n">userEvents</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"WindowIntoSessions"</span><span class="o">,</span> <span class="n">Window</span><span class="o">.<</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span><span class="n">into</span><span class="o">(</span>
- <span class="n">Sessions</span><span class="o">.</span><span class="na">withGapDuration</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getSessionGap</span><span class="o">())))</span>
- <span class="o">.</span><span class="na">withTimestampCombiner</span><span class="o">(</span><span class="n">TimestampCombiner</span><span class="o">.</span><span class="na">END_OF_WINDOW</span><span class="o">))</span>
- <span class="c1">// For this use, we care only about the existence of the session, not any particular</span>
- <span class="c1">// information aggregated over it, so the following is an efficient way to do that.</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Combine</span><span class="o">.</span><span class="na">perKey</span><span class="o">(</span><span class="n">x</span> <span class="o">-></span> <span class="mi">0</span><span class="o">))</span>
- <span class="c1">// Get the duration per session.</span>
- <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"UserSessionActivity"</span><span class="o">,</span> <span class="n">ParDo</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="k">new</span> <span class="n">UserSessionInfoFn</span><span class="o">()))</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"WindowIntoSessions"</span><span class="o">,</span>
+ <span class="n">Window</span><span class="o">.<</span><span class="n">KV</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>></span><span class="n">into</span><span class="o">(</span>
+ <span class="n">Sessions</span><span class="o">.</span><span class="na">withGapDuration</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getSessionGap</span><span class="o">())))</span>
+ <span class="o">.</span><span class="na">withTimestampCombiner</span><span class="o">(</span><span class="n">TimestampCombiner</span><span class="o">.</span><span class="na">END_OF_WINDOW</span><span class="o">))</span>
+ <span class="c1">// For this use, we care only about the existence of the session, not any particular</span>
+ <span class="c1">// information aggregated over it, so the following is an efficient way to do that.</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Combine</span><span class="o">.</span><span class="na">perKey</span><span class="o">(</span><span class="n">x</span> <span class="o">-></span> <span class="mi">0</span><span class="o">))</span>
+ <span class="c1">// Get the duration per session.</span>
+ <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"UserSessionActivity"</span><span class="o">,</span> <span class="n">ParDo</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="k">new</span> <span class="n">UserSessionInfoFn</span><span class="o">()))</span>
</code></pre>
</div>
<div class="language-py highlighter-rouge"><pre class="highlight"><code><span class="c"># Detect user sessions-- that is, a burst of activity separated by a gap</span>
@@ -1048,16 +1059,20 @@ between instances are.</em></p>
<p>This gives us a set of user sessions, each with an attached duration. We can then calculate the <em>average</em> session length by re-windowing the data into fixed time windows, and then calculating the average for all sessions that end in each hour:</p>
<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="c1">// Re-window to process groups of session sums according to when the sessions complete.</span>
-<span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"WindowToExtractSessionMean"</span><span class="o">,</span> <span class="n">Window</span><span class="o">.<</span><span class="n">Integer</span><span class="o">></span><span class="n">into</span><span class="o">(</span>
- <span class="n">FixedWindows</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getUserActivityWindowDuration</span><span class="o">()))))</span>
+<span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"WindowToExtractSessionMean"</span><span class="o">,</span>
+ <span class="n">Window</span><span class="o">.</span><span class="na">into</span><span class="o">(</span>
+ <span class="n">FixedWindows</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="n">options</span><span class="o">.</span><span class="na">getUserActivityWindowDuration</span><span class="o">()))))</span>
<span class="c1">// Find the mean session duration in each window.</span>
<span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">Mean</span><span class="o">.<</span><span class="n">Integer</span><span class="o">></span><span class="n">globally</span><span class="o">().</span><span class="na">withoutDefaults</span><span class="o">())</span>
<span class="c1">// Write this info to a BigQuery table.</span>
-<span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="s">"WriteAvgSessionLength"</span><span class="o">,</span>
- <span class="k">new</span> <span class="n">WriteWindowedToBigQuery</span><span class="o"><</span><span class="n">Double</span><span class="o">>(</span>
- <span class="n">options</span><span class="o">.</span><span class="na">as</span><span class="o">(</span><span class="n">GcpOptions</span><span class="o">.</span><span class="na">class</span><span class="o">).</span><span class="na">getProject</span><span class="o">(),</span>
- <span class="n">options</span><span class="o">.</span><span class="na">getDataset</span><span class="o">(),</span>
- <span class="n">options</span><span class="o">.</span><span class="na">getGameStatsTablePrefix</span><span class="o">()</span> <span class="o">+</span> <span class="s">"_sessions"</span><span class="o">,</span> <span class="n">configureSessionWindowWrite</span><span class="o">()));</span>
+<span class="o">.</span><span class="na">apply</span><span class="o">(</span>
+ <span class="s">"WriteAvgSessionLength"</span><span class="o">,</span>
+ <span class="k">new</span> <span class="n">WriteWindowedToBigQuery</span><span class="o"><>(</span>
+ <span class="n">options</span><span class="o">.</span><span class="na">as</span><span class="o">(</span><span class="n">GcpOptions</span><span class="o">.</span><span class="na">class</span><span class="o">).</span><span class="na">getProject</span><span class="o">(),</span>
+ <span class="n">options</span><span class="o">.</span><span class="na">getDataset</span><span class="o">(),</span>
+ <span class="n">options</span><span class="o">.</span><span class="na">getGameStatsTablePrefix</span><span class="o">()</span> <span class="o">+</span> <span class="s">"_sessions"</span><span class="o">,</span>
+ <span class="n">configureSessionWindowWrite</span><span class="o">()));</span>
</code></pre>
</div>
<div class="language-py highlighter-rouge"><pre class="highlight"><code><span class="c"># Re-window to process groups of session sums according to when the</span>
--
To stop receiving notification emails like this one, please contact
mergebot-role@apache.org.