You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/03/16 00:02:54 UTC
[beam] branch asf-site updated: Publishing website 2022/03/16 00:01:56 at commit e2ca693
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new af8db99 Publishing website 2022/03/16 00:01:56 at commit e2ca693
af8db99 is described below
commit af8db99b9ca9aa4ec026f0c39456d48af6b0fbc9
Author: jenkins <bu...@apache.org>
AuthorDate: Wed Mar 16 00:01:57 2022 +0000
Publishing website 2022/03/16 00:01:56 at commit e2ca693
---
website/generated-content/documentation/index.xml | 4 -
.../documentation/io/built-in/snowflake/index.html | 4 +-
website/generated-content/get-started/index.xml | 501 +++++++++++----------
.../get-started/quickstart-java/index.html | 315 ++++++-------
website/generated-content/sitemap.xml | 2 +-
5 files changed, 403 insertions(+), 423 deletions(-)
diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml
index d743d96..5c2b7ee 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -1987,7 +1987,6 @@ options.getPrivateKeyPassphrase())
</div>
</li>
</ul>
-<p><strong>Important notice</strong>: Only encrypted private key are supported. Unencrypted (without pasphrase) private key are not supported. For details, see: <a href="https://issues.apache.org/jira/browse/BEAM-13818">BEAM-13818</a>.</p>
<h3 id="oauth-token">OAuth token</h3>
<p>SnowflakeIO also supports OAuth token.</p>
<p><strong>IMPORTANT</strong>: SnowflakeIO requires a valid OAuth access token. It will neither be able to refresh the token nor obtain it using a web-based flow. For information on configuring an OAuth integration and obtaining the token, see the <a href="https://docs.snowflake.com/en/user-guide/oauth-intro.html">Snowflake documentation</a>.</p>
@@ -3177,9 +3176,6 @@ You can read about Snowflake data types at <a href="https://docs.snowflake.co
<p>Streaming writing supports only pair key authentication. For details, see: <a href="https://issues.apache.org/jira/browse/BEAM-13817">BEAM-13817</a>.</p>
</li>
<li>
-<p>Only encrypted private key are supported. Unencrypted private key are not supported. For details, see: <a href="https://issues.apache.org/jira/browse/BEAM-13818">BEAM-13818</a>.</p>
-</li>
-<li>
<p>The role parameter configured in <code>SnowflakeIO.DataSourceConfiguration</code> object is ignored for streaming writing. For details, see: <a href="https://issues.apache.org/jira/browse/BEAM-13819">BEAM-13819</a></p>
</li>
</ol></description></item><item><title>Documentation: ApproximateQuantiles</title><link>/documentation/transforms/java/aggregation/approximatequantiles/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>/documentation/transforms/java/aggregation/approximatequantiles/</guid><description>
diff --git a/website/generated-content/documentation/io/built-in/snowflake/index.html b/website/generated-content/documentation/io/built-in/snowflake/index.html
index c497791..707b35f 100644
--- a/website/generated-content/documentation/io/built-in/snowflake/index.html
+++ b/website/generated-content/documentation/io/built-in/snowflake/index.html
@@ -54,7 +54,7 @@ function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfi
.withWarehouse(options.getWarehouse())
.withSchema(options.getSchema());
- </code></pre></div></div></li></ul><p><strong>Important notice</strong>: Only encrypted private key are supported. Unencrypted (without pasphrase) private key are not supported. For details, see: <a href=https://issues.apache.org/jira/browse/BEAM-13818>BEAM-13818</a>.</p><h3 id=oauth-token>OAuth token</h3><p>SnowflakeIO also supports OAuth token.</p><p><strong>IMPORTANT</strong>: SnowflakeIO requires a valid OAuth access token. It will neither be able to refresh the token nor obtain it [...]
+ </code></pre></div></div></li></ul><h3 id=oauth-token>OAuth token</h3><p>SnowflakeIO also supports OAuth token.</p><p><strong>IMPORTANT</strong>: SnowflakeIO requires a valid OAuth access token. It will neither be able to refresh the token nor obtain it using a web-based flow. For information on configuring an OAuth integration and obtaining the token, see the <a href=https://docs.snowflake.com/en/user-guide/oauth-intro.html>Snowflake documentation</a>.</p><p>Once you have the token, i [...]
.create()
.withUrl(options.getUrl())
.withServerName(options.getServerName())
@@ -311,7 +311,7 @@ Example:<div class=snippet><div class="notebook-skip code-snippet without_switch
{"type":"string","length":null},
{"type":"text","length":null},
{"type":"varbinary","size":null},
-{"type":"varchar","length":100}]</code></pre></div></div>You can read about Snowflake data types at <a href=https://docs.snowflake.com/en/sql-reference/data-types.html>Snowflake data types</a>.</p></li><li><p><code>expansion_service</code> Specifies URL of expansion service.</p></li></ul><h2 id=limitations>Limitations</h2><p>SnowflakeIO currently has the following limitations.</p><ol><li><p>Streaming writing supports only pair key authentication. For details, see: [...]
+{"type":"varchar","length":100}]</code></pre></div></div>You can read about Snowflake data types at <a href=https://docs.snowflake.com/en/sql-reference/data-types.html>Snowflake data types</a>.</p></li><li><p><code>expansion_service</code> Specifies URL of expansion service.</p></li></ul><h2 id=limitations>Limitations</h2><p>SnowflakeIO currently has the following limitations.</p><ol><li><p>Streaming writing supports only pair key authentication. For details, see: [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div></div></div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/get-started/index.xml b/website/generated-content/get-started/index.xml
index 55fc830..15be962 100644
--- a/website/generated-content/get-started/index.xml
+++ b/website/generated-content/get-started/index.xml
@@ -1124,45 +1124,53 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
-<h1 id="apache-beam-java-sdk-quickstart">Apache Beam Java SDK Quickstart</h1>
-<p>This quickstart shows you how to set up a Java development environment and run an <a href="/get-started/wordcount-example">example pipeline</a> written with the <a href="/documentation/sdks/java">Apache Beam Java SDK</a>, using a <a href="/documentation#runners">runner</a> of your choice.</p>
-<p>If you&rsquo;re interested in contributing to the Apache Beam Java codebase, see the <a href="/contribute">Contribution Guide</a>.</p>
+<h1 id="apache-beam-java-sdk-quickstart">Apache Beam Java SDK quickstart</h1>
+<p>This quickstart shows you how to set up a Java development environment and run
+an <a href="/get-started/wordcount-example">example pipeline</a> written with the
+<a href="/documentation/sdks/java">Apache Beam Java SDK</a>, using a
+<a href="/documentation#runners">runner</a> of your choice.</p>
+<p>If you&rsquo;re interested in contributing to the Apache Beam Java codebase, see the
+<a href="/contribute">Contribution Guide</a>.</p>
+<p>On this page:</p>
<nav id="TableOfContents">
<ul>
-<li><a href="#set-up-your-development-environment">Set up your Development Environment</a></li>
-<li><a href="#get-the-example-code">Get the Example Code</a></li>
-<li><a href="#optional-convert-from-maven-to-gradle-project">Optional: Convert from Maven to Gradle Project</a></li>
+<li><a href="#set-up-your-development-environment">Set up your development environment</a></li>
+<li><a href="#get-the-example-code">Get the example code</a></li>
+<li><a href="#optional-convert-from-maven-to-gradle">Optional: Convert from Maven to Gradle</a></li>
<li><a href="#get-sample-text">Get sample text</a></li>
<li><a href="#run-a-pipeline">Run a pipeline</a>
<ul>
-<li><a href="#run-wordcount-using-maven">Run WordCount Using Maven</a></li>
-<li><a href="#run-wordcount-using-gradle">Run WordCount Using Gradle</a></li>
+<li><a href="#run-wordcount-using-maven">Run WordCount using Maven</a></li>
+<li><a href="#run-wordcount-using-gradle">Run WordCount using Gradle</a></li>
</ul>
</li>
<li><a href="#inspect-the-results">Inspect the results</a></li>
<li><a href="#next-steps">Next Steps</a></li>
</ul>
</nav>
-<h2 id="set-up-your-development-environment">Set up your Development Environment</h2>
+<h2 id="set-up-your-development-environment">Set up your development environment</h2>
<ol>
-<li>
-<p>Download and install the <a href="https://www.oracle.com/technetwork/java/javase/downloads/index.html">Java Development Kit (JDK)</a> version 8. Verify that the <a href="https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/envvars001.html">JAVA_HOME</a> environment variable is set and points to your JDK installation.</p>
-</li>
-<li>
-<p>Download and install <a href="https://maven.apache.org/download.cgi">Apache Maven</a> by following Maven&rsquo;s <a href="https://maven.apache.org/install.html">installation guide</a> for your specific operating system.</p>
-</li>
-<li>
-<p>Optional: Install <a href="https://gradle.org/install/">Gradle</a> if you would like to convert your Maven project into Gradle.</p>
-</li>
+<li>Download and install the
+<a href="https://www.oracle.com/technetwork/java/javase/downloads/index.html">Java Development Kit (JDK)</a>
+version 8, 11, or 17. Verify that the
+<a href="https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/envvars001.html">JAVA_HOME</a>
+environment variable is set and points to your JDK installation.</li>
+<li>Download and install <a href="https://maven.apache.org/download.cgi">Apache Maven</a> by
+following the <a href="https://maven.apache.org/install.html">installation guide</a>
+for your operating system.</li>
+<li>Optional: If you want to convert your Maven project to Gradle, install
+<a href="https://gradle.org/install/">Gradle</a>.</li>
</ol>
-<h2 id="get-the-example-code">Get the Example Code</h2>
-<p>Use the following command to generate a Maven project that contains Beam&rsquo;s WordCount examples and builds against the most recent Beam release:</p>
+<h2 id="get-the-example-code">Get the example code</h2>
+<ol>
+<li>
+<p>Generate a Maven example project that builds against the latest Beam release:
<div class='shell-unix snippet'>
<div class="notebook-skip code-snippet">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-unix" data-lang="unix">$ mvn archetype:generate \
+<pre><code class="language-unix" data-lang="unix">mvn archetype:generate \
-DarchetypeGroupId=org.apache.beam \
-DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
-DarchetypeVersion=2.37.0 \
@@ -1170,7 +1178,8 @@ limitations under the License.
-DartifactId=word-count-beam \
-Dversion=&#34;0.1&#34; \
-Dpackage=org.apache.beam.examples \
--DinteractiveMode=false</code></pre>
+-DinteractiveMode=false
+</code></pre>
</div>
</div>
<div class='shell-powerShell snippet'>
@@ -1178,7 +1187,7 @@ limitations under the License.
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<div class="highlight"><pre class="chroma"><code class="language-powerShell" data-lang="powerShell"><span class="n">PS</span><span class="p">&gt;</span> <span class="n">mvn</span> <span class="n">archetype</span><span class="err">:</span><span class="n">generate</span> <span class="p">`</span>
+<div class="highlight"><pre class="chroma"><code class="language-powerShell" data-lang="powerShell"><span class="n">mvn</span> <span class="n">archetype</span><span class="err">:</span><span class="n">generate</span> <span class="p">`</span>
<span class="n">-D</span> <span class="n">archetypeGroupId</span><span class="p">=</span><span class="n">org</span><span class="p">.</span><span class="n">apache</span><span class="p">.</span><span class="n">beam</span> <span class="p">`</span>
<span class="n">-D</span> <span class="n">archetypeArtifactId</span><span class="p">=</span><span class="n">beam-sdks-java-maven-archetypes-examples</span> <span class="p">`</span>
<span class="n">-D</span> <span class="n">archetypeVersion</span><span class="p">=</span><span class="n">2</span><span class="p">.</span><span class="n">37</span><span class="p">.</span><span class="n">0</span> <span class="p">`</span>
@@ -1186,21 +1195,22 @@ limitations under the License.
<span class="n">-D</span> <span class="n">artifactId</span><span class="p">=</span><span class="n">word-count-beam</span> <span class="p">`</span>
<span class="n">-D</span> <span class="n">version</span><span class="p">=</span><span class="s2">&#34;0.1&#34;</span> <span class="p">`</span>
<span class="n">-D</span> <span class="n">package</span><span class="p">=</span><span class="n">org</span><span class="p">.</span><span class="n">apache</span><span class="p">.</span><span class="n">beam</span><span class="p">.</span><span class="n">examples</span> <span class="p">`</span>
-<span class="n">-D</span> <span class="n">interactiveMode</span><span class="p">=</span><span class="n">false</span></code></pre></div>
+<span class="n">-D</span> <span class="n">interactiveMode</span><span class="p">=</span><span class="n">false</span>
+</code></pre></div>
</div>
</div>
-<p>This will create a <code>word-count-beam</code> directory that contains a <code>pom.xml</code> and several example pipelines that count words in text files.</p>
+</p>
+<p>Maven creates a new project in the <strong>word-count-beam</strong> directory.</p>
+</li>
+<li>
+<p>Change into <strong>word-count-beam</strong>:
<div class='shell-unix snippet'>
<div class="notebook-skip code-snippet">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-unix" data-lang="unix">$ cd word-count-beam/
-$ ls
-pom.xml src
-$ ls src/main/java/org/apache/beam/examples/
-DebuggingWordCount.java WindowedWordCount.java common
-MinimalWordCount.java WordCount.java</code></pre>
+<pre><code class="language-unix" data-lang="unix">cd word-count-beam/
+</code></pre>
</div>
</div>
<div class='shell-powerShell snippet'>
@@ -1208,116 +1218,189 @@ MinimalWordCount.java WordCount.java</code></pre>
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<div class="highlight"><pre class="chroma"><code class="language-powerShell" data-lang="powerShell"><span class="n">PS</span><span class="p">&gt;</span> <span class="nb">cd </span><span class="p">.\</span><span class="n">word-count-beam</span>
-<span class="n">PS</span><span class="p">&gt;</span> <span class="nb">dir
-</span><span class="nb"></span>
-<span class="p">...</span>
-<span class="n">Mode</span> <span class="n">LastWriteTime</span> <span class="n">Length</span> <span class="n">Name</span>
-<span class="p">----</span> <span class="p">-------------</span> <span class="p">------</span> <span class="p">----</span>
-<span class="n">d</span><span class="p">-----</span> <span class="n">7</span><span class="p">/</span><span class="n">19</span><span class="p">/</span><span class="n">2018</span> <span class="n">11</span><span class="err">:</span><span class="n">00</span> <span class="n">PM</span> <span class="n">src</span>
-<span class="n">-a</span><span class="p">----</span> <span class="n">7</span><span class="p">/</span><span class="n">19</span><span class="p">/</span><span class="n">2018</span> <span class="n">11</span><span class="err">:</span><span class="n">00</span> <span class="n">PM</span> <span class="n">16051</span> <span class="n">pom</span><span class="p">.</span><span class="n">xml</span>
-<span class="n">PS</span><span class="p">&gt;</span> <span class="nb">dir </span><span class="p">.\</span><span class="n">src</span><span class="p">\</span><span class="n">main</span><span class="p">\</span><span class="n">java</span><span class="p">\</span><span class="n">org</span><span class="p">\</span><span class="n">apache</span><span class="p">\</span><span class="n">beam</span><span c [...]
-<span class="p">...</span>
-<span class="n">Mode</span> <span class="n">LastWriteTime</span> <span class="n">Length</span> <span class="n">Name</span>
-<span class="p">----</span> <span class="p">-------------</span> <span class="p">------</span> <span class="p">----</span>
-<span class="n">d</span><span class="p">-----</span> <span class="n">7</span><span class="p">/</span><span class="n">19</span><span class="p">/</span><span class="n">2018</span> <span class="n">11</span><span class="err">:</span><span class="n">00</span> <span class="n">PM</span> <span class="n">common</span>
-<span class="n">d</span><span class="p">-----</span> <span class="n">7</span><span class="p">/</span><span class="n">19</span><span class="p">/</span><span class="n">2018</span> <span class="n">11</span><span class="err">:</span><span class="n">00</span> <span class="n">PM</span> <span class="n">complete</span>
-<span class="n">d</span><span class="p">-----</span> <span class="n">7</span><span class="p">/</span><span class="n">19</span><span class="p">/</span><span class="n">2018</span> <span class="n">11</span><span class="err">:</span><span class="n">00</span> <span class="n">PM</span> <span class="n">subprocess</span>
-<span class="n">-a</span><span class="p">----</span> <span class="n">7</span><span class="p">/</span><span class="n">19</span><span class="p">/</span><span class="n">2018</span> <span class="n">11</span><span class="err">:</span><span class="n">00</span> <span class="n">PM</span> <span class="n">7073</span> <span class="n">DebuggingWordCount</span><span class="p">.</span><span class="n">java</span>
-<span class="n">-a</span><span class="p">----</span> <span class="n">7</span><span class="p">/</span><span class="n">19</span><span class="p">/</span><span class="n">2018</span> <span class="n">11</span><span class="err">:</span><span class="n">00</span> <span class="n">PM</span> <span class="n">5945</span> <span class="n">MinimalWordCount</span><span class="p">.</span><span class="n">java</span>
-<span class="n">-a</span><span class="p">----</span> <span class="n">7</span><span class="p">/</span><span class="n">19</span><span class="p">/</span><span class="n">2018</span> <span class="n">11</span><span class="err">:</span><span class="n">00</span> <span class="n">PM</span> <span class="n">9490</span> <span class="n">WindowedWordCount</span><span class="p">.</span><span class="n">java</span>
-<span class="n">-a</span><span class="p">----</span> <span class="n">7</span><span class="p">/</span><span class="n">19</span><span class="p">/</span><span class="n">2018</span> <span class="n">11</span><span class="err">:</span><span class="n">00</span> <span class="n">PM</span> <span class="n">7662</span> <span class="n">WordCount</span><span class="p">.</span><span class="n">java</span></code> [...]
-</div>
-</div>
-<p>For a detailed introduction to the Beam concepts used in these examples, see the <a href="/get-started/wordcount-example">WordCount Example Walkthrough</a>. Here, we&rsquo;ll just focus on executing <code>WordCount.java</code>.</p>
-<h2 id="optional-convert-from-maven-to-gradle-project">Optional: Convert from Maven to Gradle Project</h2>
-<p>The steps below explain how to convert the build for the Direct Runner from Maven to Gradle. Converting the builds for the other runners is a more involved process and is out of scope for this guide. For additional guidance, see <a href="https://docs.gradle.org/current/userguide/migrating_from_maven.html">Migrating Builds From Apache Maven</a>.</p>
+<div class="highlight"><pre class="chroma"><code class="language-powerShell" data-lang="powerShell"><span class="nb">cd </span><span class="p">.\</span><span class="n">word-count-beam</span>
+</code></pre></div>
+</div>
+</div>
+The directory contains a <strong>pom.xml</strong> and a <strong>src</strong> directory with example
+pipelines.</p>
+</li>
+<li>
+<p>List the example pipelines:
+<div class='shell-unix snippet'>
+<div class="notebook-skip code-snippet">
+<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
+<img src="/images/copy-icon.svg"/>
+</a>
+<pre><code class="language-unix" data-lang="unix">ls src/main/java/org/apache/beam/examples/
+</code></pre>
+</div>
+</div>
+<div class='shell-powerShell snippet'>
+<div class="notebook-skip code-snippet">
+<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
+<img src="/images/copy-icon.svg"/>
+</a>
+<div class="highlight"><pre class="chroma"><code class="language-powerShell" data-lang="powerShell"><span class="nb">dir </span><span class="p">.\</span><span class="n">src</span><span class="p">\</span><span class="n">main</span><span class="p">\</span><span class="n">java</span><span class="p">\</span><span class="n">org</span><span class="p">\</span><span class="n">apache</span><span class="p">\</span>< [...]
+</code></pre></div>
+</div>
+</div>
+You should see the following examples:</p>
+<ul>
+<li><strong>DebuggingWordCount.java</strong> (<a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/DebuggingWordCount.java">GitHub</a>)</li>
+<li><strong>MinimalWordCount.java</strong> (<a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java">GitHub</a>)</li>
+<li><strong>WindowedWordCount.java</strong> (<a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/WindowedWordCount.java">GitHub</a>)</li>
+<li><strong>WordCount.java</strong> (<a href="https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/WordCount.java">GitHub</a>)</li>
+</ul>
+<p>The example used in this tutorial, <strong>WordCount.java</strong>, defines a
+Beam pipeline that counts words from an input file (by default, a <strong>.txt</strong>
+file containing Shakespeare&rsquo;s &ldquo;King Lear&rdquo;). To learn more about the examples,
+see the <a href="/get-started/wordcount-example">WordCount Example Walkthrough</a>.</p>
+</li>
+</ol>
+<h2 id="optional-convert-from-maven-to-gradle">Optional: Convert from Maven to Gradle</h2>
+<p>The steps below explain how to convert the build from Maven to Gradle for the
+following runners:</p>
+<ul>
+<li>Direct runner</li>
+<li>Dataflow runner</li>
+</ul>
+<p>The conversion process for other runners is similar. For additional guidance,
+see
+<a href="https://docs.gradle.org/current/userguide/migrating_from_maven.html">Migrating Builds From Apache Maven</a>.</p>
<ol>
-<li>Ensure you are in the same directory as the <code>pom.xml</code> file generated from the previous step. Automatically convert your project from Maven to Gradle by running:
+<li>In the directory with the <strong>pom.xml</strong> file, run the automated Maven-to-Gradle
+conversion:
<div class="snippet">
<div class="notebook-skip code-snippet without_switcher">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code>$ gradle init</code></pre>
+<pre><code>gradle init
+</code></pre>
</div>
</div>
-You&rsquo;ll be asked if you want to generate a Gradle build. Enter <strong>yes</strong>. You&rsquo;ll also be prompted to choose a DSL (Groovy or Kotlin). This tutorial uses Groovy, so select that if you don&rsquo;t have a preference.</li>
-<li>After you&rsquo;ve converted the project to Gradle, open the generated <code>build.gradle</code> file, and, in the <code>repositories</code> block, replace <code>mavenLocal()</code> with <code>mavenCentral()</code>:
+You&rsquo;ll be asked if you want to generate a Gradle build. Enter <strong>yes</strong>. You&rsquo;ll
+also be prompted to choose a DSL (Groovy or Kotlin). For this tutorial, enter
+<strong>2</strong> for Kotlin.</li>
+<li>Open the generated <strong>build.gradle.kts</strong> file and make the following changes:
+<ol>
+<li>In <code>repositories</code>, replace <code>mavenLocal()</code> with <code>mavenCentral()</code>.</li>
+<li>In <code>repositories</code>, declare a repository for Confluent Kafka dependencies:
+<div class="snippet">
+<div class="notebook-skip code-snippet without_switcher">
+<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
+<img src="/images/copy-icon.svg"/>
+</a>
+<pre><code>maven {
+url = uri(&#34;https://packages.confluent.io/maven/&#34;)
+}
+</code></pre>
+</div>
+</div>
+</li>
+<li>At the end of the build script, add the following conditional dependency:
<div class="snippet">
<div class="notebook-skip code-snippet without_switcher">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code>repositories {
-mavenCentral()
-maven {
-url = uri(&#39;https://repository.apache.org/content/repositories/snapshots/&#39;)
+<pre><code>if (project.hasProperty(&#34;dataflow-runner&#34;)) {
+dependencies {
+runtimeOnly(&#34;org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0&#34;)
}
-maven {
-url = uri(&#39;http://repo.maven.apache.org/maven2&#39;)
}
-}</code></pre>
+</code></pre>
</div>
</div>
</li>
-<li>Add the following task in <code>build.gradle</code> to allow you to execute pipelines with Gradle:
+<li>At the end of the build script, add the following task:
<div class="snippet">
<div class="notebook-skip code-snippet without_switcher">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code>task execute (type:JavaExec) {
-mainClass = System.getProperty(&#34;mainClass&#34;)
-classpath = sourceSets.main.runtimeClasspath
-systemProperties System.getProperties()
-args System.getProperty(&#34;exec.args&#34;, &#34;&#34;).split()
-}</code></pre>
+<pre><code>task(&#34;execute&#34;, JavaExec::class) {
+classpath = sourceSets[&#34;main&#34;].runtimeClasspath
+mainClass.set(System.getProperty(&#34;mainClass&#34;))
+}
+</code></pre>
</div>
</div>
</li>
-<li>Rebuild your project by running:
+</ol>
+</li>
+<li>Build your project:
<div class="snippet">
<div class="notebook-skip code-snippet without_switcher">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code>$ gradle build</code></pre>
+<pre><code>gradle build
+</code></pre>
</div>
</div>
</li>
</ol>
<h2 id="get-sample-text">Get sample text</h2>
<blockquote>
-<p>If you&rsquo;re planning to use the DataflowRunner, you can skip this step. The runner will pull text directly from Google Cloud Storage.</p>
+<p>If you&rsquo;re planning to use the DataflowRunner, you can skip this step. The
+runner will pull text directly from Google Cloud Storage.</p>
</blockquote>
<ol>
<li>In the <strong>word-count-beam</strong> directory, create a file called <strong>sample.txt</strong>.</li>
-<li>Add some text to the file. For this example, you can use the text of Shakespeare&rsquo;s <a href="https://storage.cloud.google.com/apache-beam-samples/shakespeare/sonnets.txt">Sonnets</a>.</li>
+<li>Add some text to the file. For this example, use the text of Shakespeare&rsquo;s
+<a href="https://storage.cloud.google.com/apache-beam-samples/shakespeare/kinglear.txt">King Lear</a>.</li>
</ol>
<h2 id="run-a-pipeline">Run a pipeline</h2>
-<p>A single Beam pipeline can run on multiple Beam <a href="/documentation#runners">runners</a>, including the <a href="/documentation/runners/flink">FlinkRunner</a>, <a href="/documentation/runners/spark">SparkRunner</a>, <a href="/documentation/runners/nemo">NemoRunner</a>, <a href="/documentation/runners/jet">JetRunner</a>, or <a href="/documentation/runners/dataflow">DataflowRunner</a>. The <a href="/documentation/runners/direct">DirectRunner [...]
+<p>A single Beam pipeline can run on multiple Beam
+<a href="/documentation#runners">runners</a>. The
+<a href="/documentation/runners/direct">DirectRunner</a> is useful for getting started,
+because it runs on your machine and requires no specific setup. If you&rsquo;re just
+trying out Beam and you&rsquo;re not sure what to use, use the
+<a href="/documentation/runners/direct">DirectRunner</a>.</p>
<p>The general process for running a pipeline goes like this:</p>
<ol>
-<li>Ensure you&rsquo;ve done any runner-specific setup.</li>
+<li>Complete any runner-specific setup.</li>
<li>Build your command line:
<ol>
-<li>Specify a runner with <code>--runner=&lt;runner&gt;</code> (defaults to the <a href="/documentation/runners/direct">DirectRunner</a>).</li>
+<li>Specify a runner with <code>--runner=&lt;runner&gt;</code> (defaults to the
+<a href="/documentation/runners/direct">DirectRunner</a>).</li>
<li>Add any runner-specific required options.</li>
-<li>Choose input files and an output location that are accessible to the runner. (For example, you can&rsquo;t access a local file if you are running the pipeline on an external cluster.)</li>
+<li>Choose input files and an output location that are accessible to the
+runner. (For example, you can&rsquo;t access a local file if you are running
+the pipeline on an external cluster.)</li>
</ol>
</li>
<li>Run the command.</li>
</ol>
-<p>To run the WordCount pipeline, see the Maven and Gradle examples below.</p>
-<h3 id="run-wordcount-using-maven">Run WordCount Using Maven</h3>
+<p>To run the WordCount pipeline:</p>
+<ol>
+<li>
+<p>Follow the setup steps for your runner:</p>
+<ul>
+<li><a href="/documentation/runners/flink">FlinkRunner</a></li>
+<li><a href="/documentation/runners/spark">SparkRunner</a></li>
+<li><a href="/documentation/runners/dataflow">DataflowRunner</a></li>
+<li><a href="/documentation/runners/samza">SamzaRunner</a></li>
+<li><a href="/documentation/runners/nemo">NemoRunner</a></li>
+<li><a href="/documentation/runners/jet">JetRunner</a></li>
+</ul>
+<p>The DirectRunner will work without additional setup.</p>
+</li>
+<li>
+<p>Run the corresponding Maven or Gradle command below.</p>
+</li>
+</ol>
+<h3 id="run-wordcount-using-maven">Run WordCount using Maven</h3>
<p>For Unix shells:</p>
+<p>
<div class='runner-direct snippet'>
<div class="notebook-skip code-snippet">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-direct" data-lang="direct">$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+<pre><code class="language-direct" data-lang="direct">mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args=&#34;--inputFile=sample.txt --output=counts&#34; -Pdirect-runner</code></pre>
</div>
</div>
@@ -1326,7 +1409,7 @@ args System.getProperty(&#34;exec.args&#34;, &#34;&#34;).split()
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-flink" data-lang="flink">$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+<pre><code class="language-flink" data-lang="flink">mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args=&#34;--runner=FlinkRunner --inputFile=sample.txt --output=counts&#34; -Pflink-runner</code></pre>
</div>
</div>
@@ -1335,10 +1418,9 @@ args System.getProperty(&#34;exec.args&#34;, &#34;&#34;).split()
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-flinkCluster" data-lang="flinkCluster">$ mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+<pre><code class="language-flinkCluster" data-lang="flinkCluster">mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args=&#34;--runner=FlinkRunner --flinkMaster=&lt;flink master&gt; --filesToStage=target/word-count-beam-bundled-0.1.jar \
---inputFile=sample.txt --output=/tmp/counts&#34; -Pflink-runner
-You can monitor the running job by visiting the Flink dashboard at http://&lt;flink master&gt;:8081</code></pre>
+--inputFile=sample.txt --output=/tmp/counts&#34; -Pflink-runner</code></pre>
</div>
</div>
<div class='runner-spark snippet'>
@@ -1346,7 +1428,7 @@ You can monitor the running job by visiting the Flink dashboard at http://&l
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-spark" data-lang="spark">$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+<pre><code class="language-spark" data-lang="spark">mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args=&#34;--runner=SparkRunner --inputFile=sample.txt --output=counts&#34; -Pspark-runner</code></pre>
</div>
</div>
@@ -1355,8 +1437,7 @@ You can monitor the running job by visiting the Flink dashboard at http://&l
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-dataflow" data-lang="dataflow">Make sure you complete the setup steps at /documentation/runners/dataflow/#setup
-$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+<pre><code class="language-dataflow" data-lang="dataflow">mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args=&#34;--runner=DataflowRunner --project=&lt;your-gcp-project&gt; \
--region=&lt;your-gcp-region&gt; \
--gcpTempLocation=gs://&lt;your-gcs-bucket&gt;/tmp \
@@ -1369,7 +1450,7 @@ $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-samza" data-lang="samza">$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+<pre><code class="language-samza" data-lang="samza">mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args=&#34;--inputFile=sample.txt --output=/tmp/counts --runner=SamzaRunner&#34; -Psamza-runner</code></pre>
</div>
</div>
@@ -1378,7 +1459,7 @@ $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-nemo" data-lang="nemo">$ mvn package -Pnemo-runner &amp;&amp; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount \
+<pre><code class="language-nemo" data-lang="nemo">mvn package -Pnemo-runner &amp;&amp; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount \
--runner=NemoRunner --inputFile=`pwd`/sample.txt --output=counts</code></pre>
</div>
</div>
@@ -1387,18 +1468,20 @@ $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-jet" data-lang="jet">$ mvn package -Pjet-runner
-$ java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount \
+<pre><code class="language-jet" data-lang="jet">mvn package -Pjet-runner
+java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount \
--runner=JetRunner --jetLocalMode=3 --inputFile=`pwd`/sample.txt --output=counts</code></pre>
</div>
</div>
+</p>
<p>For Windows PowerShell:</p>
+<p>
<div class='runner-direct snippet'>
<div class="notebook-skip code-snippet">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-direct" data-lang="direct">PS&gt; mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+<pre><code class="language-direct" data-lang="direct">mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args=&#34;--inputFile=sample.txt --output=counts&#34; -P direct-runner</code></pre>
</div>
</div>
@@ -1407,7 +1490,7 @@ $ java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordC
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-flink" data-lang="flink">PS&gt; mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+<pre><code class="language-flink" data-lang="flink">mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args=&#34;--runner=FlinkRunner --inputFile=sample.txt --output=counts&#34; -P flink-runner</code></pre>
</div>
</div>
@@ -1416,10 +1499,9 @@ $ java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordC
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-flinkCluster" data-lang="flinkCluster">PS&gt; mvn package exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+<pre><code class="language-flinkCluster" data-lang="flinkCluster">mvn package exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args=&#34;--runner=FlinkRunner --flinkMaster=&lt;flink master&gt; --filesToStage=.\target\word-count-beam-bundled-0.1.jar `
---inputFile=C:\path\to\quickstart\sample.txt --output=C:\tmp\counts&#34; -P flink-runner
-You can monitor the running job by visiting the Flink dashboard at http://&lt;flink master&gt;:8081</code></pre>
+--inputFile=C:\path\to\quickstart\sample.txt --output=C:\tmp\counts&#34; -P flink-runner</code></pre>
</div>
</div>
<div class='runner-spark snippet'>
@@ -1427,7 +1509,7 @@ You can monitor the running job by visiting the Flink dashboard at http://&l
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-spark" data-lang="spark">PS&gt; mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+<pre><code class="language-spark" data-lang="spark">mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args=&#34;--runner=SparkRunner --inputFile=sample.txt --output=counts&#34; -P spark-runner</code></pre>
</div>
</div>
@@ -1436,8 +1518,7 @@ You can monitor the running job by visiting the Flink dashboard at http://&l
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-dataflow" data-lang="dataflow">Make sure you complete the setup steps at /documentation/runners/dataflow/#setup
-PS&gt; mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+<pre><code class="language-dataflow" data-lang="dataflow">mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args=&#34;--runner=DataflowRunner --project=&lt;your-gcp-project&gt; `
--region=&lt;your-gcp-region&gt; \
--gcpTempLocation=gs://&lt;your-gcs-bucket&gt;/tmp `
@@ -1450,7 +1531,7 @@ PS&gt; mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.Word
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-samza" data-lang="samza">PS&gt; mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+<pre><code class="language-samza" data-lang="samza">mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args=&#34;--inputFile=sample.txt --output=/tmp/counts --runner=SamzaRunner&#34; -P samza-runner</code></pre>
</div>
</div>
@@ -1459,8 +1540,8 @@ PS&gt; mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.Word
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-nemo" data-lang="nemo">PS&gt; mvn package -P nemo-runner -DskipTests
-PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
+<pre><code class="language-nemo" data-lang="nemo">mvn package -P nemo-runner -DskipTests
+java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
--runner=NemoRunner --inputFile=`pwd`/sample.txt --output=counts</code></pre>
</div>
</div>
@@ -1469,20 +1550,22 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-jet" data-lang="jet">PS&gt; mvn package -P jet-runner
-PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
+<pre><code class="language-jet" data-lang="jet">mvn package -P jet-runner
+java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
--runner=JetRunner --jetLocalMode=3 --inputFile=$pwd/sample.txt --output=counts</code></pre>
</div>
</div>
-<h3 id="run-wordcount-using-gradle">Run WordCount Using Gradle</h3>
-<p>For Unix shells (Instructions currently only available for Direct, Spark, and Dataflow):</p>
+</p>
+<h3 id="run-wordcount-using-gradle">Run WordCount using Gradle</h3>
+<p>For Unix shells:</p>
+<p>
<div class='runner-direct snippet'>
<div class="notebook-skip code-snippet">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-direct" data-lang="direct">$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
--Dexec.args=&#34;--inputFile=sample.txt --output=counts&#34; -Pdirect-runner</code></pre>
+<pre><code class="language-direct" data-lang="direct">gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
+--args=&#34;--inputFile=sample.txt --output=counts&#34;</code></pre>
</div>
</div>
<div class='runner-flink snippet'>
@@ -1490,7 +1573,7 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-flink" data-lang="flink">We are working on adding the instruction for this runner!</code></pre>
+<pre><code class="language-flink" data-lang="flink">TODO: document Flink on Gradle: https://issues.apache.org/jira/browse/BEAM-14057</code></pre>
</div>
</div>
<div class='runner-flinkCluster snippet'>
@@ -1498,7 +1581,7 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-flinkCluster" data-lang="flinkCluster">We are working on adding the instruction for this runner!</code></pre>
+<pre><code class="language-flinkCluster" data-lang="flinkCluster">TODO: document FlinkCluster on Gradle: https://issues.apache.org/jira/browse/BEAM-14060</code></pre>
</div>
</div>
<div class='runner-spark snippet'>
@@ -1506,8 +1589,7 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-spark" data-lang="spark">$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
--Dexec.args=&#34;--inputFile=sample.txt --output=counts&#34; -Pspark-runner</code></pre>
+<pre><code class="language-spark" data-lang="spark">TODO: document Spark on Gradle: https://issues.apache.org/jira/browse/BEAM-14063</code></pre>
</div>
</div>
<div class='runner-dataflow snippet'>
@@ -1515,8 +1597,8 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-dataflow" data-lang="dataflow">$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
--Dexec.args=&#34;--project=&lt;your-gcp-project&gt; --inputFile=gs://apache-beam-samples/shakespeare/* \
+<pre><code class="language-dataflow" data-lang="dataflow">gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
+--args=&#34;--project=&lt;your-gcp-project&gt; --inputFile=gs://apache-beam-samples/shakespeare/* \
--output=gs://&lt;your-gcs-bucket&gt;/counts&#34; -Pdataflow-runner</code></pre>
</div>
</div>
@@ -1525,7 +1607,7 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-samza" data-lang="samza">We are working on adding the instruction for this runner!</code></pre>
+<pre><code class="language-samza" data-lang="samza">TODO: document Samza on Gradle: https://issues.apache.org/jira/browse/BEAM-14061</code></pre>
</div>
</div>
<div class='runner-nemo snippet'>
@@ -1533,7 +1615,7 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-nemo" data-lang="nemo">We are working on adding the instruction for this runner!</code></pre>
+<pre><code class="language-nemo" data-lang="nemo">TODO: document Nemo on Gradle: https://issues.apache.org/jira/browse/BEAM-14058</code></pre>
</div>
</div>
<div class='runner-jet snippet'>
@@ -1541,17 +1623,23 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-jet" data-lang="jet">We are working on adding the instruction for this runner!</code></pre>
+<pre><code class="language-jet" data-lang="jet">TODO: document Jet on Gradle: https://issues.apache.org/jira/browse/BEAM-14062</code></pre>
</div>
</div>
+</p>
<h2 id="inspect-the-results">Inspect the results</h2>
-<p>Once the pipeline has completed, you can view the output. You&rsquo;ll notice that there may be multiple output files prefixed by <code>count</code>. The exact number of these files is decided by the runner, giving it the flexibility to do efficient, distributed execution.</p>
+<p>After the pipeline has completed, you can view the output. There might be
+multiple output files prefixed by <code>count</code>. The number of output files is decided
+by the runner, giving it the flexibility to do efficient, distributed execution.</p>
+<ol>
+<li>View the output files in a Unix shell:
<div class='runner-direct snippet'>
<div class="notebook-skip code-snippet">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-direct" data-lang="direct">$ ls counts*</code></pre>
+<pre><code class="language-direct" data-lang="direct">ls counts*
+</code></pre>
</div>
</div>
<div class='runner-flink snippet'>
@@ -1559,7 +1647,8 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-flink" data-lang="flink">$ ls counts*</code></pre>
+<pre><code class="language-flink" data-lang="flink">ls counts*
+</code></pre>
</div>
</div>
<div class='runner-flinkCluster snippet'>
@@ -1567,7 +1656,8 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-flinkCluster" data-lang="flinkCluster">$ ls /tmp/counts*</code></pre>
+<pre><code class="language-flinkCluster" data-lang="flinkCluster">ls /tmp/counts*
+</code></pre>
</div>
</div>
<div class='runner-spark snippet'>
@@ -1575,7 +1665,8 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-spark" data-lang="spark">$ ls counts*</code></pre>
+<pre><code class="language-spark" data-lang="spark">ls counts*
+</code></pre>
</div>
</div>
<div class='runner-dataflow snippet'>
@@ -1583,7 +1674,8 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-dataflow" data-lang="dataflow">$ gsutil ls gs://&lt;your-gcs-bucket&gt;/counts*</code></pre>
+<pre><code class="language-dataflow" data-lang="dataflow">gsutil ls gs://&lt;your-gcs-bucket&gt;/counts*
+</code></pre>
</div>
</div>
<div class='runner-samza snippet'>
@@ -1591,7 +1683,8 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-samza" data-lang="samza">$ ls /tmp/counts*</code></pre>
+<pre><code class="language-samza" data-lang="samza">ls /tmp/counts*
+</code></pre>
</div>
</div>
<div class='runner-nemo snippet'>
@@ -1599,7 +1692,8 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-nemo" data-lang="nemo">$ ls counts*</code></pre>
+<pre><code class="language-nemo" data-lang="nemo">ls counts*
+</code></pre>
</div>
</div>
<div class='runner-jet snippet'>
@@ -1607,27 +1701,20 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-jet" data-lang="jet">$ ls counts*</code></pre>
+<pre><code class="language-jet" data-lang="jet">ls counts*
+</code></pre>
</div>
</div>
-<p>When you look into the contents of the file, you&rsquo;ll see that they contain unique words and the number of occurrences of each word. The order of elements within the file may differ because the Beam model does not generally guarantee ordering, again to allow runners to optimize for efficiency.</p>
+The output files contain unique words and the number of occurrences of each
+word.</li>
+<li>View the output content in a Unix shell:
<div class='runner-direct snippet'>
<div class="notebook-skip code-snippet">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-direct" data-lang="direct">$ more counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre>
+<pre><code class="language-direct" data-lang="direct">more counts*
+</code></pre>
</div>
</div>
<div class='runner-flink snippet'>
@@ -1635,18 +1722,8 @@ single: 4
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-flink" data-lang="flink">$ more counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre>
+<pre><code class="language-flink" data-lang="flink">more counts*
+</code></pre>
</div>
</div>
<div class='runner-flinkCluster snippet'>
@@ -1654,18 +1731,8 @@ single: 4
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-flinkCluster" data-lang="flinkCluster">$ more /tmp/counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre>
+<pre><code class="language-flinkCluster" data-lang="flinkCluster">more /tmp/counts*
+</code></pre>
</div>
</div>
<div class='runner-spark snippet'>
@@ -1673,18 +1740,8 @@ single: 4
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-spark" data-lang="spark">$ more counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre>
+<pre><code class="language-spark" data-lang="spark">more counts*
+</code></pre>
</div>
</div>
<div class='runner-dataflow snippet'>
@@ -1692,18 +1749,8 @@ single: 4
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-dataflow" data-lang="dataflow">$ gsutil cat gs://&lt;your-gcs-bucket&gt;/counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre>
+<pre><code class="language-dataflow" data-lang="dataflow">gsutil cat gs://&lt;your-gcs-bucket&gt;/counts*
+</code></pre>
</div>
</div>
<div class='runner-samza snippet'>
@@ -1711,18 +1758,8 @@ single: 4
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-samza" data-lang="samza">$ more /tmp/counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre>
+<pre><code class="language-samza" data-lang="samza">more /tmp/counts*
+</code></pre>
</div>
</div>
<div class='runner-nemo snippet'>
@@ -1730,18 +1767,8 @@ single: 4
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-nemo" data-lang="nemo">$ more counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre>
+<pre><code class="language-nemo" data-lang="nemo">more counts*
+</code></pre>
</div>
</div>
<div class='runner-jet snippet'>
@@ -1749,30 +1776,40 @@ single: 4
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
<img src="/images/copy-icon.svg"/>
</a>
-<pre><code class="language-jet" data-lang="jet">$ more counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre>
+<pre><code class="language-jet" data-lang="jet">more counts*
+</code></pre>
</div>
</div>
+The order of elements is not guaranteed, to allow runners to optimize for
+efficiency. But the output should look something like this:
+<pre><code>...
+Think: 3
+slower: 1
+Having: 1
+revives: 1
+these: 33
+wipe: 1
+arrives: 1
+concluded: 1
+begins: 3
+...
+</code></pre></li>
+</ol>
<h2 id="next-steps">Next Steps</h2>
<ul>
<li>Learn more about the <a href="/documentation/sdks/java/">Beam SDK for Java</a>
-and look through the <a href="https://beam.apache.org/releases/javadoc">Java SDK API reference</a>.</li>
-<li>Walk through these WordCount examples in the <a href="/get-started/wordcount-example">WordCount Example Walkthrough</a>.</li>
-<li>Take a self-paced tour through our <a href="/documentation/resources/learning-resources">Learning Resources</a>.</li>
-<li>Dive in to some of our favorite <a href="/documentation/resources/videos-and-podcasts">Videos and Podcasts</a>.</li>
+and look through the
+<a href="https://beam.apache.org/releases/javadoc">Java SDK API reference</a>.</li>
+<li>Walk through the WordCount examples in the
+<a href="/get-started/wordcount-example">WordCount Example Walkthrough</a>.</li>
+<li>Take a self-paced tour through our
+<a href="/documentation/resources/learning-resources">Learning Resources</a>.</li>
+<li>Dive in to some of our favorite
+<a href="/documentation/resources/videos-and-podcasts">Videos and Podcasts</a>.</li>
<li>Join the Beam <a href="/community/contact-us">users@</a> mailing list.</li>
</ul>
-<p>Please don&rsquo;t hesitate to <a href="/community/contact-us">reach out</a> if you encounter any issues!</p></description></item><item><title>Get-Started: Beam Quickstart for Python</title><link>/get-started/quickstart-py/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>/get-started/quickstart-py/</guid><description>
+<p>Please don&rsquo;t hesitate to <a href="/community/contact-us">reach out</a> if you encounter any
+issues!</p></description></item><item><title>Get-Started: Beam Quickstart for Python</title><link>/get-started/quickstart-py/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>/get-started/quickstart-py/</guid><description>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
diff --git a/website/generated-content/get-started/quickstart-java/index.html b/website/generated-content/get-started/quickstart-java/index.html
index 5de01a2..7bc8b95 100644
--- a/website/generated-content/get-started/quickstart-java/index.html
+++ b/website/generated-content/get-started/quickstart-java/index.html
@@ -18,196 +18,143 @@
function addPlaceholder(){$('input:text').attr('placeholder',"What are you looking for?");}
function endSearch(){var search=document.querySelector(".searchBar");search.classList.add("disappear");var icons=document.querySelector("#iconsBar");icons.classList.remove("disappear");}
function blockScroll(){$("body").toggleClass("fixedPosition");}
-function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Get started</span></li><li><a href=/get-started/beam-overview/>Beam Overview</a></li><li><a href=/get-started/tour-of-beam/>Tour of Beam</a></li><li><s [...]
- -DarchetypeGroupId=org.apache.beam \
- -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
- -DarchetypeVersion=2.37.0 \
- -DgroupId=org.example \
- -DartifactId=word-count-beam \
- -Dversion="0.1" \
- -Dpackage=org.apache.beam.examples \
- -DinteractiveMode=false</code></pre></div></div><div class="shell-powerShell snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-powerShell data-lang=powerShell><span class=n>PS</span><span class=p>></span> <span class=n>mvn</span> <span class=n>archetype</span><span class=err>:</span><span [...]
- <span class=n>-D</span> <span class=n>archetypeGroupId</span><span class=p>=</span><span class=n>org</span><span class=p>.</span><span class=n>apache</span><span class=p>.</span><span class=n>beam</span> <span class=p>`</span>
- <span class=n>-D</span> <span class=n>archetypeArtifactId</span><span class=p>=</span><span class=n>beam-sdks-java-maven-archetypes-examples</span> <span class=p>`</span>
- <span class=n>-D</span> <span class=n>archetypeVersion</span><span class=p>=</span><span class=n>2</span><span class=p>.</span><span class=n>37</span><span class=p>.</span><span class=n>0</span> <span class=p>`</span>
- <span class=n>-D</span> <span class=n>groupId</span><span class=p>=</span><span class=n>org</span><span class=p>.</span><span class=n>example</span> <span class=p>`</span>
- <span class=n>-D</span> <span class=n>artifactId</span><span class=p>=</span><span class=n>word-count-beam</span> <span class=p>`</span>
- <span class=n>-D</span> <span class=n>version</span><span class=p>=</span><span class=s2>"0.1"</span> <span class=p>`</span>
- <span class=n>-D</span> <span class=n>package</span><span class=p>=</span><span class=n>org</span><span class=p>.</span><span class=n>apache</span><span class=p>.</span><span class=n>beam</span><span class=p>.</span><span class=n>examples</span> <span class=p>`</span>
- <span class=n>-D</span> <span class=n>interactiveMode</span><span class=p>=</span><span class=n>false</span></code></pre></div></div></div><p>This will create a <code>word-count-beam</code> directory that contains a <code>pom.xml</code> and several example pipelines that count words in text files.</p><div class="shell-unix snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/ [...]
-
-$ ls
-pom.xml src
-
-$ ls src/main/java/org/apache/beam/examples/
-DebuggingWordCount.java WindowedWordCount.java common
-MinimalWordCount.java WordCount.java</code></pre></div></div><div class="shell-powerShell snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-powerShell data-lang=powerShell><span class=n>PS</span><span class=p>></span> <span class=nb>cd </span><span class=p>.\</span><span class=n>word-count-beam</span>
-
-<span class=n>PS</span><span class=p>></span> <span class=nb>dir
-</span><span class=nb></span>
-<span class=p>...</span>
-
-<span class=n>Mode</span> <span class=n>LastWriteTime</span> <span class=n>Length</span> <span class=n>Name</span>
-<span class=p>----</span> <span class=p>-------------</span> <span class=p>------</span> <span class=p>----</span>
-<span class=n>d</span><span class=p>-----</span> <span class=n>7</span><span class=p>/</span><span class=n>19</span><span class=p>/</span><span class=n>2018</span> <span class=n>11</span><span class=err>:</span><span class=n>00</span> <span class=n>PM</span> <span class=n>src</span>
-<span class=n>-a</span><span class=p>----</span> <span class=n>7</span><span class=p>/</span><span class=n>19</span><span class=p>/</span><span class=n>2018</span> <span class=n>11</span><span class=err>:</span><span class=n>00</span> <span class=n>PM</span> <span class=n>16051</span> <span class=n>pom</span><span class=p>.</span><span class=n>xml</span>
-
-<span class=n>PS</span><span class=p>></span> <span class=nb>dir </span><span class=p>.\</span><span class=n>src</span><span class=p>\</span><span class=n>main</span><span class=p>\</span><span class=n>java</span><span class=p>\</span><span class=n>org</span><span class=p>\</span><span class=n>apache</span><span class=p>\</span><span class=n>beam</span><span class=p>\</span><span class=n>examples</span>
-
-<span class=p>...</span>
-<span class=n>Mode</span> <span class=n>LastWriteTime</span> <span class=n>Length</span> <span class=n>Name</span>
-<span class=p>----</span> <span class=p>-------------</span> <span class=p>------</span> <span class=p>----</span>
-<span class=n>d</span><span class=p>-----</span> <span class=n>7</span><span class=p>/</span><span class=n>19</span><span class=p>/</span><span class=n>2018</span> <span class=n>11</span><span class=err>:</span><span class=n>00</span> <span class=n>PM</span> <span class=n>common</span>
-<span class=n>d</span><span class=p>-----</span> <span class=n>7</span><span class=p>/</span><span class=n>19</span><span class=p>/</span><span class=n>2018</span> <span class=n>11</span><span class=err>:</span><span class=n>00</span> <span class=n>PM</span> <span class=n>complete</span>
-<span class=n>d</span><span class=p>-----</span> <span class=n>7</span><span class=p>/</span><span class=n>19</span><span class=p>/</span><span class=n>2018</span> <span class=n>11</span><span class=err>:</span><span class=n>00</span> <span class=n>PM</span> <span class=n>subprocess</span>
-<span class=n>-a</span><span class=p>----</span> <span class=n>7</span><span class=p>/</span><span class=n>19</span><span class=p>/</span><span class=n>2018</span> <span class=n>11</span><span class=err>:</span><span class=n>00</span> <span class=n>PM</span> <span class=n>7073</span> <span class=n>DebuggingWordCount</span><span class=p>.</span><span class=n>java</span>
-<span class=n>-a</span><span class=p>----</span> <span class=n>7</span><span class=p>/</span><span class=n>19</span><span class=p>/</span><span class=n>2018</span> <span class=n>11</span><span class=err>:</span><span class=n>00</span> <span class=n>PM</span> <span class=n>5945</span> <span class=n>MinimalWordCount</span><span class=p>.</span><span class=n>java</span>
-<span class=n>-a</span><span class=p>----</span> <span class=n>7</span><span class=p>/</span><span class=n>19</span><span class=p>/</span><span class=n>2018</span> <span class=n>11</span><span class=err>:</span><span class=n>00</span> <span class=n>PM</span> <span class=n>9490</span> <span class=n>WindowedWordCount</span><span class=p>.</span><span class=n>java</span>
-<span class=n>-a</span><span class=p>----</span> <span class=n>7</span><span class=p>/</span><span class=n>19</span><span class=p>/</span><span class=n>2018</span> <span class=n>11</span><span class=err>:</span><span class=n>00</span> <span class=n>PM</span> <span class=n>7662</span> <span class=n>WordCount</span><span class=p>.</span><span class=n>java</span></code></pre></div></div></div><p>For a detailed introduction to the Beam concepts used in these examples, see t [...]
- mavenCentral()
- maven {
- url = uri('https://repository.apache.org/content/repositories/snapshots/')
+function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Get started</span></li><li><a href=/get-started/beam-overview/>Beam Overview</a></li><li><a href=/get-started/tour-of-beam/>Tour of Beam</a></li><li><s [...]
+an <a href=/get-started/wordcount-example>example pipeline</a> written with the
+<a href=/documentation/sdks/java>Apache Beam Java SDK</a>, using a
+<a href=/documentation#runners>runner</a> of your choice.</p><p>If you’re interested in contributing to the Apache Beam Java codebase, see the
+<a href=/contribute>Contribution Guide</a>.</p><p>On this page:</p><nav id=TableOfContents><ul><li><a href=#set-up-your-development-environment>Set up your development environment</a></li><li><a href=#get-the-example-code>Get the example code</a></li><li><a href=#optional-convert-from-maven-to-gradle>Optional: Convert from Maven to Gradle</a></li><li><a href=#get-sample-text>Get sample text</a></li><li><a href=#run-a-pipeline>Run a pipeline</a><ul><li><a href=#run-wordcount-using-maven>R [...]
+<a href=https://www.oracle.com/technetwork/java/javase/downloads/index.html>Java Development Kit (JDK)</a>
+version 8, 11, or 17. Verify that the
+<a href=https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/envvars001.html>JAVA_HOME</a>
+environment variable is set and points to your JDK installation.</li><li>Download and install <a href=https://maven.apache.org/download.cgi>Apache Maven</a> by
+following the <a href=https://maven.apache.org/install.html>installation guide</a>
+for your operating system.</li><li>Optional: If you want to convert your Maven project to Gradle, install
+<a href=https://gradle.org/install/>Gradle</a>.</li></ol><h2 id=get-the-example-code>Get the example code</h2><ol><li><p>Generate a Maven example project that builds against the latest Beam release:<div class="shell-unix snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-unix data-lang=unix>mvn archetype:generate \
+ -DarchetypeGroupId=org.apache.beam \
+ -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
+ -DarchetypeVersion=2.37.0 \
+ -DgroupId=org.example \
+ -DartifactId=word-count-beam \
+ -Dversion="0.1" \
+ -Dpackage=org.apache.beam.examples \
+ -DinteractiveMode=false
+ </code></pre></div></div><div class="shell-powerShell snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-powerShell data-lang=powerShell><span class=n>mvn</span> <span class=n>archetype</span><span class=err>:</span><span class=n>generate</span> <span class=p>`</span>
+ <span class=n>-D</span> <span class=n>archetypeGroupId</span><span class=p>=</span><span class=n>org</span><span class=p>.</span><span class=n>apache</span><span class=p>.</span><span class=n>beam</span> <span class=p>`</span>
+ <span class=n>-D</span> <span class=n>archetypeArtifactId</span><span class=p>=</span><span class=n>beam-sdks-java-maven-archetypes-examples</span> <span class=p>`</span>
+ <span class=n>-D</span> <span class=n>archetypeVersion</span><span class=p>=</span><span class=n>2</span><span class=p>.</span><span class=n>37</span><span class=p>.</span><span class=n>0</span> <span class=p>`</span>
+ <span class=n>-D</span> <span class=n>groupId</span><span class=p>=</span><span class=n>org</span><span class=p>.</span><span class=n>example</span> <span class=p>`</span>
+ <span class=n>-D</span> <span class=n>artifactId</span><span class=p>=</span><span class=n>word-count-beam</span> <span class=p>`</span>
+ <span class=n>-D</span> <span class=n>version</span><span class=p>=</span><span class=s2>"0.1"</span> <span class=p>`</span>
+ <span class=n>-D</span> <span class=n>package</span><span class=p>=</span><span class=n>org</span><span class=p>.</span><span class=n>apache</span><span class=p>.</span><span class=n>beam</span><span class=p>.</span><span class=n>examples</span> <span class=p>`</span>
+ <span class=n>-D</span> <span class=n>interactiveMode</span><span class=p>=</span><span class=n>false</span>
+ </code></pre></div></div></div></p><p>Maven creates a new project in the <strong>word-count-beam</strong> directory.</p></li><li><p>Change into <strong>word-count-beam</strong>:<div class="shell-unix snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-unix data-lang=unix>cd word-count-beam/
+ </code></pre></div></div><div class="shell-powerShell snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-powerShell data-lang=powerShell><span class=nb>cd </span><span class=p>.\</span><span class=n>word-count-beam</span>
+ </code></pre></div></div></div>The directory contains a <strong>pom.xml</strong> and a <strong>src</strong> directory with example
+pipelines.</p></li><li><p>List the example pipelines:<div class="shell-unix snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-unix data-lang=unix>ls src/main/java/org/apache/beam/examples/
+ </code></pre></div></div><div class="shell-powerShell snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-powerShell data-lang=powerShell><span class=nb>dir </span><span class=p>.\</span><span class=n>src</span><span class=p>\</span><span class=n>main</span><span class=p>\</span><span class=n>jav [...]
+ </code></pre></div></div></div>You should see the following examples:</p><ul><li><strong>DebuggingWordCount.java</strong> (<a href=https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/DebuggingWordCount.java>GitHub</a>)</li><li><strong>MinimalWordCount.java</strong> (<a href=https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java>GitHub</a>)</li><li><strong>WindowedWordCount.java</ [...]
+Beam pipeline that counts words from an input file (by default, a <strong>.txt</strong>
+file containing Shakespeare’s “King Lear”). To learn more about the examples,
+see the <a href=/get-started/wordcount-example>WordCount Example Walkthrough</a>.</p></li></ol><h2 id=optional-convert-from-maven-to-gradle>Optional: Convert from Maven to Gradle</h2><p>The steps below explain how to convert the build from Maven to Gradle for the
+following runners:</p><ul><li>Direct runner</li><li>Dataflow runner</li></ul><p>The conversion process for other runners is similar. For additional guidance,
+see
+<a href=https://docs.gradle.org/current/userguide/migrating_from_maven.html>Migrating Builds From Apache Maven</a>.</p><ol><li>In the directory with the <strong>pom.xml</strong> file, run the automated Maven-to-Gradle
+conversion:<div class=snippet><div class="notebook-skip code-snippet without_switcher"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code>gradle init
+ </code></pre></div></div>You’ll be asked if you want to generate a Gradle build. Enter <strong>yes</strong>. You’ll
+also be prompted to choose a DSL (Groovy or Kotlin). For this tutorial, enter
+<strong>2</strong> for Kotlin.</li><li>Open the generated <strong>build.gradle.kts</strong> file and make the following changes:<ol><li>In <code>repositories</code>, replace <code>mavenLocal()</code> with <code>mavenCentral()</code>.</li><li>In <code>repositories</code>, declare a repository for Confluent Kafka dependencies:<div class=snippet><div class="notebook-skip code-snippet without_switcher"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to c [...]
+ url = uri("https://packages.confluent.io/maven/")
+}
+ </code></pre></div></div></li><li>At the end of the build script, add the following conditional dependency:<div class=snippet><div class="notebook-skip code-snippet without_switcher"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code>if (project.hasProperty("dataflow-runner")) {
+ dependencies {
+ runtimeOnly("org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0")
}
-
- maven {
- url = uri('http://repo.maven.apache.org/maven2')
- }
-}</code></pre></div></div></li><li>Add the following task in <code>build.gradle</code> to allow you to execute pipelines with Gradle:<div class=snippet><div class="notebook-skip code-snippet without_switcher"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code>task execute (type:JavaExec) {
- mainClass = System.getProperty("mainClass")
- classpath = sourceSets.main.runtimeClasspath
- systemProperties System.getProperties()
- args System.getProperty("exec.args", "").split()
-}</code></pre></div></div></li><li>Rebuild your project by running:<div class=snippet><div class="notebook-skip code-snippet without_switcher"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code>$ gradle build</code></pre></div></div></li></ol><h2 id=get-sample-text>Get sample text</h2><blockquote><p>If you’re planning to use the DataflowRunner, you can skip this step. The runner will pull [...]
- -Dexec.args="--inputFile=sample.txt --output=counts" -Pdirect-runner</code></pre></div></div><div class="runner-flink snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flink data-lang=flink>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
- -Dexec.args="--runner=FlinkRunner --inputFile=sample.txt --output=counts" -Pflink-runner</code></pre></div></div><div class="runner-flinkCluster snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flinkCluster data-lang=flinkCluster>$ mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
- -Dexec.args="--runner=FlinkRunner --flinkMaster=<flink master> --filesToStage=target/word-count-beam-bundled-0.1.jar \
- --inputFile=sample.txt --output=/tmp/counts" -Pflink-runner
-
-You can monitor the running job by visiting the Flink dashboard at http://<flink master>:8081</code></pre></div></div><div class="runner-spark snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-spark data-lang=spark>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
- -Dexec.args="--runner=SparkRunner --inputFile=sample.txt --output=counts" -Pspark-runner</code></pre></div></div><div class="runner-dataflow snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-dataflow data-lang=dataflow>Make sure you complete the setup steps at /documentation/runners/dataflow/#setup
-
-$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
- -Dexec.args="--runner=DataflowRunner --project=<your-gcp-project> \
- --region=<your-gcp-region> \
- --gcpTempLocation=gs://<your-gcs-bucket>/tmp \
- --inputFile=gs://apache-beam-samples/shakespeare/* --output=gs://<your-gcs-bucket>/counts" \
- -Pdataflow-runner</code></pre></div></div><div class="runner-samza snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-samza data-lang=samza>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
- -Dexec.args="--inputFile=sample.txt --output=/tmp/counts --runner=SamzaRunner" -Psamza-runner</code></pre></div></div><div class="runner-nemo snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-nemo data-lang=nemo>$ mvn package -Pnemo-runner && java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.exam [...]
- --runner=NemoRunner --inputFile=`pwd`/sample.txt --output=counts</code></pre></div></div><div class="runner-jet snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-jet data-lang=jet>$ mvn package -Pjet-runner
-$ java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount \
- --runner=JetRunner --jetLocalMode=3 --inputFile=`pwd`/sample.txt --output=counts</code></pre></div></div><p>For Windows PowerShell:</p><div class="runner-direct snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-direct data-lang=direct>PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
- -D exec.args="--inputFile=sample.txt --output=counts" -P direct-runner</code></pre></div></div><div class="runner-flink snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flink data-lang=flink>PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
- -D exec.args="--runner=FlinkRunner --inputFile=sample.txt --output=counts" -P flink-runner</code></pre></div></div><div class="runner-flinkCluster snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flinkCluster data-lang=flinkCluster>PS> mvn package exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+}
+ </code></pre></div></div></li><li>At the end of the build script, add the following task:<div class=snippet><div class="notebook-skip code-snippet without_switcher"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code>task("execute", JavaExec::class) {
+ classpath = sourceSets["main"].runtimeClasspath
+ mainClass.set(System.getProperty("mainClass"))
+}
+ </code></pre></div></div></li></ol></li><li>Build your project:<div class=snippet><div class="notebook-skip code-snippet without_switcher"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code>gradle build
+ </code></pre></div></div></li></ol><h2 id=get-sample-text>Get sample text</h2><blockquote><p>If you’re planning to use the DataflowRunner, you can skip this step. The
+runner will pull text directly from Google Cloud Storage.</p></blockquote><ol><li>In the <strong>word-count-beam</strong> directory, create a file called <strong>sample.txt</strong>.</li><li>Add some text to the file. For this example, use the text of Shakespeare’s
+<a href=https://storage.cloud.google.com/apache-beam-samples/shakespeare/kinglear.txt>King Lear</a>.</li></ol><h2 id=run-a-pipeline>Run a pipeline</h2><p>A single Beam pipeline can run on multiple Beam
+<a href=/documentation#runners>runners</a>. The
+<a href=/documentation/runners/direct>DirectRunner</a> is useful for getting started,
+because it runs on your machine and requires no specific setup. If you’re just
+trying out Beam and you’re not sure what to use, use the
+<a href=/documentation/runners/direct>DirectRunner</a>.</p><p>The general process for running a pipeline goes like this:</p><ol><li>Complete any runner-specific setup.</li><li>Build your command line:<ol><li>Specify a runner with <code>--runner=<runner></code> (defaults to the
+<a href=/documentation/runners/direct>DirectRunner</a>).</li><li>Add any runner-specific required options.</li><li>Choose input files and an output location that are accessible to the
+runner. (For example, you can’t access a local file if you are running
+the pipeline on an external cluster.)</li></ol></li><li>Run the command.</li></ol><p>To run the WordCount pipeline:</p><ol><li><p>Follow the setup steps for your runner:</p><ul><li><a href=/documentation/runners/flink>FlinkRunner</a></li><li><a href=/documentation/runners/spark>SparkRunner</a></li><li><a href=/documentation/runners/dataflow>DataflowRunner</a></li><li><a href=/documentation/runners/samza>SamzaRunner</a></li><li><a href=/documentation/runners/nemo>NemoRunner</a></li><li><a [...]
+ -Dexec.args="--inputFile=sample.txt --output=counts" -Pdirect-runner</code></pre></div></div><div class="runner-flink snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flink data-lang=flink>mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+ -Dexec.args="--runner=FlinkRunner --inputFile=sample.txt --output=counts" -Pflink-runner</code></pre></div></div><div class="runner-flinkCluster snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flinkCluster data-lang=flinkCluster>mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+ -Dexec.args="--runner=FlinkRunner --flinkMaster=<flink master> --filesToStage=target/word-count-beam-bundled-0.1.jar \
+ --inputFile=sample.txt --output=/tmp/counts" -Pflink-runner</code></pre></div></div><div class="runner-spark snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-spark data-lang=spark>mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+ -Dexec.args="--runner=SparkRunner --inputFile=sample.txt --output=counts" -Pspark-runner</code></pre></div></div><div class="runner-dataflow snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-dataflow data-lang=dataflow>mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+ -Dexec.args="--runner=DataflowRunner --project=<your-gcp-project> \
+ --region=<your-gcp-region> \
+ --gcpTempLocation=gs://<your-gcs-bucket>/tmp \
+ --inputFile=gs://apache-beam-samples/shakespeare/* --output=gs://<your-gcs-bucket>/counts" \
+ -Pdataflow-runner</code></pre></div></div><div class="runner-samza snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-samza data-lang=samza>mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
+ -Dexec.args="--inputFile=sample.txt --output=/tmp/counts --runner=SamzaRunner" -Psamza-runner</code></pre></div></div><div class="runner-nemo snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-nemo data-lang=nemo>mvn package -Pnemo-runner && java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.example [...]
+ --runner=NemoRunner --inputFile=`pwd`/sample.txt --output=counts</code></pre></div></div><div class="runner-jet snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-jet data-lang=jet>mvn package -Pjet-runner
+java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount \
+ --runner=JetRunner --jetLocalMode=3 --inputFile=`pwd`/sample.txt --output=counts</code></pre></div></div></p><p>For Windows PowerShell:</p><p><div class="runner-direct snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-direct data-lang=direct>mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+ -D exec.args="--inputFile=sample.txt --output=counts" -P direct-runner</code></pre></div></div><div class="runner-flink snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flink data-lang=flink>mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+ -D exec.args="--runner=FlinkRunner --inputFile=sample.txt --output=counts" -P flink-runner</code></pre></div></div><div class="runner-flinkCluster snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flinkCluster data-lang=flinkCluster>mvn package exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args="--runner=FlinkRunner --flinkMaster=<flink master> --filesToStage=.\target\word-count-beam-bundled-0.1.jar `
- --inputFile=C:\path\to\quickstart\sample.txt --output=C:\tmp\counts" -P flink-runner
-
-You can monitor the running job by visiting the Flink dashboard at http://<flink master>:8081</code></pre></div></div><div class="runner-spark snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-spark data-lang=spark>PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
- -D exec.args="--runner=SparkRunner --inputFile=sample.txt --output=counts" -P spark-runner</code></pre></div></div><div class="runner-dataflow snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-dataflow data-lang=dataflow>Make sure you complete the setup steps at /documentation/runners/dataflow/#setup
-
-PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+ --inputFile=C:\path\to\quickstart\sample.txt --output=C:\tmp\counts" -P flink-runner</code></pre></div></div><div class="runner-spark snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-spark data-lang=spark>mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+ -D exec.args="--runner=SparkRunner --inputFile=sample.txt --output=counts" -P spark-runner</code></pre></div></div><div class="runner-dataflow snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-dataflow data-lang=dataflow>mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
-D exec.args="--runner=DataflowRunner --project=<your-gcp-project> `
--region=<your-gcp-region> \
--gcpTempLocation=gs://<your-gcs-bucket>/tmp `
--inputFile=gs://apache-beam-samples/shakespeare/* --output=gs://<your-gcs-bucket>/counts" `
- -P dataflow-runner</code></pre></div></div><div class="runner-samza snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-samza data-lang=samza>PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
- -D exec.args="--inputFile=sample.txt --output=/tmp/counts --runner=SamzaRunner" -P samza-runner</code></pre></div></div><div class="runner-nemo snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-nemo data-lang=nemo>PS> mvn package -P nemo-runner -DskipTests
-PS> java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
- --runner=NemoRunner --inputFile=`pwd`/sample.txt --output=counts</code></pre></div></div><div class="runner-jet snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-jet data-lang=jet>PS> mvn package -P jet-runner
-PS> java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
- --runner=JetRunner --jetLocalMode=3 --inputFile=$pwd/sample.txt --output=counts</code></pre></div></div><h3 id=run-wordcount-using-gradle>Run WordCount Using Gradle</h3><p>For Unix shells (Instructions currently only available for Direct, Spark, and Dataflow):</p><div class="runner-direct snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code c [...]
- -Dexec.args="--inputFile=sample.txt --output=counts" -Pdirect-runner</code></pre></div></div><div class="runner-flink snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flink data-lang=flink>We are working on adding the instruction for this runner!</code></pre></div></div><div class="runner-flinkCluster snippet"><div cl [...]
- -Dexec.args="--inputFile=sample.txt --output=counts" -Pspark-runner</code></pre></div></div><div class="runner-dataflow snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-dataflow data-lang=dataflow>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
- -Dexec.args="--project=<your-gcp-project> --inputFile=gs://apache-beam-samples/shakespeare/* \
- --output=gs://<your-gcs-bucket>/counts" -Pdataflow-runner</code></pre></div></div><div class="runner-samza snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-samza data-lang=samza>We are working on adding the instruction for this runner!</code></pre></div></div><div class="runner-nemo snippet"><div class="notebook-ski [...]
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre></div></div><div class="runner-flink snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flink data-lang=flink>$ more counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre></div></div><div class="runner-flinkCluster snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flinkCluster data-lang=flinkCluster>$ more /tmp/counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre></div></div><div class="runner-spark snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-spark data-lang=spark>$ more counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre></div></div><div class="runner-dataflow snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-dataflow data-lang=dataflow>$ gsutil cat gs://<your-gcs-bucket>/counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre></div></div><div class="runner-samza snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-samza data-lang=samza>$ more /tmp/counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre></div></div><div class="runner-nemo snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-nemo data-lang=nemo>$ more counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre></div></div><div class="runner-jet snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-jet data-lang=jet>$ more counts*
-wrought: 2
-st: 32
-fresher: 1
-of: 351
-souls: 2
-CXVIII: 1
-reviewest: 1
-untold: 1
-th: 1
-single: 4
-...</code></pre></div></div><h2 id=next-steps>Next Steps</h2><ul><li>Learn more about the <a href=/documentation/sdks/java/>Beam SDK for Java</a>
-and look through the <a href=https://beam.apache.org/releases/javadoc>Java SDK API reference</a>.</li><li>Walk through these WordCount examples in the <a href=/get-started/wordcount-example>WordCount Example Walkthrough</a>.</li><li>Take a self-paced tour through our <a href=/documentation/resources/learning-resources>Learning Resources</a>.</li><li>Dive in to some of our favorite <a href=/documentation/resources/videos-and-podcasts>Videos and Podcasts</a>.</li><li>Join the Beam <a href= [...]
+ -P dataflow-runner</code></pre></div></div><div class="runner-samza snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-samza data-lang=samza>mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCount `
+ -D exec.args="--inputFile=sample.txt --output=/tmp/counts --runner=SamzaRunner" -P samza-runner</code></pre></div></div><div class="runner-nemo snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-nemo data-lang=nemo>mvn package -P nemo-runner -DskipTests
+java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
+ --runner=NemoRunner --inputFile=`pwd`/sample.txt --output=counts</code></pre></div></div><div class="runner-jet snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-jet data-lang=jet>mvn package -P jet-runner
+java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
+ --runner=JetRunner --jetLocalMode=3 --inputFile=$pwd/sample.txt --output=counts</code></pre></div></div></p><h3 id=run-wordcount-using-gradle>Run WordCount using Gradle</h3><p>For Unix shells:</p><p><div class="runner-direct snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-direct data-lang=direct>gradle clean execute -DmainCl [...]
+ --args="--inputFile=sample.txt --output=counts"</code></pre></div></div><div class="runner-flink snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flink data-lang=flink>TODO: document Flink on Gradle: https://issues.apache.org/jira/browse/BEAM-14057</code></pre></div></div><div class="runner-flinkCluster snippet"><div [...]
+ --args="--project=<your-gcp-project> --inputFile=gs://apache-beam-samples/shakespeare/* \
+ --output=gs://<your-gcs-bucket>/counts" -Pdataflow-runner</code></pre></div></div><div class="runner-samza snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-samza data-lang=samza>TODO: document Samza on Gradle: https://issues.apache.org/jira/browse/BEAM-14061</code></pre></div></div><div class="runner-nemo snippet">< [...]
+multiple output files prefixed by <code>count</code>. The number of output files is decided
+by the runner, giving it the flexibility to do efficient, distributed execution.</p><ol><li>View the output files in a Unix shell:<div class="runner-direct snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-direct data-lang=direct>ls counts*
+ </code></pre></div></div><div class="runner-flink snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flink data-lang=flink>ls counts*
+ </code></pre></div></div><div class="runner-flinkCluster snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flinkCluster data-lang=flinkCluster>ls /tmp/counts*
+ </code></pre></div></div><div class="runner-spark snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-spark data-lang=spark>ls counts*
+ </code></pre></div></div><div class="runner-dataflow snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-dataflow data-lang=dataflow>gsutil ls gs://<your-gcs-bucket>/counts*
+ </code></pre></div></div><div class="runner-samza snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-samza data-lang=samza>ls /tmp/counts*
+ </code></pre></div></div><div class="runner-nemo snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-nemo data-lang=nemo>ls counts*
+ </code></pre></div></div><div class="runner-jet snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-jet data-lang=jet>ls counts*
+ </code></pre></div></div>The output files contain unique words and the number of occurrences of each
+word.</li><li>View the output content in a Unix shell:<div class="runner-direct snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-direct data-lang=direct>more counts*
+ </code></pre></div></div><div class="runner-flink snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flink data-lang=flink>more counts*
+ </code></pre></div></div><div class="runner-flinkCluster snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-flinkCluster data-lang=flinkCluster>more /tmp/counts*
+ </code></pre></div></div><div class="runner-spark snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-spark data-lang=spark>more counts*
+ </code></pre></div></div><div class="runner-dataflow snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-dataflow data-lang=dataflow>gsutil cat gs://<your-gcs-bucket>/counts*
+ </code></pre></div></div><div class="runner-samza snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-samza data-lang=samza>more /tmp/counts*
+ </code></pre></div></div><div class="runner-nemo snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-nemo data-lang=nemo>more counts*
+ </code></pre></div></div><div class="runner-jet snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre><code class=language-jet data-lang=jet>more counts*
+ </code></pre></div></div>The order of elements is not guaranteed, to allow runners to optimize for
+efficiency. But the output should look something like this:<pre><code>...
+Think: 3
+slower: 1
+Having: 1
+revives: 1
+these: 33
+wipe: 1
+arrives: 1
+concluded: 1
+begins: 3
+...
+</code></pre></li></ol><h2 id=next-steps>Next Steps</h2><ul><li>Learn more about the <a href=/documentation/sdks/java/>Beam SDK for Java</a>
+and look through the
+<a href=https://beam.apache.org/releases/javadoc>Java SDK API reference</a>.</li><li>Walk through the WordCount examples in the
+<a href=/get-started/wordcount-example>WordCount Example Walkthrough</a>.</li><li>Take a self-paced tour through our
+<a href=/documentation/resources/learning-resources>Learning Resources</a>.</li><li>Dive in to some of our favorite
+<a href=/documentation/resources/videos-and-podcasts>Videos and Podcasts</a>.</li><li>Join the Beam <a href=/community/contact-us>users@</a> mailing list.</li></ul><p>Please don’t hesitate to <a href=/community/contact-us>reach out</a> if you encounter any
+issues!</p><div class=feedback><p class=update>Last updated on 2022/03/15</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:dev@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class="footer__cols__col foo [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div></div></div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index 45bdf60..fa548fe 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.37.0/</loc><lastmod>2022-03-04T10:14:02-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2022-03-04T10:14:02-08:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-03-04T10:14:02-08:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-03-04T10:14:02-08:00</lastmod></url><url><loc>/blog/u [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.37.0/</loc><lastmod>2022-03-04T10:14:02-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2022-03-04T10:14:02-08:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-03-04T10:14:02-08:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-03-04T10:14:02-08:00</lastmod></url><url><loc>/blog/u [...]
\ No newline at end of file