You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2020/06/18 00:03:18 UTC
[beam] branch asf-site updated: Publishing website 2020/06/18
00:03:05 at commit e7ca038
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 8cae0b3 Publishing website 2020/06/18 00:03:05 at commit e7ca038
8cae0b3 is described below
commit 8cae0b3b3347d1d6b687dacefc1b22046b09d50b
Author: jenkins <us...@infra.apache.org>
AuthorDate: Thu Jun 18 00:03:06 2020 +0000
Publishing website 2020/06/18 00:03:05 at commit e7ca038
---
website/generated-content/get-started/index.xml | 74 +++++++++++++++++++++-
.../get-started/quickstart-java/index.html | 24 ++++++-
website/generated-content/sitemap.xml | 2 +-
3 files changed, 95 insertions(+), 5 deletions(-)
diff --git a/website/generated-content/get-started/index.xml b/website/generated-content/get-started/index.xml
index 7e086da..5cd8418 100644
--- a/website/generated-content/get-started/index.xml
+++ b/website/generated-content/get-started/index.xml
@@ -978,7 +978,13 @@ limitations under the License.
<ul>
<li><a href="#set-up-your-development-environment">Set up your Development Environment</a></li>
<li><a href="#get-the-wordcount-code">Get the WordCount Code</a></li>
-<li><a href="#run-wordcount">Run WordCount</a></li>
+<li><a href="#optional-convert-from-maven-to-gradle-project">Optional: Convert from Maven to Gradle Project</a></li>
+<li><a href="#run-wordcount">Run WordCount</a>
+<ul>
+<li><a href="#run-wordcount-using-maven">Run WordCount Using Maven</a></li>
+<li><a href="#run-wordcount-using-gradle">Run WordCount Using Gradle</a></li>
+</ul>
+</li>
<li><a href="#inspect-the-results">Inspect the results</a></li>
<li><a href="#next-steps">Next Steps</a></li>
</ul>
@@ -991,6 +997,9 @@ limitations under the License.
<li>
<p>Download and install <a href="https://maven.apache.org/download.cgi">Apache Maven</a> by following Maven&rsquo;s <a href="https://maven.apache.org/install.html">installation guide</a> for your specific operating system.</p>
</li>
+<li>
+<p>Optional: Install <a href="https://gradle.org/install/">Gradle</a> if you would like to convert your Maven project into Gradle.</p>
+</li>
</ol>
<h2 id="get-the-wordcount-code">Get the WordCount Code</h2>
<p>The easiest way to get a copy of the WordCount pipeline is to use the following command to generate a simple Maven project that contains Beam&rsquo;s WordCount examples and builds against the most recent Beam release:</p>
@@ -1046,6 +1055,35 @@ d----- 7/19/2018 11:00 PM subprocess
-a---- 7/19/2018 11:00 PM 7662 WordCount.java</code></pre>
</div>
<p>For a detailed introduction to the Beam concepts used in these examples, see the <a href="/get-started/wordcount-example">WordCount Example Walkthrough</a>. Here, we&rsquo;ll just focus on executing <code>WordCount.java</code>.</p>
+<h2 id="optional-convert-from-maven-to-gradle-project">Optional: Convert from Maven to Gradle Project</h2>
+<p>Ensure you are in the same directory as the <code>pom.xml</code> file generated from the previous step. Automatically convert your project from Maven to Gradle by running:
+<pre><code>$ gradle init</code></pre>
+</p>
+<p>After you have converted the project to Gradle:</p>
+<ol>
+<li>Edit the generated <code>build.gradle</code> file by adding <code>mavenCentral()</code> under <code>repositories</code>:
+<pre><code>repositories {
+mavenCentral()
+maven {
+url = uri(&#39;https://repository.apache.org/content/repositories/snapshots/&#39;)
+}
+maven {
+url = uri(&#39;http://repo.maven.apache.org/maven2&#39;)
+}
+}</code></pre>
+</li>
+<li>Add the following task in <code>build.gradle</code> to allow you to execute pipelines with Gradle:
+<pre><code>task execute (type:JavaExec) {
+main = System.getProperty(&#34;mainClass&#34;)
+classpath = sourceSets.main.runtimeClasspath
+systemProperties System.getProperties()
+args System.getProperty(&#34;exec.args&#34;).split()
+}</code></pre>
+</li>
+<li>Rebuild your project by running:
+<pre><code>$ gradle build</code></pre>
+</li>
+</ol>
<h2 id="run-wordcount">Run WordCount</h2>
<p>A single Beam pipeline can run on multiple Beam <a href="/documentation#runners">runners</a>, including the <a href="/documentation/runners/apex">ApexRunner</a>, <a href="/documentation/runners/flink">FlinkRunner</a>, <a href="/documentation/runners/spark">SparkRunner</a>, <a href="/documentation/runners/nemo">NemoRunner</a>, <a href="/documentation/runners/jet">JetRunner</a>, or <a href="/documentation/runners/dataflow">DataflowRunner</a>. [...]
<p>After you&rsquo;ve chosen which runner you&rsquo;d like to use:</p>
@@ -1060,6 +1098,7 @@ d----- 7/19/2018 11:00 PM subprocess
</li>
<li>Run your first WordCount pipeline.</li>
</ol>
+<h3 id="run-wordcount-using-maven">Run WordCount Using Maven</h3>
<p>For Unix shells:</p>
<div class=runner-direct>
<pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
@@ -1151,6 +1190,39 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp
PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
--runner=JetRunner --jetLocalMode=3 --inputFile=$pwd/pom.xml --output=counts</code></pre>
</div>
+<h3 id="run-wordcount-using-gradle">Run WordCount Using Gradle</h3>
+<p>For Unix shells (Instructions currently only available for Direct, Spark, and Dataflow):</p>
+<div class=runner-direct>
+<pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
+-Dexec.args=&#34;--inputFile=pom.xml --output=counts&#34; -Pdirect-runner</code></pre>
+</div>
+<div class=runner-apex>
+<pre><code>We are working on adding the instruction for this runner!</code></pre>
+</div>
+<div class=runner-flink-local>
+<pre><code>We are working on adding the instruction for this runner!</code></pre>
+</div>
+<div class=runner-flink-cluster>
+<pre><code>We are working on adding the instruction for this runner!</code></pre>
+</div>
+<div class=runner-spark>
+<pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
+-Dexec.args=&#34;--inputFile=pom.xml --output=counts&#34; -Pspark-runner</code></pre>
+</div>
+<div class=runner-dataflow>
+<pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
+-Dexec.args=&#34;--project=&lt;your-gcp-project&gt; --inputFile=gs://apache-beam-samples/shakespeare/* \
+--output=gs://&lt;your-gcs-bucket&gt;/counts&#34; -Pdataflow-runner</code></pre>
+</div>
+<div class=runner-samza-local>
+<pre><code>We are working on adding the instruction for this runner!</code></pre>
+</div>
+<div class=runner-nemo>
+<pre><code>We are working on adding the instruction for this runner!</code></pre>
+</div>
+<div class=runner-jet>
+<pre><code>We are working on adding the instruction for this runner!</code></pre>
+</div>
<h2 id="inspect-the-results">Inspect the results</h2>
<p>Once the pipeline has completed, you can view the output. You&rsquo;ll notice that there may be multiple output files prefixed by <code>count</code>. The exact number of these files is decided by the runner, giving it the flexibility to do efficient, distributed execution.</p>
<div class=runner-direct>
diff --git a/website/generated-content/get-started/quickstart-java/index.html b/website/generated-content/get-started/quickstart-java/index.html
index 14b35a3..057f901 100644
--- a/website/generated-content/get-started/quickstart-java/index.html
+++ b/website/generated-content/get-started/quickstart-java/index.html
@@ -1,7 +1,7 @@
<!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Beam Quickstart for Java</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Doma [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button>
-<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
+<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...]
-DarchetypeGroupId=org.apache.beam \
-DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
-DarchetypeVersion=2.22.0 \
@@ -46,7 +46,21 @@ d----- 7/19/2018 11:00 PM subprocess
-a---- 7/19/2018 11:00 PM 7073 DebuggingWordCount.java
-a---- 7/19/2018 11:00 PM 5945 MinimalWordCount.java
-a---- 7/19/2018 11:00 PM 9490 WindowedWordCount.java
--a---- 7/19/2018 11:00 PM 7662 WordCount.java</code></pre></div><p>For a detailed introduction to the Beam concepts used in these examples, see the <a href=/get-started/wordcount-example>WordCount Example Walkthrough</a>. Here, we’ll just focus on executing <code>WordCount.java</code>.</p><h2 id=run-wordcount>Run WordCount</h2><p>A single Beam pipeline can run on multiple Beam <a href=/documentation#runners>runners</a>, including the <a href=/documentation/runners [...]
+-a---- 7/19/2018 11:00 PM 7662 WordCount.java</code></pre></div><p>For a detailed introduction to the Beam concepts used in these examples, see the <a href=/get-started/wordcount-example>WordCount Example Walkthrough</a>. Here, we’ll just focus on executing <code>WordCount.java</code>.</p><h2 id=optional-convert-from-maven-to-gradle-project>Optional: Convert from Maven to Gradle Project</h2><p>Ensure you are in the same directory as the <code>pom.xml</code> file g [...]
+ mavenCentral()
+ maven {
+ url = uri('https://repository.apache.org/content/repositories/snapshots/')
+ }
+
+ maven {
+ url = uri('http://repo.maven.apache.org/maven2')
+ }
+}</code></pre></li><li>Add the following task in <code>build.gradle</code> to allow you to execute pipelines with Gradle:<pre><code>task execute (type:JavaExec) {
+ main = System.getProperty("mainClass")
+ classpath = sourceSets.main.runtimeClasspath
+ systemProperties System.getProperties()
+ args System.getProperty("exec.args").split()
+}</code></pre></li><li>Rebuild your project by running:<pre><code>$ gradle build</code></pre></li></ol><h2 id=run-wordcount>Run WordCount</h2><p>A single Beam pipeline can run on multiple Beam <a href=/documentation#runners>runners</a>, including the <a href=/documentation/runners/apex>ApexRunner</a>, <a href=/documentation/runners/flink>FlinkRunner</a>, <a href=/documentation/runners/spark>SparkRunner</a>, <a href=/documentation/runners/nemo>NemoRunner</a>, <a href=/documentation/runner [...]
-Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-apex><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--inputFile=pom.xml --output=counts --runner=ApexRunner" -Papex-runner</code></pre></div><div class=runner-flink-local><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Dexec.args="--runner=FlinkRunner --inputFile=pom.xml --output=counts" -Pflink-runner</code></pre></div><div class=runner-flink-cluster><pre><code>$ mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
@@ -85,7 +99,11 @@ PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCoun
PS> java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
--runner=NemoRunner --inputFile=`pwd`/pom.xml --output=counts</code></pre></div><div class=runner-jet><pre><code>PS> mvn package -P jet-runner
PS> java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount `
- --runner=JetRunner --jetLocalMode=3 --inputFile=$pwd/pom.xml --output=counts</code></pre></div><h2 id=inspect-the-results>Inspect the results</h2><p>Once the pipeline has completed, you can view the output. You’ll notice that there may be multiple output files prefixed by <code>count</code>. The exact number of these files is decided by the runner, giving it the flexibility to do efficient, distributed execution.</p><div class=runner-direct><pre><code>$ ls counts*</code></pre [...]
+ --runner=JetRunner --jetLocalMode=3 --inputFile=$pwd/pom.xml --output=counts</code></pre></div><h3 id=run-wordcount-using-gradle>Run WordCount Using Gradle</h3><p>For Unix shells (Instructions currently only available for Direct, Spark, and Dataflow):</p><div class=runner-direct><pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
+ -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-apex><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-flink-local><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-flink-cluster><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-spark><pre><code>$ gradle clean [...]
+ -Dexec.args="--inputFile=pom.xml --output=counts" -Pspark-runner</code></pre></div><div class=runner-dataflow><pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \
+ -Dexec.args="--project=<your-gcp-project> --inputFile=gs://apache-beam-samples/shakespeare/* \
+ --output=gs://<your-gcs-bucket>/counts" -Pdataflow-runner</code></pre></div><div class=runner-samza-local><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-nemo><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-jet><pre><code>We are working on adding the instruction for this runner!</code></pre></div><h2 id=inspect-the-results>Inspect the results</h2><p>Once the pip [...]
api: 9
bundled: 1
old: 4
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index b31f7bd..a889e3a 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.22.0/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/b [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.22.0/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/b [...]
\ No newline at end of file