You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2019/04/18 09:48:37 UTC
[beam] branch asf-site updated: Publishing website 2019/04/18
09:48:28 at commit 0447bf9
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 3b45ad6 Publishing website 2019/04/18 09:48:28 at commit 0447bf9
3b45ad6 is described below
commit 3b45ad611d0b0b0665a9b71642fbb4307a2dc1ff
Author: jenkins <bu...@apache.org>
AuthorDate: Thu Apr 18 09:48:29 2019 +0000
Publishing website 2019/04/18 09:48:28 at commit 0447bf9
---
.../generated-content/roadmap/portability/index.html | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/website/generated-content/roadmap/portability/index.html b/website/generated-content/roadmap/portability/index.html
index 01a56a5..2472ebb 100644
--- a/website/generated-content/roadmap/portability/index.html
+++ b/website/generated-content/roadmap/portability/index.html
@@ -213,7 +213,7 @@
<li><a href="#issues">Issues</a></li>
<li><a href="#status">Status</a>
<ul>
- <li><a href="#python-on-flink">Running Python wordcount on Flink</a></li>
+ <li><a href="#python-on-flink">Running Python wordcount on Flink or Spark</a></li>
</ul>
</li>
</ul>
@@ -379,18 +379,23 @@ See the
<a href="https://s.apache.org/apache-beam-portability-support-table">Portability support table</a>
for details.</p>
-<h3 id="python-on-flink">Running Python wordcount on Flink</h3>
+<h3 id="python-on-flink">Running Python wordcount on Flink or Spark</h3>
-<p>The Flink runner is currently the only runner to support portable pipeline execution.
-To run a basic Python wordcount (in batch mode) with embedded Flink:</p>
+<p>Currently, the Flink and Spark runners support portable pipeline execution.
+To run a basic Python wordcount (in batch mode) with embedded Flink or Spark:</p>
<ol>
<li>Run once to build the SDK harness container: <code class="highlighter-rouge">./gradlew :beam-sdks-python-container:docker</code></li>
- <li>Start the Flink portable JobService endpoint: <code class="highlighter-rouge">./gradlew :beam-runners-flink_2.11-job-server:runShadow</code></li>
- <li>Submit the wordcount pipeline to above endpoint: <code class="highlighter-rouge">./gradlew :beam-sdks-python:portableWordCount -PjobEndpoint=localhost:8099</code></li>
+ <li>Choose one:
+ <ul>
+ <li>Start the Flink portable JobService endpoint: <code class="highlighter-rouge">./gradlew :beam-runners-flink_2.11-job-server:runShadow</code></li>
+ <li>Or start the Spark portable JobService endpoint: <code class="highlighter-rouge">./gradlew :beam-runners-spark-job-server:runShadow</code></li>
+ </ul>
+ </li>
+ <li>Submit the wordcount pipeline to above endpoint: <code class="highlighter-rouge">./gradlew :beam-sdks-python:portableWordCount -PjobEndpoint=localhost:8099 -PenvironmentType=LOOPBACK</code></li>
</ol>
-<p>To run the pipeline in streaming mode: <code class="highlighter-rouge">./gradlew :beam-sdks-python:portableWordCount -PjobEndpoint=localhost:8099 -Pstreaming</code></p>
+<p>To run the pipeline in streaming mode (currently only supported on Flink): <code class="highlighter-rouge">./gradlew :beam-sdks-python:portableWordCount -PjobEndpoint=localhost:8099 -Pstreaming</code></p>
<p>Please see the <a href="/documentation/runners/flink/">Flink Runner page</a> for more information on
how to run portable pipelines on top of Flink.</p>