You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ib...@apache.org on 2021/03/09 22:28:02 UTC
[beam] branch master updated: [BEAM-9689] Add Go quickstart for
Spark runner.
This is an automated email from the ASF dual-hosted git repository.
ibzib pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new d587d42 [BEAM-9689] Add Go quickstart for Spark runner.
new d772aea Merge pull request #14177 from ibzib/BEAM-9689
d587d42 is described below
commit d587d42abebaf635da0d072f2191c6bf9781ed92
Author: Kyle Weaver <kc...@google.com>
AuthorDate: Tue Mar 9 13:56:55 2021 -0800
[BEAM-9689] Add Go quickstart for Spark runner.
---
website/www/site/content/en/get-started/quickstart-go.md | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/website/www/site/content/en/get-started/quickstart-go.md b/website/www/site/content/en/get-started/quickstart-go.md
index c66067e..f92cc6e 100644
--- a/website/www/site/content/en/get-started/quickstart-go.md
+++ b/website/www/site/content/en/get-started/quickstart-go.md
@@ -69,8 +69,17 @@ $ wordcount --input gs://dataflow-samples/shakespeare/kinglear.txt \
--worker_harness_container_image=apache/beam_go_sdk:latest
{{< /highlight >}}
-{{< highlight class="runner-nemo" >}}
-This runner is not yet available for the Go SDK.
+{{< highlight class="runner-spark" >}}
+# Build and run the Spark job server from Beam source.
+# -PsparkMasterUrl is optional. If it is unset the job will be run inside an embedded Spark cluster.
+$ ./gradlew :runners:spark:job-server:runShadow -PsparkMasterUrl=spark://localhost:7077
+
+# In a separate terminal, run:
+$ go install github.com/apache/beam/sdks/go/examples/wordcount
+$ wordcount --input <PATH_TO_INPUT_FILE> \
+ --output counts \
+ --runner spark \
+ --endpoint localhost:8099
{{< /highlight >}}
## Next Steps