You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2023/09/18 12:31:54 UTC
[spark-website] branch asf-site updated: [SPARK-45195] Update examples with docker official image
This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/spark-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 6b10f7fd85 [SPARK-45195] Update examples with docker official image
6b10f7fd85 is described below
commit 6b10f7fd85327f97cc12bede9ce5c60a744d9063
Author: Ruifeng Zheng <ru...@apache.org>
AuthorDate: Mon Sep 18 07:31:46 2023 -0500
[SPARK-45195] Update examples with docker official image
1, add `docker run` commands for PySpark and SparkR;
2, switch to docker official image for SQL, Scala and Java;
refer to https://hub.docker.com/_/spark
also manually checked all the commands, e,g,:
```
ruifeng.zhengxxxxx:~$ docker run -it --rm spark:python3 /opt/spark/bin/pyspark
Python 3.8.10 (default, May 26 2023, 14:05:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
23/09/18 06:02:30 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/__ / .__/\_,_/_/ /_/\_\ version 3.5.0
/_/
Using Python version 3.8.10 (default, May 26 2023 14:05:08)
Spark context Web UI available at http://4861f70118ab:4040
Spark context available as 'sc' (master = local[*], app id = local-1695016951087).
SparkSession available as 'spark'.
>>> spark.range(0, 10).show()
+---+
| id|
+---+
| 0|
| 1|
| 2|
| 3|
| 4|
| 5|
| 6|
| 7|
| 8|
| 9|
+---+
```
Author: Ruifeng Zheng <ru...@apache.org>
Closes #477 from zhengruifeng/offical_image.
---
index.md | 12 +++++++-----
site/index.html | 12 +++++++-----
2 files changed, 14 insertions(+), 10 deletions(-)
diff --git a/index.md b/index.md
index a41e4c9e81..ada6242742 100644
--- a/index.md
+++ b/index.md
@@ -88,11 +88,13 @@ navigation:
<div class="tab-content py-5 spark-install" id="nav-tabContent">
<div class="tab-pane fade show active" id="nav-python" role="tabpanel" aria-labelledby="nav-python-tab">
<div class="mb-2 title">Run now</div>
- <div style="font-size: 16px;">Installing with 'pip'
+ <div style="font-size: 16px;">Install with 'pip' or try offical image
</div>
<div class="code">
<p>$ pip install pyspark</p>
<p>$ pyspark</p>
+ <p>$ </p>
+ <p>$ docker run -it --rm spark:python3 /opt/spark/bin/pyspark</p>
</div>
<div class="examples mt-5">
<div class="window"><span class="circle red"></span><span class="circle yellow"></span><span
@@ -155,7 +157,7 @@ filtered_df.summary().show()
<div class="tab-pane fade" id="nav-sql" role="tabpanel" aria-labelledby="nav-sql-tab">
<div class="mb-2 title">Run now</div>
<div class="code">
- <p>$ docker run -it --rm apache/spark /opt/spark/bin/spark-sql</p>
+ <p>$ docker run -it --rm spark /opt/spark/bin/spark-sql</p>
<p>spark-sql></p>
</div>
<div class="examples mt-5">
@@ -175,7 +177,7 @@ FROM json.`logs.json`
<div class="tab-pane fade" id="nav-scala" role="tabpanel" aria-labelledby="nav-scala-tab">
<div class="mb-2 title">Run now</div>
<div class="code">
- <p>$ docker run -it --rm apache/spark /opt/spark/bin/spark-shell</p>
+ <p>$ docker run -it --rm spark /opt/spark/bin/spark-shell</p>
<p>scala></p>
</div>
<div class="examples mt-5">
@@ -193,7 +195,7 @@ df.where("age > 21")
<div class="tab-pane fade" id="nav-java" role="tabpanel" aria-labelledby="nav-java-tab">
<div class="mb-2 title">Run now</div>
<div class="code">
- <p>$ docker run -it --rm apache/spark /opt/spark/bin/spark-shell</p>
+ <p>$ docker run -it --rm spark /opt/spark/bin/spark-shell</p>
<p>scala></p>
</div>
<div class="examples mt-5">
@@ -210,7 +212,7 @@ df.where("age > 21")
<div class="tab-pane fade" id="nav-r" role="tabpanel" aria-labelledby="nav-r-tab">
<div class="mb-2 title">Run now</div>
<div class="code">
- <p>$ SPARK-HOME/bin/sparkR</p>
+ <p>$ docker run -it --rm spark:r /opt/spark/bin/sparkR</p>
<p>></p>
</div>
<div class="examples mt-5">
diff --git a/site/index.html b/site/index.html
index e1b0b7e416..3ccc7104ce 100644
--- a/site/index.html
+++ b/site/index.html
@@ -213,11 +213,13 @@
<div class="tab-content py-5 spark-install" id="nav-tabContent">
<div class="tab-pane fade show active" id="nav-python" role="tabpanel" aria-labelledby="nav-python-tab">
<div class="mb-2 title">Run now</div>
- <div style="font-size: 16px;">Installing with 'pip'
+ <div style="font-size: 16px;">Install with 'pip' or try offical image
</div>
<div class="code">
<p>$ pip install pyspark</p>
<p>$ pyspark</p>
+ <p>$ </p>
+ <p>$ docker run -it --rm spark:python3 /opt/spark/bin/pyspark</p>
</div>
<div class="examples mt-5">
<div class="window"><span class="circle red"></span><span class="circle yellow"></span><span class="circle green"></span></div>
@@ -273,7 +275,7 @@
<div class="tab-pane fade" id="nav-sql" role="tabpanel" aria-labelledby="nav-sql-tab">
<div class="mb-2 title">Run now</div>
<div class="code">
- <p>$ docker run -it --rm apache/spark /opt/spark/bin/spark-sql</p>
+ <p>$ docker run -it --rm spark /opt/spark/bin/spark-sql</p>
<p>spark-sql></p>
</div>
<div class="examples mt-5">
@@ -293,7 +295,7 @@
<div class="tab-pane fade" id="nav-scala" role="tabpanel" aria-labelledby="nav-scala-tab">
<div class="mb-2 title">Run now</div>
<div class="code">
- <p>$ docker run -it --rm apache/spark /opt/spark/bin/spark-shell</p>
+ <p>$ docker run -it --rm spark /opt/spark/bin/spark-shell</p>
<p>scala></p>
</div>
<div class="examples mt-5">
@@ -310,7 +312,7 @@
<div class="tab-pane fade" id="nav-java" role="tabpanel" aria-labelledby="nav-java-tab">
<div class="mb-2 title">Run now</div>
<div class="code">
- <p>$ docker run -it --rm apache/spark /opt/spark/bin/spark-shell</p>
+ <p>$ docker run -it --rm spark /opt/spark/bin/spark-shell</p>
<p>scala></p>
</div>
<div class="examples mt-5">
@@ -327,7 +329,7 @@
<div class="tab-pane fade" id="nav-r" role="tabpanel" aria-labelledby="nav-r-tab">
<div class="mb-2 title">Run now</div>
<div class="code">
- <p>$ SPARK-HOME/bin/sparkR</p>
+ <p>$ docker run -it --rm spark:r /opt/spark/bin/sparkR</p>
<p>></p>
</div>
<div class="examples mt-5">
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org