You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sh...@apache.org on 2021/08/02 17:16:06 UTC
[spark-website] branch asf-site updated: updating local
k8s/minikube testing instructions
This is an automated email from the ASF dual-hosted git repository.
shaneknapp pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/spark-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new d60611b updating local k8s/minikube testing instructions
d60611b is described below
commit d60611b5c464db0fb99ca072a9d8b55e824ca7c2
Author: shane knapp <in...@gmail.com>
AuthorDate: Mon Aug 2 10:15:46 2021 -0700
updating local k8s/minikube testing instructions
a small update to the k8s/minikube integration test instructions
Author: shane knapp <in...@gmail.com>
Closes #351 from shaneknapp/updating-k8s-docs.
---
developer-tools.md | 9 ++++++---
site/developer-tools.html | 9 ++++++---
2 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/developer-tools.md b/developer-tools.md
index 9551533..bf3ee6e 100644
--- a/developer-tools.md
+++ b/developer-tools.md
@@ -169,8 +169,8 @@ Please check other available options via `python/run-tests[-with-coverage] --hel
If you have made changes to the K8S bindings in Apache Spark, it would behoove you to test locally before submitting a PR. This is relatively simple to do, but it will require a local (to you) installation of [minikube](https://kubernetes.io/docs/setup/minikube/). Due to how minikube interacts with the host system, please be sure to set things up as follows:
-- minikube version v1.7.3 (or greater)
-- You must use a VM driver! Running minikube with the `--vm-driver=none` option requires that the user launching minikube/k8s have root access. Our Jenkins workers use the [kvm2](https://minikube.sigs.k8s.io/docs/drivers/kvm2/) drivers. More details [here](https://minikube.sigs.k8s.io/docs/drivers/).
+- minikube version v1.18.1 (or greater)
+- You must use a VM driver! Running minikube with the `--vm-driver=none` option requires that the user launching minikube/k8s have root access, which could impact how the tests are run. Our Jenkins workers use the default [docker](https://minikube.sigs.k8s.io/docs/drivers/docker/) drivers. More details [here](https://minikube.sigs.k8s.io/docs/drivers/).
- kubernetes version v1.17.3 (can be set by executing `minikube config set kubernetes-version v1.17.3`)
- the current kubernetes context must be minikube's default context (called 'minikube'). This can be selected by `minikube kubectl -- config use-context minikube`. This is only needed when after minikube is started another kubernetes context is selected.
@@ -196,8 +196,11 @@ PVC_TMP_DIR=$(mktemp -d)
export PVC_TESTS_HOST_PATH=$PVC_TMP_DIR
export PVC_TESTS_VM_PATH=$PVC_TMP_DIR
-minikube --vm-driver=<YOUR VM DRIVER HERE> start --memory 6000 --cpus 8
minikube config set kubernetes-version v1.17.3
+minikube --vm-driver=<YOUR VM DRIVER HERE> start --memory 6000 --cpus 8
+
+# for macos only (see https://github.com/apache/spark/pull/32793):
+# minikube ssh "sudo useradd spark -u 185 -g 0 -m -s /bin/bash"
minikube mount ${PVC_TESTS_HOST_PATH}:${PVC_TESTS_VM_PATH} --9p-version=9p2000.L --gid=0 --uid=185 &; MOUNT_PID=$!
diff --git a/site/developer-tools.html b/site/developer-tools.html
index 5558945..81d64c7 100644
--- a/site/developer-tools.html
+++ b/site/developer-tools.html
@@ -348,8 +348,8 @@ Generating HTML files for PySpark coverage under /.../spark/python/test_coverage
<p>If you have made changes to the K8S bindings in Apache Spark, it would behoove you to test locally before submitting a PR. This is relatively simple to do, but it will require a local (to you) installation of <a href="https://kubernetes.io/docs/setup/minikube/">minikube</a>. Due to how minikube interacts with the host system, please be sure to set things up as follows:</p>
<ul>
- <li>minikube version v1.7.3 (or greater)</li>
- <li>You must use a VM driver! Running minikube with the <code class="language-plaintext highlighter-rouge">--vm-driver=none</code> option requires that the user launching minikube/k8s have root access. Our Jenkins workers use the <a href="https://minikube.sigs.k8s.io/docs/drivers/kvm2/">kvm2</a> drivers. More details <a href="https://minikube.sigs.k8s.io/docs/drivers/">here</a>.</li>
+ <li>minikube version v1.18.1 (or greater)</li>
+ <li>You must use a VM driver! Running minikube with the <code class="language-plaintext highlighter-rouge">--vm-driver=none</code> option requires that the user launching minikube/k8s have root access, which could impact how the tests are run. Our Jenkins workers use the default <a href="https://minikube.sigs.k8s.io/docs/drivers/docker/">docker</a> drivers. More details <a href="https://minikube.sigs.k8s.io/docs/drivers/">here</a>.</li>
<li>kubernetes version v1.17.3 (can be set by executing <code class="language-plaintext highlighter-rouge">minikube config set kubernetes-version v1.17.3</code>)</li>
<li>the current kubernetes context must be minikube’s default context (called ‘minikube’). This can be selected by <code class="language-plaintext highlighter-rouge">minikube kubectl -- config use-context minikube</code>. This is only needed when after minikube is started another kubernetes context is selected.</li>
</ul>
@@ -374,8 +374,11 @@ export TARBALL_TO_TEST=($(pwd)/spark-*${DATE}-${REVISION}.tgz)
export PVC_TESTS_HOST_PATH=$PVC_TMP_DIR
export PVC_TESTS_VM_PATH=$PVC_TMP_DIR
-minikube --vm-driver=<YOUR VM DRIVER HERE> start --memory 6000 --cpus 8
minikube config set kubernetes-version v1.17.3
+minikube --vm-driver=<YOUR VM DRIVER HERE> start --memory 6000 --cpus 8
+
+# for macos only (see https://github.com/apache/spark/pull/32793):
+# minikube ssh "sudo useradd spark -u 185 -g 0 -m -s /bin/bash"
minikube mount ${PVC_TESTS_HOST_PATH}:${PVC_TESTS_VM_PATH} --9p-version=9p2000.L --gid=0 --uid=185 &; MOUNT_PID=$!
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org