You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/06/13 10:37:30 UTC
[spark] branch master updated: [SPARK-44038][DOCS][K8S] Update YuniKorn docs with v1.3
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 223c196d242 [SPARK-44038][DOCS][K8S] Update YuniKorn docs with v1.3
223c196d242 is described below
commit 223c196d242f9c79ca76b226bf9489759652203a
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Tue Jun 13 03:37:08 2023 -0700
[SPARK-44038][DOCS][K8S] Update YuniKorn docs with v1.3
### What changes were proposed in this pull request?
This PR aims to update `Apache YuniKorn` batch scheduler docs with v1.3.0 to recommend it in Apache Spark 3.4.1 and 3.5.0 users.
### Why are the changes needed?
Apache YuniKorn v1.3.0 was released on 2023-06-12 with 160 resolved JIRAs.
https://yunikorn.apache.org/release-announce/1.3.0
I installed YuniKorn v1.3.0 and tested manually.
```
$ helm list -n yunikorn
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
yunikorn yunikorn 1 2023-06-13 01:56:32.784863 -0700 PDT deployed yunikorn-1.3.0
```
```
$ build/sbt -Pkubernetes -Pkubernetes-integration-tests -Dspark.kubernetes.test.deployMode=docker-desktop "kubernetes-integration-tests/testOnly *.YuniKornSuite" -Dtest.exclude.tags=minikube,local,decom,r -Dtest.default.exclude.tags=
...
[info] YuniKornSuite:
[info] - SPARK-42190: Run SparkPi with local[*] (12 seconds, 49 milliseconds)
[info] - Run SparkPi with no resources (20 seconds, 378 milliseconds)
[info] - Run SparkPi with no resources & statefulset allocation (20 seconds, 583 milliseconds)
[info] - Run SparkPi with a very long application name. (20 seconds, 606 milliseconds)
[info] - Use SparkLauncher.NO_RESOURCE (19 seconds, 676 milliseconds)
[info] - Run SparkPi with a master URL without a scheme. (20 seconds, 631 milliseconds)
[info] - Run SparkPi with an argument. (22 seconds, 320 milliseconds)
[info] - Run SparkPi with custom labels, annotations, and environment variables. (20 seconds, 469 milliseconds)
[info] - All pods have the same service account by default (22 seconds, 537 milliseconds)
[info] - Run extraJVMOptions check on driver (12 seconds, 268 milliseconds)
...
```
```
$ k describe pod spark-test-app-33ec515e453e4301a90f626812db1153-driver -n spark-dbe522106eac40d4a17447bfa2947c45
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduling 82s yunikorn spark-dbe522106eac40d4a17447bfa2947c45/spark-test-app-33ec515e453e4301a90f626812db1153-driver is queued and waiting for allocation
Normal Scheduled 82s yunikorn Successfully assigned spark-dbe522106eac40d4a17447bfa2947c45/spark-test-app-33ec515e453e4301a90f626812db1153-driver to node docker-desktop
Normal PodBindSuccessful 82s yunikorn Pod spark-dbe522106eac40d4a17447bfa2947c45/spark-test-app-33ec515e453e4301a90f626812db1153-driver is successfully bound to node docker-desktop
Normal Pulled 82s kubelet Container image "docker.io/kubespark/spark:dev" already present on machine
Normal Created 82s kubelet Created container spark-kubernetes-driver
Normal Started 82s kubelet Started container spark-kubernetes-driver
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manual review.
Closes #41571 from dongjoon-hyun/SPARK-44038.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
docs/running-on-kubernetes.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/running-on-kubernetes.md b/docs/running-on-kubernetes.md
index 1a5f2743405..ac3051b9cbf 100644
--- a/docs/running-on-kubernetes.md
+++ b/docs/running-on-kubernetes.md
@@ -1917,10 +1917,10 @@ Install Apache YuniKorn:
```bash
helm repo add yunikorn https://apache.github.io/yunikorn-release
helm repo update
-helm install yunikorn yunikorn/yunikorn --namespace yunikorn --version 1.2.0 --create-namespace --set embedAdmissionController=false
+helm install yunikorn yunikorn/yunikorn --namespace yunikorn --version 1.3.0 --create-namespace --set embedAdmissionController=false
```
-The above steps will install YuniKorn v1.2.0 on an existing Kubernetes cluster.
+The above steps will install YuniKorn v1.3.0 on an existing Kubernetes cluster.
##### Get started
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org