You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@yunikorn.apache.org by yu...@apache.org on 2022/11/28 13:41:25 UTC
[yunikorn-site] branch master updated: [YUNIKORN-1390] MPIOperator Example to Yunikorn Site(#208)
This is an automated email from the ASF dual-hosted git repository.
yuteng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/yunikorn-site.git
The following commit(s) were added to refs/heads/master by this push:
new b5f437a8f [YUNIKORN-1390] MPIOperator Example to Yunikorn Site(#208)
b5f437a8f is described below
commit b5f437a8f4d44d04623971d2d94e852500edea8b
Author: Kevin Hannon <ka...@gmail.com>
AuthorDate: Mon Nov 28 08:41:18 2022 -0500
[YUNIKORN-1390] MPIOperator Example to Yunikorn Site(#208)
---
docs/user_guide/workloads/run_mpi.md | 112 +++++++++++++++++++++++++
docs/user_guide/workloads/workload_overview.md | 1 +
sidebars.js | 3 +-
3 files changed, 115 insertions(+), 1 deletion(-)
diff --git a/docs/user_guide/workloads/run_mpi.md b/docs/user_guide/workloads/run_mpi.md
new file mode 100644
index 000000000..f69f8c705
--- /dev/null
+++ b/docs/user_guide/workloads/run_mpi.md
@@ -0,0 +1,112 @@
+---
+id: run_mpi
+title: Run MPI Jobs
+description: How to run MPI jobs with YuniKorn
+keywords:
+ - mpi
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+This guide walks through how to setup the [MPI Operator](https://github.com/kubeflow/mpi-operator) and to how to run a MPIJob with the YuniKorn scheduler.
+
+## Installing the MPI Operator
+
+You can use the following command to install the mpi operator. If you have problems with installation,
+please refer to [this doc](https://github.com/kubeflow/mpi-operator) for details.
+```
+kubectl create -f https://raw.githubusercontent.com/kubeflow/mpi-operator/master/deploy/v2beta1/mpi-operator.yaml
+```
+
+## Run a MPI Job
+
+This example shows to run a pure MPI application.
+
+The program prints some basic information about the workers.
+Then, it calculates an approximate value for pi.
+
+Here is a Pi YAML example [example](https://github.com/apache/yunikorn-k8shim/blob/master/deployments/examples/mpioperator/Pi/pi.yaml).
+```yaml
+apiVersion: kubeflow.org/v2beta1
+kind: MPIJob
+metadata:
+ name: pi
+spec:
+ slotsPerWorker: 1
+ runPolicy:
+ cleanPodPolicy: Running
+ ttlSecondsAfterFinished: 60
+ sshAuthMountPath: /home/mpiuser/.ssh
+ mpiReplicaSpecs:
+ Launcher:
+ replicas: 1
+ template:
+ labels:
+ applicationId: "mpi_job_pi"
+ queue: root.mpi
+ spec:
+ schedulerName: yunikorn
+ containers:
+ - image: mpioperator/mpi-pi
+ name: mpi-launcher
+ securityContext:
+ runAsUser: 1000
+ command:
+ - mpirun
+ args:
+ - -n
+ - "2"
+ - /home/mpiuser/pi
+ resources:
+ limits:
+ cpu: 1
+ memory: 1Gi
+ Worker:
+ replicas: 2
+ template:
+ labels:
+ applicationId: "mpi_job_pi"
+ queue: root.mpi
+ spec:
+ schedulerName: yunikorn
+ containers:
+ - image: mpioperator/mpi-pi
+ name: mpi-worker
+ securityContext:
+ runAsUser: 1000
+ command:
+ - /usr/sbin/sshd
+ args:
+ - -De
+ - -f
+ - /home/mpiuser/.sshd_config
+ resources:
+ limits:
+ cpu: 1
+ memory: 1Gi
+```
+Create the MPIJob.
+```
+kubectl create -f deployments/examples/mpioperator/Pi/pi.yaml
+```
+
+We added Yunikorn labels to the Pi example to demonstrate using the yunikorn scheduler.
+
+
diff --git a/docs/user_guide/workloads/workload_overview.md b/docs/user_guide/workloads/workload_overview.md
index cae7392bf..c8722c7fe 100644
--- a/docs/user_guide/workloads/workload_overview.md
+++ b/docs/user_guide/workloads/workload_overview.md
@@ -56,3 +56,4 @@ Examples of more advanced use cases can be found here:
* [Run Spark Jobs](run_spark)
* [Run Flink Jobs](run_flink)
* [Run TensorFlow Jobs](run_tf)
+* [Run MPI Jobs](run_mpi)
diff --git a/sidebars.js b/sidebars.js
index c8b862afa..7f8eb922c 100644
--- a/sidebars.js
+++ b/sidebars.js
@@ -38,7 +38,8 @@ module.exports = {
'user_guide/workloads/workload_overview',
'user_guide/workloads/run_spark',
'user_guide/workloads/run_flink',
- 'user_guide/workloads/run_tf'
+ 'user_guide/workloads/run_tf',
+ 'user_guide/workloads/run_mpi'
],
},
{