You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/03/04 06:44:03 UTC
[GitHub] [spark] martin-g commented on a change in pull request #35728: [SPARK-38189][K8S][DOC] Add `Priority scheduling` doc for Spark on K8S
martin-g commented on a change in pull request #35728:
URL: https://github.com/apache/spark/pull/35728#discussion_r819304189
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1707,6 +1707,30 @@ Spark automatically handles translating the Spark configs <code>spark.{driver/ex
Kubernetes does not tell Spark the addresses of the resources allocated to each container. For that reason, the user must specify a discovery script that gets run by the executor on startup to discover what resources are available to that executor. You can find an example scripts in `examples/src/main/scripts/getGpusResources.sh`. The script must have execute permissions set and the user should setup permissions to not allow malicious users to modify it. The script should write to STDOUT a JSON string in the format of the ResourceInformation class. This has the resource name and an array of resource addresses available to just that executor.
+### Resource Level Scheduling Overview
+
+There are several resource level scheduling features are supported by Spark on Kubernetes.
Review comment:
```suggestion
There are several resource level scheduling features supported by Spark on Kubernetes.
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1707,6 +1707,30 @@ Spark automatically handles translating the Spark configs <code>spark.{driver/ex
Kubernetes does not tell Spark the addresses of the resources allocated to each container. For that reason, the user must specify a discovery script that gets run by the executor on startup to discover what resources are available to that executor. You can find an example scripts in `examples/src/main/scripts/getGpusResources.sh`. The script must have execute permissions set and the user should setup permissions to not allow malicious users to modify it. The script should write to STDOUT a JSON string in the format of the ResourceInformation class. This has the resource name and an array of resource addresses available to just that executor.
+### Resource Level Scheduling Overview
+
+There are several resource level scheduling features are supported by Spark on Kubernetes.
+
+#### Priority Scheduling
+
+Kubernetes support [Pod priority](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption) by default.
Review comment:
```suggestion
Kubernetes supports [Pod priority](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption) by default.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org