You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/03/16 07:26:03 UTC
[GitHub] [spark] martin-g commented on a change in pull request #35869: [SPARK-38561][K8S][DOCS] Add doc for `Customized Kubernetes Schedulers`
martin-g commented on a change in pull request #35869:
URL: https://github.com/apache/spark/pull/35869#discussion_r827681954
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,25 @@ spec:
image: will-be-overwritten
```
+#### Customized Kubernetes Schedulers for Spark on Kubernetes
+
+Spark allows users to specify a customized scheduler as Spark on Kubernetes scheduler.
Review comment:
```suggestion
Spark allows users to specify a custom Kubernetes schedulers.
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,25 @@ spec:
image: will-be-overwritten
```
+#### Customized Kubernetes Schedulers for Spark on Kubernetes
+
+Spark allows users to specify a customized scheduler as Spark on Kubernetes scheduler.
+
+1. Specify scheduler name.
+
+ Users can specify customized scheduler using <code>spark.kubernetes.scheduler.name</code> or
Review comment:
```suggestion
Users can specify a custom scheduler using <code>spark.kubernetes.scheduler.name</code> or
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,25 @@ spec:
image: will-be-overwritten
```
+#### Customized Kubernetes Schedulers for Spark on Kubernetes
+
+Spark allows users to specify a customized scheduler as Spark on Kubernetes scheduler.
+
+1. Specify scheduler name.
+
+ Users can specify customized scheduler using <code>spark.kubernetes.scheduler.name</code> or
+ <code>spark.kubernetes.{driver/executor}.scheduler.name</code> configuration.
+
+2. Specify scheduler related configurations.
+
+ Users can use [Pod template](#pod-template), existing configurations to specify label (<code>spark.kubernetes.{driver,executor}.label.*</code>), annotations style (<code>spark.kubernetes.{driver/executor}.annotation.*</code>) scheduler hints.
+
+3. Specify scheduler feature step.
+
+ Users may also consider to use <code>spark.kubernetes.{driver/executor}.pod.featureSteps</code> to support more complex requirements and more centralized scheduler hints configure, included but not limited to:
+ - Creating a scheduler needed additional Kubernetes custom resource for driver/executor scheduling.
Review comment:
```suggestion
- Create additional Kubernetes custom resources for driver/executor scheduling.
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,25 @@ spec:
image: will-be-overwritten
```
+#### Customized Kubernetes Schedulers for Spark on Kubernetes
+
+Spark allows users to specify a customized scheduler as Spark on Kubernetes scheduler.
+
+1. Specify scheduler name.
+
+ Users can specify customized scheduler using <code>spark.kubernetes.scheduler.name</code> or
+ <code>spark.kubernetes.{driver/executor}.scheduler.name</code> configuration.
+
+2. Specify scheduler related configurations.
+
+ Users can use [Pod template](#pod-template), existing configurations to specify label (<code>spark.kubernetes.{driver,executor}.label.*</code>), annotations style (<code>spark.kubernetes.{driver/executor}.annotation.*</code>) scheduler hints.
Review comment:
```suggestion
To configure the custom scheduler the user can use [Pod templates](#pod-template), add labels (<code>spark.kubernetes.{driver,executor}.label.*</code>) and/or annotations (<code>spark.kubernetes.{driver/executor}.annotation.*</code>).
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,25 @@ spec:
image: will-be-overwritten
```
+#### Customized Kubernetes Schedulers for Spark on Kubernetes
+
+Spark allows users to specify a customized scheduler as Spark on Kubernetes scheduler.
+
+1. Specify scheduler name.
+
+ Users can specify customized scheduler using <code>spark.kubernetes.scheduler.name</code> or
+ <code>spark.kubernetes.{driver/executor}.scheduler.name</code> configuration.
+
+2. Specify scheduler related configurations.
+
+ Users can use [Pod template](#pod-template), existing configurations to specify label (<code>spark.kubernetes.{driver,executor}.label.*</code>), annotations style (<code>spark.kubernetes.{driver/executor}.annotation.*</code>) scheduler hints.
+
+3. Specify scheduler feature step.
+
+ Users may also consider to use <code>spark.kubernetes.{driver/executor}.pod.featureSteps</code> to support more complex requirements and more centralized scheduler hints configure, included but not limited to:
Review comment:
```suggestion
Users may also consider to use <code>spark.kubernetes.{driver/executor}.pod.featureSteps</code> to support more complex requirements, including but not limited to:
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,25 @@ spec:
image: will-be-overwritten
```
+#### Customized Kubernetes Schedulers for Spark on Kubernetes
+
+Spark allows users to specify a customized scheduler as Spark on Kubernetes scheduler.
+
+1. Specify scheduler name.
+
+ Users can specify customized scheduler using <code>spark.kubernetes.scheduler.name</code> or
+ <code>spark.kubernetes.{driver/executor}.scheduler.name</code> configuration.
+
+2. Specify scheduler related configurations.
+
+ Users can use [Pod template](#pod-template), existing configurations to specify label (<code>spark.kubernetes.{driver,executor}.label.*</code>), annotations style (<code>spark.kubernetes.{driver/executor}.annotation.*</code>) scheduler hints.
+
+3. Specify scheduler feature step.
+
+ Users may also consider to use <code>spark.kubernetes.{driver/executor}.pod.featureSteps</code> to support more complex requirements and more centralized scheduler hints configure, included but not limited to:
+ - Creating a scheduler needed additional Kubernetes custom resource for driver/executor scheduling.
+ - Setting scheduler hints according to configuration or existing Pod info dynamically.
Review comment:
```suggestion
- Set scheduler hints according to configuration or existing Pod info dynamically.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org