You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/03/16 04:10:49 UTC
[GitHub] [spark] Yikun opened a new pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Yikun opened a new pull request #35870:
URL: https://github.com/apache/spark/pull/35870
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
If benchmark tests were added, please run the benchmarks in GitHub Actions for the consistent environment, and the instructions could accord to: https://spark.apache.org/developer-tools.html#github-workflow-benchmarks.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837103346
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] martin-g commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
martin-g commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837104858
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
Review comment:
Also I think you can remove `similarly`. The sentence starts with `similar` already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837032645
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
Review comment:
Here, `PodGroup` should be linked to Volcano homepage's PodGroup definition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837102913
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
Review comment:
done
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
Review comment:
done
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
Review comment:
done
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #35870:
URL: https://github.com/apache/spark/pull/35870#issuecomment-1073445150
@holdenk @martin-g comment addressed, Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] holdenk commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
holdenk commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r828309309
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create Volcano PodGroup and set driver/executor pod annotation to link this PodGroup.
+
+Note that, currently only supported driver/job level PodGroup in Volcano Feature Step, executor separate PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
+<table class="table">
+<tr><th>Scheduling</th><th>Description</th><th>Configuration</th></tr>
+<tr>
+ <td>Queue scheduling</td>
+ <td>
+ Queue indicates the resource queue, which adopts FIFO. is also used as the basis for resource division.
+ help users specify which queue the job to submit.
+ </td>
+ <td>`spec.queue` field in PodGroup template</td>
+</tr>
+<tr>
+ <td>Resource reservation</td>
+ <td>
+ Resource reservation, aka `Gang` scheduling (start all or nothing), helps users reserve resources for specific jobs.
+ It's useful for ensuring resource are meet the minimum requirements of spark job and avoiding all drivers stuck
+ due to all executor pending, especially, when cluster resources are very limited.
+ </td>
+ <td>`spec.minResources` field in PodGroup template</td>
+</tr>
+<tr>
+ <td>Priority scheduling</td>
+ <td>
+ It is used to help users to specify job priority in the queue during scheduling.
Review comment:
Maybe add a link here
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
Review comment:
+1 clarifying versions
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create Volcano PodGroup and set driver/executor pod annotation to link this PodGroup.
+
+Note that, currently only supported driver/job level PodGroup in Volcano Feature Step, executor separate PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
+<table class="table">
+<tr><th>Scheduling</th><th>Description</th><th>Configuration</th></tr>
+<tr>
+ <td>Queue scheduling</td>
+ <td>
+ Queue indicates the resource queue, which adopts FIFO. is also used as the basis for resource division.
+ help users specify which queue the job to submit.
+ </td>
+ <td>`spec.queue` field in PodGroup template</td>
+</tr>
+<tr>
+ <td>Resource reservation</td>
+ <td>
+ Resource reservation, aka `Gang` scheduling (start all or nothing), helps users reserve resources for specific jobs.
+ It's useful for ensuring resource are meet the minimum requirements of spark job and avoiding all drivers stuck
Review comment:
I think maybe, `It is useful for ensource the available resources meet the minimum requirements of the Spark job and avoiding the situation where drivers are scheduled and then they are unable to schedule sufficient executors to progress.`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] holdenk commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
holdenk commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r828310472
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
Review comment:
Do we also want to call out that this feature may change (e.g. first pass at the API)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837110029
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun edited a comment on pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun edited a comment on pull request #35870:
URL: https://github.com/apache/spark/pull/35870#issuecomment-1081491696
@dongjoon-hyun Thanks for review, addressed. https://github.com/apache/spark/pull/35870/commits/75accf88ee8ca4e989d7f7fd72480ccea9053353
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837110170
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #35870:
URL: https://github.com/apache/spark/pull/35870#issuecomment-1081360954
@dongjoon-hyun @holdenk Would you mind take a look again on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837032176
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
Review comment:
`help users to create` -> `help users create`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r830729996
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
Review comment:
```
This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.
```
Add soemthing like this. : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r834087890
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
Review comment:
The latest doc has been added the description for install specific version volcano:
> You can also replace master of above url with specific tag/branch (such as release-1.5 branch for latest v1.5.x version, v1.5.1 tag for v1.5.1 version) to install Volcano with specific version.
Once this https://github.com/volcano-sh/volcano/issues/2080 supported by volcano, we could do some improvement for this later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #35870:
URL: https://github.com/apache/spark/pull/35870#issuecomment-1082207419
I backported this to branch-3.3, too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837035197
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
Review comment:
For consistency, please add a comment for `queue` like the other items. `queue` is new concept to the users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837111337
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
Review comment:
Done
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837035020
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
Review comment:
Please be clear to use one of the existing K8s priority. Otherwise, the reader may be confused if `high-priority` is provided by Volcano like the built-in `default` queue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837102501
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
Review comment:
sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r833885234
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step. Executor PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
Review comment:
My intention was this is a list to show some information for spark scenarioes, I personaly think it might be better to keep
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #35870:
URL: https://github.com/apache/spark/pull/35870#issuecomment-1077414507
@dongjoon-hyun comments addressed, Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837123270
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
Review comment:
@martin-g done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] william-wang commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
william-wang commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837236032
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
Review comment:
@dongjoon-hyun Thanks for the reminding, we are already working on the multi-arch and will make it ready as soon as possible. For the v1beta1, I understand your concern, we have stayed in v1beta1 for a long time, it is stable and there are 50+ enterprise users use Volcano in their production environment. We have planed to upgrade it to v1 version this year :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #35870:
URL: https://github.com/apache/spark/pull/35870
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] william-wang commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
william-wang commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837236032
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
Review comment:
@dongjoon-hyun Thanks for the reminding, we are already working on the multi-arch and will make it ready as soon as possible. For the v1beta1, I understand your concern, we have stayed in v1beta1 for a long time, it is stable and there are 50+ enterprise users use Volcano in their production environment. We have planed to upgrade it to v1 version this year and v1 will also keep complete compatibility with v1beta :)
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
Review comment:
@dongjoon-hyun Thanks for the reminding, we are already working on the multi-arch and will make it ready as soon as possible. For the v1beta1, I understand your concern, we have stayed in v1beta1 for a long time, it is stable and there are 50+ enterprise users use Volcano in their production environment. We have planed to upgrade it to v1 version this year and v1 will also keep complete compatibility with v1beta1 :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837756823
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
Review comment:
Thank you for the confirmation, @william-wang .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837031801
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
Review comment:
`Volcano` is a reference model for all the other custom scheduler. Shall we move `PodGroup` to the top like this?
```
# Specify volcano scheduler and PodGroup template
--conf spark.kubernetes.scheduler.name=volcano
--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
# Specify driver/executor VolcanoFeatureStep
--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837034463
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
Review comment:
`make driver` -> `make a driver pod`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837120714
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837034295
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
Review comment:
As a side note, `v1beta1` could be a blocker for the users until Volcano community makes it official. I mentioned this before to you, @Yikun . The multi-arch image support is also another TODO, IIRC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837034660
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
Review comment:
Could you be clear to the customer if this should include `driver pod resource` + `executor pod resource`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837102625
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837033127
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
Review comment:
`properties` -> `property`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837030130
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
Review comment:
I mean this is still not addressed yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r833854425
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
Review comment:
I don't think this is a good way because it's pointing `master` branch which is not `v1.5.1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r833855300
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step. Executor PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
Review comment:
The volcano link will be enough instead of enumerating here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837031061
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
Review comment:
Could you remove `-Pmesos` and `-Pyarn` because this is `running-on-kubernetes.md` document?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #35870:
URL: https://github.com/apache/spark/pull/35870#issuecomment-1081491696
@dongjoon-hyun Thanks for review, addressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837033696
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
Review comment:
Shall we remove `, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction)`? If we have a correct link on `PodGroup` before, I believe we don't need to link `Introduction` here again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837107382
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
Review comment:
> I mentioned this before to you, @Yikun . The multi-arch image support is also another TODO, IIRC.
also cc @william-wang are working on this, will ready soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837118505
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step. Executor PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
Review comment:
Okay, I cleaned up this section, and plus some notes in podgroup template
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #35870:
URL: https://github.com/apache/spark/pull/35870#issuecomment-1069821041
@holdenk Thanks for your suggestion, will address soon!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] martin-g commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
martin-g commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r827913886
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
Review comment:
```suggestion
* Spark on Kubernetes with Volcano as a scheduler is supported since Spark v3.3.0 and Volcano v1.5.1. See also [Volcano installation](https://volcano.sh/en/docs/installation).
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
Review comment:
```suggestion
Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create Volcano PodGroup and set driver/executor pod annotation to link this PodGroup.
+
+Note that, currently only supported driver/job level PodGroup in Volcano Feature Step, executor separate PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
Review comment:
```suggestion
To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
Review comment:
```suggestion
support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create Volcano PodGroup and set driver/executor pod annotation to link this PodGroup.
+
+Note that, currently only supported driver/job level PodGroup in Volcano Feature Step, executor separate PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
+<table class="table">
+<tr><th>Scheduling</th><th>Description</th><th>Configuration</th></tr>
+<tr>
+ <td>Queue scheduling</td>
+ <td>
+ Queue indicates the resource queue, which adopts FIFO. is also used as the basis for resource division.
Review comment:
Maybe also add a link to Volcano documentation about Queue ?
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create Volcano PodGroup and set driver/executor pod annotation to link this PodGroup.
Review comment:
```suggestion
Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create Volcano PodGroup and set driver/executor pod annotation to link this PodGroup.
+
+Note that, currently only supported driver/job level PodGroup in Volcano Feature Step, executor separate PodGroup is not supported yet.
Review comment:
```suggestion
Note that currently only driver/job level PodGroup is supported in Volcano Feature Step. Executor PodGroup is not supported yet.
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create Volcano PodGroup and set driver/executor pod annotation to link this PodGroup.
+
+Note that, currently only supported driver/job level PodGroup in Volcano Feature Step, executor separate PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
+<table class="table">
+<tr><th>Scheduling</th><th>Description</th><th>Configuration</th></tr>
+<tr>
+ <td>Queue scheduling</td>
+ <td>
+ Queue indicates the resource queue, which adopts FIFO. is also used as the basis for resource division.
+ help users specify which queue the job to submit.
+ </td>
+ <td>`spec.queue` field in PodGroup template</td>
+</tr>
+<tr>
+ <td>Resource reservation</td>
+ <td>
+ Resource reservation, aka `Gang` scheduling (start all or nothing), helps users reserve resources for specific jobs.
+ It's useful for ensuring resource are meet the minimum requirements of spark job and avoiding all drivers stuck
Review comment:
```suggestion
It's useful for ensuring resource are meet the minimum requirements of Spark job and avoiding all drivers stuck
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create Volcano PodGroup and set driver/executor pod annotation to link this PodGroup.
+
+Note that, currently only supported driver/job level PodGroup in Volcano Feature Step, executor separate PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
+<table class="table">
+<tr><th>Scheduling</th><th>Description</th><th>Configuration</th></tr>
+<tr>
+ <td>Queue scheduling</td>
+ <td>
+ Queue indicates the resource queue, which adopts FIFO. is also used as the basis for resource division.
+ help users specify which queue the job to submit.
Review comment:
```suggestion
Helps the user to specify to which queue the job should be submitted to.
```
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
+
Review comment:
+ To use Volcano as a scheduler the user needs to specify the following configuration options:
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1722,6 +1722,83 @@ spec:
image: will-be-overwritten
```
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+##### Prerequisites
+* Volcano supports Spark on Kubernetes since v1.5. Mini version: v1.5.1+. See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a customized scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, for example:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create Volcano PodGroup and set driver/executor pod annotation to link this PodGroup.
+
+Note that, currently only supported driver/job level PodGroup in Volcano Feature Step, executor separate PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
+<table class="table">
+<tr><th>Scheduling</th><th>Description</th><th>Configuration</th></tr>
+<tr>
+ <td>Queue scheduling</td>
+ <td>
+ Queue indicates the resource queue, which adopts FIFO. is also used as the basis for resource division.
Review comment:
```suggestion
Queue indicates the resource queue, which adopts FIFO. It is also used as the basis for resource division.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r833855628
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step. Executor PodGroup is not supported yet.
Review comment:
Please remove `Executor PodGroup is not supported yet.`. IIRC, you replied that we don't have any plan for this yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r833885234
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step. Executor PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
Review comment:
This is a list to show some information for spark scenarioes, I personaly think it might be better to keep
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step. Executor PodGroup is not supported yet.
Review comment:
sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837033250
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
Review comment:
Please remove this empty line.
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
Review comment:
ditto.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837033696
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
Review comment:
Shall we remove `, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction)`? If we have a correct link on `PodGroup` before, I believe we don't need to link here again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837035594
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,88 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
+
+Note that currently only driver/job level PodGroup is supported in Volcano Feature Step. Executor PodGroup is not supported yet.
+
+##### Volcano PodGroup Template
+Volcano defines PodGroup spec using [CRD yaml](https://volcano.sh/en/docs/podgroup/#example)
+
+Similar to [Pod template](#pod-template), Spark users can similarly use Volcano PodGroup Template to define the PodGroup spec configurations.
+
+To do so, specify the Spark properties `spark.kubernetes.scheduler.volcano.podGroupTemplateFile` to point to files accessible to the `spark-submit` process.
+
+Below is an example of PodGroup template, see also [PodGroup Introduction](https://volcano.sh/en/docs/podgroup/#introduction):
+
+```
+apiVersion: scheduling.volcano.sh/v1beta1
+kind: PodGroup
+spec:
+ # Specify minMember to 1 to make driver
+ minMember: 1
+ # Specify minResources to support resource reservation
+ minResources:
+ cpu: "2"
+ memory: "3Gi"
+ # Specify the priority
+ priorityClassName: high-priority
+ queue: default
+```
+
+##### Features
Review comment:
Sorry, but I disagree with you at this part, @Yikun .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #35870:
URL: https://github.com/apache/spark/pull/35870#discussion_r837032645
##########
File path: docs/running-on-kubernetes.md
##########
@@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers.
- Create additional Kubernetes custom resources for driver/executor scheduling.
- Set scheduler hints according to configuration or existing Pod info dynamically.
+#### Using Volcano as Customized Scheduler for Spark on Kubernetes
+
+**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.**
+
+##### Prerequisites
+* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1.
+* See also [Volcano installation](https://volcano.sh/en/docs/installation).
+
+##### Build
+To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html):
+
+```
+./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano
+```
+
+##### Usage
+Spark on Kubernetes allows using Volcano as a custom scheduler. Users can use Volcano to
+support more advanced resource scheduling: queue scheduling, resource reservation, priority scheduling, and more.
+
+To use Volcano as a custom scheduler the user needs to specify the following configuration options:
+
+```
+# Specify volcano scheduler
+--conf spark.kubernetes.scheduler.name=volcano
+# Specify driver/executor VolcanoFeatureStep
+--conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+--conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
+# Specify PodGroup template
+--conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
+```
+
+##### Volcano Feature Step
+Volcano feature steps help users to create a Volcano PodGroup and set driver/executor pod annotation to link with this PodGroup.
Review comment:
Here, `PodGroup` should be linked to Volcano homepage's PodGroup definition.
```
[PodGroup](https://volcano.sh/en/docs/podgroup/)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on pull request #35870: [SPARK-38562][K8S][DOCS] Add doc for `Volcano` scheduler
Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #35870:
URL: https://github.com/apache/spark/pull/35870#issuecomment-1082523826
@dongjoon-hyun thank you for all your help! Thanks all! @holdenk @william-wang @martin-g
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org