[jira] [Updated] (SPARK-40350) Use spark config to configure the parameters of volcano podgroup

["Sun BiaoBiao (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-40350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sun BiaoBiao updated SPARK-40350:
---------------------------------
    Summary: Use spark config to configure the parameters of volcano podgroup  (was: Improve the configuration of volcano scheduler)

> Use spark config to configure the parameters of volcano podgroup
> ----------------------------------------------------------------
>
>                 Key: SPARK-40350
>                 URL: https://issues.apache.org/jira/browse/SPARK-40350
>             Project: Spark
>          Issue Type: Improvement
>          Components: Kubernetes
>    Affects Versions: 3.3.0
>            Reporter: Sun BiaoBiao
>            Priority: Major
>
> Now we use volcano as our scheduler， we need  specify  the following configuration options:
>  
>  
> {code:java}
> spark.kubernetes.scheduler.name=volcano
> spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml
> spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep
> spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep {code}
>  
>  
> we should use configMap to mount the /path/to/podgroup-template.yaml file
>  
> If we use spark config to specify the parameters of the podgroup, it will be much more convenient,we don't need configmap to mount static files
>  
> In our scenario, we need to dynamically specify the volcano queue, but it is not convenient to create a static podgroup configuration file to mount 
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org