[jira] [Updated] (SPARK-45250) Support stage level task resource profile for yarn cluster when dynamic allocation disabled

["Thomas Graves (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-45250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thomas Graves updated SPARK-45250:
----------------------------------
    Fix Version/s: 3.5.1

> Support stage level task resource profile for yarn cluster when dynamic allocation disabled
> -------------------------------------------------------------------------------------------
>
>                 Key: SPARK-45250
>                 URL: https://issues.apache.org/jira/browse/SPARK-45250
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 3.4.1
>            Reporter: Bobby Wang
>            Assignee: Bobby Wang
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0, 3.5.1
>
>
> [https://github.com/apache/spark/pull/37268] has introduced a new feature that supports stage-level schedule task resource profile for standalone cluster when dynamic allocation is disabled. It's really cool feature, especially for ML/DL cases, more details can be found in that PR.
>  
> The problem here is that the feature is only available for standalone cluster for now, but most users would also expect it can be used for other spark clusters like yarn and k8s.
>  
> So I file this issue to track this task.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org