[jira] [Updated] (SPARK-56158) [CORE] Add ActiveProcessorCount JVM option support for local mode

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-56158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-56158:
-----------------------------------
    Labels: pull-request-available  (was: )

> [CORE] Add ActiveProcessorCount JVM option support for local mode
> -----------------------------------------------------------------
>
>                 Key: SPARK-56158
>                 URL: https://issues.apache.org/jira/browse/SPARK-56158
>             Project: Spark
>          Issue Type: New Feature
>          Components: Spark Core
>    Affects Versions: 4.2.0
>            Reporter: John Zhuge
>            Priority: Major
>              Labels: pull-request-available
>
> SPARK-53209 added spark.driver.limitActiveProcessorCount.enabled to inject -XX:ActiveProcessorCount into YARN JVM launch commands.
> In local mode, Spark runs entirely in a single in-process JVM, so -XX:ActiveProcessorCount cannot be injected at runtime. This ticket investigates the right mechanism (e.g., auto-prepending to driver JVM options at submit time via SparkSubmit) so that local mode users on machines with many cores can benefit from the same CPU limiting behavior when each app only uses a few cores.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org