[jira] [Commented] (SPARK-41188) Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark executor JVM processes

["Apache Spark (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-41188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17635632#comment-17635632 ] 

Apache Spark commented on SPARK-41188:
--------------------------------------

User 'WeichenXu123' has created a pull request for this issue:
https://github.com/apache/spark/pull/38699

> Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark executor JVM processes
> -------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-41188
>                 URL: https://issues.apache.org/jira/browse/SPARK-41188
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML, Spark Core
>    Affects Versions: 3.3.1
>            Reporter: Weichen Xu
>            Assignee: Weichen Xu
>            Priority: Major
>             Fix For: 3.4.0
>
>
> Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark executor JVM processes, this is for limiting the thread number for OpenBLAS routine to the number of cores assigned to this executor because some spark ML algorithms calls OpenBlAS via netlib-java



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org