You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Weichen Xu (Jira)" <ji...@apache.org> on 2022/11/18 03:21:00 UTC

[jira] [Assigned] (SPARK-41188) Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark executor JVM processes

     [ https://issues.apache.org/jira/browse/SPARK-41188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Weichen Xu reassigned SPARK-41188:
----------------------------------

    Assignee: Weichen Xu

> Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark executor JVM processes
> -------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-41188
>                 URL: https://issues.apache.org/jira/browse/SPARK-41188
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML
>    Affects Versions: 3.3.1
>            Reporter: Weichen Xu
>            Assignee: Weichen Xu
>            Priority: Major
>             Fix For: 3.4.0
>
>
> Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark executor JVM processes, this is for limiting the thread number for OpenBLAS routine to the number of cores assigned to this executor because some spark ML algorithms calls OpenBlAS via netlib-java



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org