You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/10/12 00:17:00 UTC

[jira] [Updated] (SPARK-43396) Add config to control max ratio of decommissioning executors

     [ https://issues.apache.org/jira/browse/SPARK-43396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-43396:
-----------------------------------
    Labels: pull-request-available  (was: )

> Add config to control max ratio of decommissioning executors
> ------------------------------------------------------------
>
>                 Key: SPARK-43396
>                 URL: https://issues.apache.org/jira/browse/SPARK-43396
>             Project: Spark
>          Issue Type: New Feature
>          Components: Spark Core
>    Affects Versions: 3.4.0
>            Reporter: Zhongwei Zhu
>            Priority: Major
>              Labels: pull-request-available
>
> Decommission too many executors at the same time with shuffle or rdd migration could severely hurt performance of shuffle fetch. Block manager decommissioner try to migrate shuffle or rdd as soon as possible, this will compete network and disk IO with shuffle fetch in the target executor. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org