You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Holden Karau (Jira)" <ji...@apache.org> on 2020/08/31 02:58:00 UTC

[jira] [Created] (SPARK-32745) Add an option to reject block migrations when under disk pressure

Holden Karau created SPARK-32745:
------------------------------------

             Summary: Add an option to reject block migrations when under disk pressure
                 Key: SPARK-32745
                 URL: https://issues.apache.org/jira/browse/SPARK-32745
             Project: Spark
          Issue Type: Sub-task
          Components: Spark Core
    Affects Versions: 3.1.0
            Reporter: Holden Karau


If we have a very low minimum number of nodes, we may fill up the disks of executors by migrating shuffle files to them. We should add a configuration option to only accept remote shuffle blocks when there is sufficient free disk space.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org