You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Imran Rashid (JIRA)" <ji...@apache.org> on 2019/05/10 21:41:01 UTC

[jira] [Created] (SPARK-27677) Disk-persisted RDD blocks served by shuffle service, and ignored for Dynamic Allocation

Imran Rashid created SPARK-27677:
------------------------------------

             Summary: Disk-persisted RDD blocks served by shuffle service, and ignored for Dynamic Allocation
                 Key: SPARK-27677
                 URL: https://issues.apache.org/jira/browse/SPARK-27677
             Project: Spark
          Issue Type: New Feature
          Components: Block Manager, Spark Core
    Affects Versions: 2.4.3
            Reporter: Imran Rashid


Disk-cached RDD blocks are currently unavailable after an executor is removed.  However, when there is an external shuffle service, the data remains available on disk and could be served by the shuffle service.  This would allow dynamic allocation to reclaim executors with only disk-cached blocks more rapidly, but still keep the cached data available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org