You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2019/05/11 14:01:00 UTC

[jira] [Assigned] (SPARK-27677) Disk-persisted RDD blocks served by shuffle service, and ignored for Dynamic Allocation

     [ https://issues.apache.org/jira/browse/SPARK-27677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-27677:
------------------------------------

    Assignee:     (was: Apache Spark)

> Disk-persisted RDD blocks served by shuffle service, and ignored for Dynamic Allocation
> ---------------------------------------------------------------------------------------
>
>                 Key: SPARK-27677
>                 URL: https://issues.apache.org/jira/browse/SPARK-27677
>             Project: Spark
>          Issue Type: New Feature
>          Components: Block Manager, Spark Core
>    Affects Versions: 2.4.3
>            Reporter: Imran Rashid
>            Priority: Major
>
> Disk-cached RDD blocks are currently unavailable after an executor is removed.  However, when there is an external shuffle service, the data remains available on disk and could be served by the shuffle service.  This would allow dynamic allocation to reclaim executors with only disk-cached blocks more rapidly, but still keep the cached data available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org