You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Lianhui Wang (JIRA)" <ji...@apache.org> on 2014/12/29 13:13:13 UTC
[jira] [Updated] (SPARK-4994) Cleanup removed executors'
ShuffleInfo in yarn shuffle service
[ https://issues.apache.org/jira/browse/SPARK-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lianhui Wang updated SPARK-4994:
--------------------------------
Description: when the application is completed, yarn's nodemanager can remove application's local-dirs.but all executors' metadata of completed application havenot be removed. now it let yarn ShuffleService to have much more memory to store Executors' ShuffleInfo. so these metadata need to be removed. (was: when the application is completed, yarn's nodemanager can remove application's local-dirs.but all executors' metadata of completed application havenot be removed. so these metadata need to be removed.)
> Cleanup removed executors' ShuffleInfo in yarn shuffle service
> --------------------------------------------------------------
>
> Key: SPARK-4994
> URL: https://issues.apache.org/jira/browse/SPARK-4994
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.2.0
> Reporter: Lianhui Wang
>
> when the application is completed, yarn's nodemanager can remove application's local-dirs.but all executors' metadata of completed application havenot be removed. now it let yarn ShuffleService to have much more memory to store Executors' ShuffleInfo. so these metadata need to be removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org