You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Navina Ramesh <nr...@linkedin.com.INVALID> on 2015/03/26 22:31:40 UTC
Retention policy on Yarn application's file cache
Hi yarn-dev,
I am trying to understand the extent to which the NMs support retaining application related cache.
“yarn.nodemanager.delete.debug-delay-sec” can be used to control the number of seconds after an application finishes before the NM's DeletionService will delete the application's localized file directory and log directory.
Is there a way to control the retention period for the localized file directory independent of the log directory ? That is, if I want to retain certain application data longer than the log's retention, is there a way to do it in YARN ?
Thanks!
Navina
Re: Retention policy on Yarn application's file cache
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Log directories are treated separate from the local directories.
Retention of files in the local cache is debug-delay-sec which is an admin server-side configuration. Applications cannot control it.
+Vinod
On Mar 26, 2015, at 2:31 PM, Navina Ramesh <nr...@linkedin.com.INVALID> wrote:
> Hi yarn-dev,
> I am trying to understand the extent to which the NMs support retaining application related cache.
>
> “yarn.nodemanager.delete.debug-delay-sec” can be used to control the number of seconds after an application finishes before the NM's DeletionService will delete the application's localized file directory and log directory.
>
> Is there a way to control the retention period for the localized file directory independent of the log directory ? That is, if I want to retain certain application data longer than the log's retention, is there a way to do it in YARN ?
>
> Thanks!
> Navina