You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Navina Ramesh <nr...@linkedin.com.INVALID> on 2015/03/26 22:31:40 UTC

Retention policy on Yarn application's file cache

Hi yarn-dev,
I am trying to understand the extent to which the NMs support retaining application related cache.

“yarn.nodemanager.delete.debug-delay-sec” can be used to control the number of seconds after an application finishes before the NM's DeletionService will delete the application's localized file directory and log directory.

Is there a way to control the retention period for the localized file directory independent of the log directory ? That is, if I want to retain certain application data longer than the log's retention, is there a way to do it in YARN ?

Thanks!
Navina

Re: Retention policy on Yarn application's file cache

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Log directories are treated separate from the local directories.

Retention of files in the local cache is debug-delay-sec which is an admin server-side configuration. Applications cannot control it.

+Vinod

On Mar 26, 2015, at 2:31 PM, Navina Ramesh <nr...@linkedin.com.INVALID> wrote:

> Hi yarn-dev,
> I am trying to understand the extent to which the NMs support retaining application related cache.
> 
> “yarn.nodemanager.delete.debug-delay-sec” can be used to control the number of seconds after an application finishes before the NM's DeletionService will delete the application's localized file directory and log directory.
> 
> Is there a way to control the retention period for the localized file directory independent of the log directory ? That is, if I want to retain certain application data longer than the log's retention, is there a way to do it in YARN ?
> 
> Thanks!
> Navina