You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by John Lilley <jo...@redpoint.net> on 2014/01/04 18:02:02 UTC
LocalResource size/time limits
Are there any limits on the total size of LocalResources that a YARN app requests? Do the PUBLIC ones age out of cache over time? Are there settable controls?
Thanks
John
RE: LocalResource size/time limits
Posted by John Lilley <jo...@redpoint.net>.
Fantastic. Just what I needed to know.
john
From: Hardik Pandya [mailto:smarty.juice@gmail.com]
Sent: Saturday, January 04, 2014 11:15 AM
To: user@hadoop.apache.org
Subject: Re: LocalResource size/time limits
May be this would clarify some aspect of your questions
Resource Localization in YARN Deep Dive<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>
The threshold for local files is dictated by the configuration property yarn.nodemanager.localizer.cache.target-size-mb described below.
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
yarn.nodemanager.localizer.cache.target-size-mb
10240
Target size of localizer cache in MB, per local directory.
On Sat, Jan 4, 2014 at 12:02 PM, John Lilley <jo...@redpoint.net>> wrote:
Are there any limits on the total size of LocalResources that a YARN app requests? Do the PUBLIC ones age out of cache over time? Are there settable controls?
Thanks
John
RE: LocalResource size/time limits
Posted by John Lilley <jo...@redpoint.net>.
Fantastic. Just what I needed to know.
john
From: Hardik Pandya [mailto:smarty.juice@gmail.com]
Sent: Saturday, January 04, 2014 11:15 AM
To: user@hadoop.apache.org
Subject: Re: LocalResource size/time limits
May be this would clarify some aspect of your questions
Resource Localization in YARN Deep Dive<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>
The threshold for local files is dictated by the configuration property yarn.nodemanager.localizer.cache.target-size-mb described below.
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
yarn.nodemanager.localizer.cache.target-size-mb
10240
Target size of localizer cache in MB, per local directory.
On Sat, Jan 4, 2014 at 12:02 PM, John Lilley <jo...@redpoint.net>> wrote:
Are there any limits on the total size of LocalResources that a YARN app requests? Do the PUBLIC ones age out of cache over time? Are there settable controls?
Thanks
John
RE: LocalResource size/time limits
Posted by John Lilley <jo...@redpoint.net>.
Fantastic. Just what I needed to know.
john
From: Hardik Pandya [mailto:smarty.juice@gmail.com]
Sent: Saturday, January 04, 2014 11:15 AM
To: user@hadoop.apache.org
Subject: Re: LocalResource size/time limits
May be this would clarify some aspect of your questions
Resource Localization in YARN Deep Dive<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>
The threshold for local files is dictated by the configuration property yarn.nodemanager.localizer.cache.target-size-mb described below.
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
yarn.nodemanager.localizer.cache.target-size-mb
10240
Target size of localizer cache in MB, per local directory.
On Sat, Jan 4, 2014 at 12:02 PM, John Lilley <jo...@redpoint.net>> wrote:
Are there any limits on the total size of LocalResources that a YARN app requests? Do the PUBLIC ones age out of cache over time? Are there settable controls?
Thanks
John
RE: LocalResource size/time limits
Posted by John Lilley <jo...@redpoint.net>.
Fantastic. Just what I needed to know.
john
From: Hardik Pandya [mailto:smarty.juice@gmail.com]
Sent: Saturday, January 04, 2014 11:15 AM
To: user@hadoop.apache.org
Subject: Re: LocalResource size/time limits
May be this would clarify some aspect of your questions
Resource Localization in YARN Deep Dive<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>
The threshold for local files is dictated by the configuration property yarn.nodemanager.localizer.cache.target-size-mb described below.
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
yarn.nodemanager.localizer.cache.target-size-mb
10240
Target size of localizer cache in MB, per local directory.
On Sat, Jan 4, 2014 at 12:02 PM, John Lilley <jo...@redpoint.net>> wrote:
Are there any limits on the total size of LocalResources that a YARN app requests? Do the PUBLIC ones age out of cache over time? Are there settable controls?
Thanks
John
Re: LocalResource size/time limits
Posted by Hardik Pandya <sm...@gmail.com>.
May be this would clarify some aspect of your questions
Resource Localization in YARN Deep
Dive<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>
The threshold for local files is dictated by the configuration property
*yarn.nodemanager.localizer.cache.target-size-mb* described below.
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
yarn.nodemanager.localizer.cache.target-size-mb10240Target size of
localizer cache in MB, per local directory.
On Sat, Jan 4, 2014 at 12:02 PM, John Lilley <jo...@redpoint.net>wrote:
> Are there any limits on the total size of LocalResources that a YARN app
> requests? Do the PUBLIC ones age out of cache over time? Are there
> settable controls?
>
> Thanks
>
> John
>
>
>
>
>
Re: LocalResource size/time limits
Posted by Hardik Pandya <sm...@gmail.com>.
May be this would clarify some aspect of your questions
Resource Localization in YARN Deep
Dive<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>
The threshold for local files is dictated by the configuration property
*yarn.nodemanager.localizer.cache.target-size-mb* described below.
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
yarn.nodemanager.localizer.cache.target-size-mb10240Target size of
localizer cache in MB, per local directory.
On Sat, Jan 4, 2014 at 12:02 PM, John Lilley <jo...@redpoint.net>wrote:
> Are there any limits on the total size of LocalResources that a YARN app
> requests? Do the PUBLIC ones age out of cache over time? Are there
> settable controls?
>
> Thanks
>
> John
>
>
>
>
>
Re: LocalResource size/time limits
Posted by Hardik Pandya <sm...@gmail.com>.
May be this would clarify some aspect of your questions
Resource Localization in YARN Deep
Dive<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>
The threshold for local files is dictated by the configuration property
*yarn.nodemanager.localizer.cache.target-size-mb* described below.
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
yarn.nodemanager.localizer.cache.target-size-mb10240Target size of
localizer cache in MB, per local directory.
On Sat, Jan 4, 2014 at 12:02 PM, John Lilley <jo...@redpoint.net>wrote:
> Are there any limits on the total size of LocalResources that a YARN app
> requests? Do the PUBLIC ones age out of cache over time? Are there
> settable controls?
>
> Thanks
>
> John
>
>
>
>
>
Re: LocalResource size/time limits
Posted by Hardik Pandya <sm...@gmail.com>.
May be this would clarify some aspect of your questions
Resource Localization in YARN Deep
Dive<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>
The threshold for local files is dictated by the configuration property
*yarn.nodemanager.localizer.cache.target-size-mb* described below.
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
yarn.nodemanager.localizer.cache.target-size-mb10240Target size of
localizer cache in MB, per local directory.
On Sat, Jan 4, 2014 at 12:02 PM, John Lilley <jo...@redpoint.net>wrote:
> Are there any limits on the total size of LocalResources that a YARN app
> requests? Do the PUBLIC ones age out of cache over time? Are there
> settable controls?
>
> Thanks
>
> John
>
>
>
>
>