You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by "Bibin A Chundatt (JIRA)" <ji...@apache.org> on 2017/06/10 10:26:21 UTC
[jira] [Created] (YARN-6708) Nodemanager container after ext3
folder limit
Bibin A Chundatt created YARN-6708:
--------------------------------------
Summary: Nodemanager container after ext3 folder limit
Key: YARN-6708
URL: https://issues.apache.org/jira/browse/YARN-6708
Project: Hadoop YARN
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Critical
Configure umask as *027* for nodemanager service user
and {{yarn.nodemanager.local-cache.max-files-per-directory}} as {{40}}. After 4 *private* dir localization next directory will be *0/14*
Local Directory cache manager
{code}
vm2:/opt/hadoop/release/data/nmlocal/usercache/mapred/filecache # l
total 28
drwx--x--- 7 mapred hadoop 4096 Jun 10 14:35 ./
drwxr-s--- 4 mapred hadoop 4096 Jun 10 12:07 ../
drwxr-x--- 3 mapred users 4096 Jun 10 14:36 0/
drwxr-xr-x 3 mapred users 4096 Jun 10 12:15 10/
drwxr-xr-x 3 mapred users 4096 Jun 10 12:22 11/
drwxr-xr-x 3 mapred users 4096 Jun 10 12:27 12/
drwxr-xr-x 3 mapred users 4096 Jun 10 12:31 13/
{code}
*drwxr-x---* 3 mapred users 4096 Jun 10 14:36 0/ is only *750*
Nodemanager user will not be able check for localization path exists or not.
{{LocalResourcesTrackerImpl}}
{code}
case REQUEST:
if (rsrc != null && (!isResourcePresent(rsrc))) {
LOG.info("Resource " + rsrc.getLocalPath()
+ " is missing, localizing it again");
removeResource(req);
rsrc = null;
}
if (null == rsrc) {
rsrc = new LocalizedResource(req, dispatcher);
localrsrc.put(req, rsrc);
}
break;
{code}
*isResourcePresent* will always return false and same resource will be localized to {{0}} to next unique number
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org