You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2018/02/16 13:14:10 UTC
[jira] [Commented] (AMBARI-20670) Node manager start extremely slow
when YARN NM local dirs are very large
[ https://issues.apache.org/jira/browse/AMBARI-20670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16367076#comment-16367076 ]
Hudson commented on AMBARI-20670:
---------------------------------
FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8744 (See [https://builds.apache.org/job/Ambari-trunk-Commit/8744/])
AMBARI-20670 Node manager start extremely slow when YARN NM local dirs (aonishuk: [https://gitbox.apache.org/repos/asf?p=ambari.git&a=commit&h=ecc3fdf68bd436d562a622255597229d2f7b4134])
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
* (edit) ambari-common/src/main/python/resource_management/core/sudo.py
* (add) ambari-agent/src/test/python/resource_management/TestUtils.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/yarn.py
* (edit) ambari-common/src/main/python/resource_management/core/utils.py
AMBARI-20670. Node manager start extremely slow when YARN NM local dirs (aonishuk: [https://gitbox.apache.org/repos/asf?p=ambari.git&a=commit&h=22e1e7ebc1b6cc5af75b9e552d39c79936d89316])
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py
> Node manager start extremely slow when YARN NM local dirs are very large
> ------------------------------------------------------------------------
>
> Key: AMBARI-20670
> URL: https://issues.apache.org/jira/browse/AMBARI-20670
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.5.1
> Reporter: Dmytro Grinenko
> Assignee: Dmytro Grinenko
> Priority: Critical
> Fix For: trunk, 2.5.1
>
> Attachments: AMBARI-20670-2.5.patch, AMBARI-20670-ut-fix.patch, AMBARI-20670.patch
>
>
> On the cluster with the YARN NM, where local dirs are 100 GB+ with lot of small files - NM starts slow with timeouts
> Reason could be in this specific call in yarn.py
> {code}
> def create_local_dir(dir_name):
> import params
> Directory(dir_name,
> create_parents = True,
> cd_access="a",
> mode=0755,
> owner=params.yarn_user,
> group=params.user_group,
> ignore_failures=True,
> recursive_mode_flags = {'f': 'a+rw', 'd': 'a+rwx'},
> )
> {code}
> was taking ~15 minutes per mount.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)