[jira] [Created] (AMBARI-20670) Node manager start extremely slow when YARN NM local dirs are very large

["Dmytro Grinenko (JIRA)" <ji...@apache.org>]

Dmytro Grinenko created AMBARI-20670:
----------------------------------------

             Summary: Node manager start extremely slow when YARN NM local dirs are very large
                 Key: AMBARI-20670
                 URL: https://issues.apache.org/jira/browse/AMBARI-20670
             Project: Ambari
          Issue Type: Bug
          Components: ambari-server
    Affects Versions: 2.5.1
            Reporter: Dmytro Grinenko
            Assignee: Dmytro Grinenko
            Priority: Critical
             Fix For: trunk, 2.5.1


On the cluster with the YARN NM, where local dirs are 100 GB+ with lot of small files - NM starts slow with timeouts

Reason could be in this specific call in  yarn.py

{code}
def create_local_dir(dir_name):
  import params
  Directory(dir_name,
            create_parents = True,
            cd_access="a",
            mode=0755,
            owner=params.yarn_user,
            group=params.user_group,
            ignore_failures=True,
            recursive_mode_flags = {'f': 'a+rw', 'd': 'a+rwx'},
  )
{code}

was taking ~15 minutes per mount.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)