[jira] [Commented] (AMBARI-20670) Node manager start extremely slow when YARN NM local dirs are very large

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMBARI-20670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958023#comment-15958023 ] 

Hadoop QA commented on AMBARI-20670:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12862101/AMBARI-20670-2.5.patch
  against trunk revision .

    {color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/11314//console

This message is automatically generated.

> Node manager start extremely slow when YARN NM local dirs are very large
> ------------------------------------------------------------------------
>
>                 Key: AMBARI-20670
>                 URL: https://issues.apache.org/jira/browse/AMBARI-20670
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.5.1
>            Reporter: Dmytro Grinenko
>            Assignee: Dmytro Grinenko
>            Priority: Critical
>             Fix For: trunk, 2.5.1
>
>         Attachments: AMBARI-20670-2.5.patch, AMBARI-20670.patch
>
>
> On the cluster with the YARN NM, where local dirs are 100 GB+ with lot of small files - NM starts slow with timeouts
> Reason could be in this specific call in  yarn.py
> {code}
> def create_local_dir(dir_name):
>   import params
>   Directory(dir_name,
>             create_parents = True,
>             cd_access="a",
>             mode=0755,
>             owner=params.yarn_user,
>             group=params.user_group,
>             ignore_failures=True,
>             recursive_mode_flags = {'f': 'a+rw', 'd': 'a+rwx'},
>   )
> {code}
> was taking ~15 minutes per mount.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)