[jira] [Commented] (YARN-1751) Improve MiniYarnCluster and LogCLIHelpers for log aggregation testing

["Jason Lowe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993565#comment-13993565 ] 

Jason Lowe commented on YARN-1751:
----------------------------------

Despite them both being small changes, I think these should be separate JIRA since they're otherwise unrelated changes for different problems and can stand on their own.  We can morph this JIRA into one of them and file a new one to cover the other.

For the LogCLIHelpers change, I think it should be calling FileContext.getFileContext(remoteAppLogDir.toUri(), conf) in case the remoteAppLogDir is not on the default filesystem.  There's also the question of whether it should guard against a null conf, since oddly despite LogCLIHelpers being Configurable it isn't using the config until after this change.  I think I'm leaning towards leaving it null and letting the NPE occur so callers will fix it.  We've had lots of performance problems and other weirdness in the past when code forgot to pass down a custom config and things sorta worked with the default one.

+1 for the MiniYarnCluster change.



> Improve MiniYarnCluster and LogCLIHelpers for log aggregation testing
> ---------------------------------------------------------------------
>
>                 Key: YARN-1751
>                 URL: https://issues.apache.org/jira/browse/YARN-1751
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>            Reporter: Ming Ma
>            Assignee: Ming Ma
>         Attachments: YARN-1751-trunk.patch
>
>
> MiniYarnCluster specifies individual remote log aggregation root dir for each NM. Test code that uses MiniYarnCluster won't be able to get the value of log aggregation root dir. The following code isn't necessary in MiniYarnCluster.
>       File remoteLogDir =
>           new File(testWorkDir, MiniYARNCluster.this.getName()
>               + "-remoteLogDir-nm-" + index);
>       remoteLogDir.mkdir();
>       config.set(YarnConfiguration.NM_REMOTE_APP_LOG_DIR,
>           remoteLogDir.getAbsolutePath());
> In LogCLIHelpers.java, dumpAllContainersLogs should pass its conf object to FileContext.getFileContext() call.



--
This message was sent by Atlassian JIRA
(v6.2#6252)