[jira] [Updated] (REEF-1081) Improved Logging in HDFSCommandRunner

["Markus Weimer (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/REEF-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Markus Weimer updated REEF-1081:
--------------------------------
    Fix Version/s:     (was: 0.14)

> Improved Logging in HDFSCommandRunner
> -------------------------------------
>
>                 Key: REEF-1081
>                 URL: https://issues.apache.org/jira/browse/REEF-1081
>             Project: REEF
>          Issue Type: Improvement
>          Components: REEF.NET IO
>    Affects Versions: 0.14
>         Environment: C#
>            Reporter: Dhruv Mahajan
>            Assignee: Dhruv Mahajan
>
> Currently in {{HdfsCommandRunner}}, the function {{LogCommandOutput(CommandResult result)}} logs the output generated when the command is executed in {{internal CommandResult Run(string hdfsCommandLineArguments)}}. However, in the current code we just check whether result is null or not and then just terminate if result exists. However, we never log or check the exit variable. If it is zero we should declare operation to be success otherwise keep trying. Infact, result will never be null. Also after maximum retries, the thrown exception does not show result logs as part of exception which contain vital information for debugging.  The code is pasted below for reference also. 
> {code}
>   for (var attemptNumber = 0; attemptNumber < _numberOfRetries; ++attemptNumber)
>             {
>                 var processName = String.Format("HDFS_Attempt_{0}_of_{1}", attemptNumber, _numberOfRetries);
>                 var result = RunAttempt(processStartInfo, _timeOutInMilliSeconds, processName);
>                 if (null != result)
>                 {
>                     LogCommandOutput(result);
>                     return result;
>                 }
>             }
>             // If we reached here, we ran out of retries.
>             throw new Exception(
>                 string.Format("HDFS Cmd {0} {1} could not be executed in the specified timeout & retry settings",
>                     _hdfsCommandPath, hdfsCommandLineArguments));
> {code}
> The main aim of JIRA is to introduce more elaborate logging and checking in the {{Run}} function. Specifically, we would like to check the exit message and then write appropriate messages.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)