[jira] [Updated] (MAPREDUCE-5902) JobHistoryServer (HistoryFileManager) needs more debug logs, fails to pick up jobs with % characters in the name.

["jay vyas (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

jay vyas updated MAPREDUCE-5902:
--------------------------------

    Summary: JobHistoryServer (HistoryFileManager) needs more debug logs, fails to pick up jobs with % characters in the name.  (was: JobHistoryServer (HistoryFileManager) needs more debug logs.)

> JobHistoryServer (HistoryFileManager) needs more debug logs, fails to pick up jobs with % characters in the name.
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5902
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5902
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver
>            Reporter: jay vyas
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> With the JobHistory Server , it appears that its possible sometimes to skip over certain history files.  I havent been able to determine why yet, but I've found that some long named .jhist files aren't getting collected into the done/ directory.
> After tracing some in the actual source, and turning on DEBUG level logging, it became clear that this snippet is an important workhorse (scanDirectoryForIntermediateFiles, and scanDirectoryForHistoryFiles ultimately boil down to scanDirectory()).  
> It would be extremely useful , then, to have a couple of gaurded logs at this level of the code, so that we can see, in the log folders, why files are being filtered out  , i.e. it is due to filterint or visibility.
> {noformat}
>   private static List<FileStatus> scanDirectory(Path path, FileContext fc,
>       PathFilter pathFilter) throws IOException {
>     path = fc.makeQualified(path);
>     List<FileStatus> jhStatusList = new ArrayList<FileStatus>();
>     RemoteIterator<FileStatus> fileStatusIter = fc.listStatus(path);
>     while (fileStatusIter.hasNext()) {
>       FileStatus fileStatus = fileStatusIter.next();
>       Path filePath = fileStatus.getPath();
>       if (fileStatus.isFile() && pathFilter.accept(filePath)) {
>         jhStatusList.add(fileStatus);
>       }
>     }
>     return jhStatusList;
>   }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)