You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Robert Kanter (JIRA)" <ji...@apache.org> on 2013/09/12 22:01:52 UTC

[jira] [Updated] (MAPREDUCE-4680) Job history cleaner should only check timestamps of files in old enough directories

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Kanter updated MAPREDUCE-4680:
-------------------------------------

    Attachment: MAPREDUCE-4680.patch

The patch makes it now only gather the directories that could possibly be old enough.
                
> Job history cleaner should only check timestamps of files in old enough directories
> -----------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4680
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4680
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver
>    Affects Versions: 2.0.0-alpha
>            Reporter: Sandy Ryza
>            Assignee: Robert Kanter
>         Attachments: MAPREDUCE-4680.patch
>
>
> Job history files are stored in yyyy/mm/dd folders.  Currently, the job history cleaner checks the modification date of each file in every one of these folders to see whether it's past the maximum age.  The load on HDFS could be reduced by only checking the ages of files in directories that are old enough, as determined by their name.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira