[jira] [Commented] (MAPREDUCE-4643) Make job-history cleanup-period configurable

["Karthik Kambatla (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450412#comment-13450412 ] 

Karthik Kambatla commented on MAPREDUCE-4643:
---------------------------------------------

The two aspects of making the job-history cleanup period configurable seem to be: 
# unit of time to configure it on (minutes vs days)
# directory naming to reflect the creation time

For the directory naming, we should be able to leave it as yyyy/mm/dd format. If the cleanup period is in hours/minutes, we can read the file modification time and use that to do the comparison. Trunk currently uses the modification time even for higher values of cleanup period (days) and has a TODO note that we should use the directory structure instead to reduce the load on HDFS.

With regards to the unit of time for configuration, minutes seems to be the better option.

Thoughts?
                
> Make job-history cleanup-period configurable
> --------------------------------------------
>
>                 Key: MAPREDUCE-4643
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4643
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver
>    Affects Versions: 1.0.3, 2.0.0-alpha
>            Reporter: Karthik Kambatla
>
> Job history cleanup should be made configurable. Currently, it is set to 1 month by default. The DEBUG_MODE (to be removed, see MAPREDUCE-4629) sets it to 20 minutes, but it should be configurable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira