[jira] Commented: (HADOOP-239) job tracker WI drops jobs after 24 hours

["Doug Cutting (JIRA)" <ji...@apache.org>]

    [ http://issues.apache.org/jira/browse/HADOOP-239?page=comments#action_12437898 ] 
            
Doug Cutting commented on HADOOP-239:
-------------------------------------

Perhaps we should change the string constants in JobInfo.java to be an enum (or two), to get the compiler to do a bit more work for us?  Or, we could use a record i/o record and XML output.  This way serializers and deserializers would be written for us.  That's perhaps overkill, but it's worth thinking about.

In any case, we need to make sure it's easy for these files to evolve.  Adding and removing fields should be trivial, and processing old files that have obsolete or missing fields should also be natural.  Right now, each new field requires a number of coordinated changes.  Is there any way we can reduce these?

> job tracker WI drops jobs after 24 hours
> ----------------------------------------
>
>                 Key: HADOOP-239
>                 URL: http://issues.apache.org/jira/browse/HADOOP-239
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Yoram Arnon
>         Assigned To: Sanjay Dahiya
>            Priority: Minor
>         Attachments: Hadoop-239.patch, Hadoop-239_1.patch
>
>
> The jobtracker's WI, keeps track of jobs executed in the past 24 hours.
> if the cluster was idle for a day (say Sunday) it drops all its history.
> Monday morning, the page is empty.
> Better would be to store a fixed number of jobs (say 10 each of succeeded and failed jobs).
> Also, if the job tracker is restarted, it loses all its history.
> The history should be persistent, withstanding restarts and upgrades.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira