You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2012/10/08 22:20:02 UTC

[jira] [Created] (MAPREDUCE-4714) Historyserver retrieves job from jobID more often than necessary

Jason Lowe created MAPREDUCE-4714:
-------------------------------------

             Summary: Historyserver retrieves job from jobID more often than necessary
                 Key: MAPREDUCE-4714
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4714
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobhistoryserver, mrv2
    Affects Versions: 2.0.0-alpha, 0.23.3
            Reporter: Jason Lowe
            Priority: Minor


When serving up a web page for job history, AppController.requireJob() is converting from a jobID to a job twice when it only needs to do it once.  The resulting job is stored into the app object, which we could, in turn, use in historyserver web page render() methods to simply retrieve the job rather than performing yet another jobID->job lookup.  That would cut the job lookups down from 3 to 1.

jobID->job lookups aren't necessarily cheap.  If the job isn't in the historyserver joblist cache then the historyserver needs to scan directories looking for it, adding an unnecessary extra load onto the namenode.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira