[jira] Updated: (HADOOP-5083) Optionally a separate daemon should serve JobHistory

["Amar Kamat (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amar Kamat updated HADOOP-5083:
-------------------------------

    Attachment: HADOOP-5083-v1.2.patch

Attaching a patch the implements what I have mentioned [here|https://issues.apache.org/jira/browse/HADOOP-5083?focusedCommentId=12665706#action_12665706]. Also by default if the _job-history-server_ configurations is not passed then jobhistory is served via the jobtracker webui. 

bq. By default we might continue to run this service in the same JVM as the jobtracker, so we don't force the maintenance of another daemon on every installation. Only folks who have huge clusters need configure things so that this is run as a separate process, potentially on a separate host. 
This is what the attached patch does. Review?

bq.So JobTracker.main() can check the configuration to see if it should, in addition to starting the jobtracker, start the job history service. In both cases, it should use independent ports from the jobtracker.
This is easy to do with the current patch. {{JobHistoryServer.java}} starts a webserver that serves the jobhistory jsps (i.e make _jobhistory_ as the main context of the webserver) and can be invoked within the same jvm. But its simpler to add the _jsp_ context to the current webserver which is done in this patch. Comments?

bq. In general, it should be simple to run all of our daemons in a single JVM, or to mix-and-match them. This should require at most a custom main() routine per JVM. We use this kind of configuration for unit testing already.
This will require more refactoring and I feel should be done in a separate (unification) jira, thoughts?


> Optionally a separate daemon should serve JobHistory
> ----------------------------------------------------
>
>                 Key: HADOOP-5083
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5083
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Arun C Murthy
>         Attachments: HADOOP-5083-v1.2.patch
>
>
> Currently the JobTracker serves the JobHistory to end-users off files local-disk/hdfs. While running very large clusters with a large user-base might result in lots of traffic for job-history which needlessly taxes the JobTracker. The proposal is to have an optional daemon which handles serving of job-history requests.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.