[jira] [Commented] (MAPREDUCE-4059) The history server should have a separate pluggable storage/query interface

["Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244218#comment-13244218 ] 

Robert Joseph Evans commented on MAPREDUCE-4059:
------------------------------------------------

Yes there is some overlap between this and MAPREDUCE-3971.  MAPREDUCE-3971 is to update the web service to support this functionality and anything else that is necessary.  This JIRA is to provide an API layer that will abstract away the caching and query of the jobs from how that data is used, whether it be a web service or a web GUI or an RPC call.  The idea is that eventually I can experiment with different back-ends that will be better at storing and querying the data than the simple LRU cache that we have now.  

I did not add in the offset to the web service because there is still work that needs to be done to update documentation, verify that it works correctly, add in tests, etc.  If you would prefer I can remove the offset parameter and just leave the count parameter, and then add it back in as part of MAPREDUCE-3971.
                
> The history server should have a separate pluggable storage/query interface
> ---------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4059
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4059
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv2
>    Affects Versions: 0.24.0, 0.23.3
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>         Attachments: MR-4059.txt, MR-4059.txt
>
>
> The history server currently caches all parsed jobs in RAM.  These jobs can be very large because of counters.  It would be nice to have a pluggable interface for the cacheing and querying of the cached data so that we can play around with different implementations.  Also just for cleanness of the code it would be nice to split the very large JobHistoryServer.java into a few smaller ones that are more understandable and readable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira