[jira] [Commented] (MAPREDUCE-5547) Job history should not be flushed to JHS until AM gets unregistered

["Jian He (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13784643#comment-13784643 ] 

Jian He commented on MAPREDUCE-5547:
------------------------------------

The problem being that the history sever can copy the history data to the done_intermediate directory and then unregister fails.  Then the AM is relaunched, but user already see the finished status of the job in history sever

bq.  If the history server has already moved it from done_intermediate to done then the history server could either re-update the history with the new copy in done_intermediate or simply delete the redundant copy in done_intermediate.
we can do this, but user still see the finished status of the job after the 1st AM unregisters, but just that the status will be updated until the next AM finishes. 

bq.  If we unregister before copying the history data to the done_intermediate directory then the client could try to query the history server before the AM has had a chance to copy the jhist file.
Yes, the job in history sever may be missing for some time, but it can eventually show up after the history data is copied to done_intermediate.


> Job history should not be flushed to JHS until AM gets unregistered
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5547
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5547
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>




--
This message was sent by Atlassian JIRA
(v6.1#6144)