You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2014/07/29 23:51:39 UTC

[jira] [Updated] (MAPREDUCE-6010) HistoryServerFileSystemStateStore fails to update tokens

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Lowe updated MAPREDUCE-6010:
----------------------------------

    Attachment: MAPREDUCE-6010.patch

The previous unit tests didn't catch the update failure because the rename doesn't fail on the local filesystem but does on HDFS.

Attaching a patch that fixes update by removing the destination token file before trying to rename the temporary file to the final token file.  Crashing after the delete but before the rename is handled during recovery since it checks for a temporary update file without a corresponding token file and will use the temporary update file.

Patch also adds an HDFS unit test and a unit test to cover the crash-during-update scenario.

> HistoryServerFileSystemStateStore fails to update tokens
> --------------------------------------------------------
>
>                 Key: MAPREDUCE-6010
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6010
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver
>    Affects Versions: 2.3.0
>            Reporter: Jason Lowe
>         Attachments: MAPREDUCE-6010.patch
>
>
> When token recovery is enabled and the file system state store is being used then tokens fail to be updated due to a rename destination conflict.



--
This message was sent by Atlassian JIRA
(v6.2#6252)