[jira] [Commented] (HDFS-6618) Edit log corruption may still happen even after HDFS-6527

["Jing Zhao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049122#comment-14049122 ] 

Jing Zhao commented on HDFS-6618:
---------------------------------

If the corruption is caused by the same issue, can we move the following code into the writelock of FSNamesystem in deleteInternal?
{code}
    dir.writeLock();
    try {
      dir.removeFromInodeMap(removedINodes);
    } finally {
      dir.writeUnlock();
    }
{code}

> Edit log corruption may still happen even after HDFS-6527
> ---------------------------------------------------------
>
>                 Key: HDFS-6618
>                 URL: https://issues.apache.org/jira/browse/HDFS-6618
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.5.0
>            Reporter: Kihwal Lee
>            Priority: Blocker
>         Attachments: HDFS-6618.patch
>
>
> After HDFS-6527, we have not seen the edit log corruption for weeks on multiple clusters until yesterday. Previously, we would see it within 30 minutes on a cluster.
> But the same condition was reproduced even with HDFS-6527.  The only explanation is that the RPC handler thread serving {{addBlock()}} was accessing stale parent value.  Although nulling out parent is done inside the {{FSNamesystem}} and {{FSDirectory}} write lock, there is no memory barrier because there is no "synchronized" block involved in the process.
> I suggest making parent volatile.



--
This message was sent by Atlassian JIRA
(v6.2#6252)