[jira] [Comment Edited] (HDFS-6618) Edit log corruption may still happen even after HDFS-6527

["Kihwal Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049195#comment-14049195 ] 

Kihwal Lee edited comment on HDFS-6618 at 7/1/14 7:06 PM:
----------------------------------------------------------

bq. can we move the following code into the writelock of FSNamesystem in deleteInternal?

That can degrade performance when deleting a large tree.  I also got worried about similar kind of race during rename and others.


was (Author: kihwal):
bq. can we move the following code into the writelock of FSNamesystem in deleteInternal?

That can degrade performance when deleting a large tree.

> Edit log corruption may still happen even after HDFS-6527
> ---------------------------------------------------------
>
>                 Key: HDFS-6618
>                 URL: https://issues.apache.org/jira/browse/HDFS-6618
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.5.0
>            Reporter: Kihwal Lee
>            Priority: Blocker
>         Attachments: HDFS-6618.patch
>
>
> After HDFS-6527, we have not seen the edit log corruption for weeks on multiple clusters until yesterday. Previously, we would see it within 30 minutes on a cluster.
> But the same condition was reproduced even with HDFS-6527.  The only explanation is that the RPC handler thread serving {{addBlock()}} was accessing stale parent value.  Although nulling out parent is done inside the {{FSNamesystem}} and {{FSDirectory}} write lock, there is no memory barrier because there is no "synchronized" block involved in the process.
> I suggest making parent volatile.



--
This message was sent by Atlassian JIRA
(v6.2#6252)