[jira] [Comment Edited] (HDFS-13135) Lease not deleted when deleting INodeReference

["Kihwal Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361517#comment-16361517 ] 

Kihwal Lee edited comment on HDFS-13135 at 2/12/18 10:06 PM:
-------------------------------------------------------------

From what I remember, a change (bug fix?) intentionally made a lease to be removed from LeaseManager, but left it in the INode in the snapshot.  I argue that it is not a correct design.  Some blocks can be left in under-construction state forever without any block recovery. This can cause data loss, since re-replication won't happen for those blocks.  When namenode is restarted, all leases will be restored based on the files under-construction section and also on INodeUnderConstructionFeature. 

I am not a snapshot expert, so I can't say what is the correct fix.  I've seen conflicting requirements on under-construction blocks in a snapshot. IMO, an under-construction block ends up only in a snapshot (current view deleted while being written), a block recovery should somehow be done.


was (Author: kihwal):
From what I remember, a change (bug fix?) intentionally made a lease to be removed from LeaseManager, but left it in the INode in the snapshot.  I argue that it is not a correct design.  Some blocks can be left in under-construction state forever without any block recovery. This can cause data loss, since re-replication won't happen for those blocks.  When namenode is restarted, all leases will be restored based on the files under-construction section and also on INodeUnderConstructionFeature. 

> Lease not deleted when deleting INodeReference
> ----------------------------------------------
>
>                 Key: HDFS-13135
>                 URL: https://issues.apache.org/jira/browse/HDFS-13135
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Sean Mackrory
>            Assignee: Sean Mackrory
>            Priority: Major
>         Attachments: HDFS-13135.001.patch
>
>
> In troubleshooting an occurrence of HDFS-13115, it seemed that there was another underlying root cause that should also be addressed. There was an INodeReference that was deleted and the lease on it was not subsequently deleted because it was never added to the reclaim context.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org