[jira] [Commented] (HDFS-10911) Change edit log OP_UPDATE_BLOCKS to store delta blocks only.

["Kihwal Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15530215#comment-15530215 ] 

Kihwal Lee commented on HDFS-10911:
-----------------------------------

IIRC, namenode didn't use to log all blocks all the time, but it became like that with the HA feature. You might want to revisit the reason and make sure things don't break with the proposed change.

> Change edit log OP_UPDATE_BLOCKS to store delta blocks only.
> ------------------------------------------------------------
>
>                 Key: HDFS-10911
>                 URL: https://issues.apache.org/jira/browse/HDFS-10911
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 2.7.3, 3.0.0-alpha1
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>
> Every time a HDFS client {{close}}  or {{hflush}} an open file, NameNode enumerates all the blocks  and stores then into edit log (OP_UPDATE_BLOCKS). 
> It would cause problem when the client is appending a large file frequently (i.e., WAL). 
> Because HDFS is append only, we can only store the blocks that have been changed (delta blocks) in edit log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org