[jira] [Updated] (HDFS-7990) IBR delete ack should not be delayed

["Daryn Sharp (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daryn Sharp updated HDFS-7990:
------------------------------
    Attachment: HDFS-7990.patch

IBRs are always sent every heartbeat, even if it only contains delete acks.  Note again that the existing behavior is: receiving = next heartbeat, received = immediate, delete = up to 5 mins later.  Receiving/delete now have equivalent behavior.

Balancer work in HDFS-7967 also discovered that SimulatedFSDataset does not send delete acks - unlike the real dataset - so I fixed that here.

> IBR delete ack should not be delayed
> ------------------------------------
>
>                 Key: HDFS-7990
>                 URL: https://issues.apache.org/jira/browse/HDFS-7990
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.0.0-alpha
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: HDFS-7990.patch
>
>
> HDFS-395 added the incremental BR feature.  A concern was avoiding a race condition with ack-ing block invalidates followed by the directory scanner re-adding the block before the async delete service removes the block, possibly resulting in a full BR that includes the previously delete ack-ed block.
> The solution was to batch & delay block deletion acks via a hardcoded interval of 100 * heartbeat interval (default: 5min).  The delay isn't required:
> #  The {{FSDatasetSpi}} tracks blocks pending deletions precisely so the scanner won't re-add the block to the volume map
> # Block receiving, received, and deleted go into the same pending report.  A block received event will trigger an immediate IBR which includes the deletion acks.  Ie. The delay is meaningless for all but a quiescent cluster
> # Failing to promptly report deleted blocks on a quiescent cluster prevents the NN from updating the block maps to remove the locations



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)