You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-dev@hadoop.apache.org by "Kihwal Lee (JIRA)" <ji...@apache.org> on 2015/08/06 19:53:05 UTC

[jira] [Created] (HDFS-8864) Padding is needed in remaining space check

Kihwal Lee created HDFS-8864:
--------------------------------

             Summary: Padding is needed in remaining space check
                 Key: HDFS-8864
                 URL: https://issues.apache.org/jira/browse/HDFS-8864
             Project: Hadoop HDFS
          Issue Type: Bug
            Reporter: Kihwal Lee
            Priority: Critical


The RBW reservation feature removed the padding of 5 block worth of space in the remaining space check.  But when disks are close to being full, a small accounting errors can make block creations to fail on the datanode. One obvious source of such error is the slow control loop formed by heartbeats.

We have been testing a patch which adjusts the padding based on the heartbeat interval and it seems to be working fine.  At the default 3 second heartbeat interval, the padding becomes 6 blocks, which is close to the previous value of 5.  If a longer heartbeat interval is used, this will increase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)