[jira] [Created] (HDFS-6147) New blocks scanning will be delayed due to issue in BlockPoolSliceScanner#updateBytesToScan(..)

["Vinayakumar B (JIRA)" <ji...@apache.org>]

Vinayakumar B created HDFS-6147:
-----------------------------------

             Summary: New blocks scanning will be delayed due to issue in BlockPoolSliceScanner#updateBytesToScan(..)
                 Key: HDFS-6147
                 URL: https://issues.apache.org/jira/browse/HDFS-6147
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: datanode
    Affects Versions: 2.4.0
            Reporter: Vinayakumar B
            Assignee: Vinayakumar B


New blocks scanning will be delayed if old blocks deleted after datanode restart.

Steps:
1. Write some blocks and wait till all scans over
2. Restart the datanode
3. Delete some of the blocks
4. Write new blocks which are less in size compared to deleted blocks.

Problem:
{{BlockPoolSliceScanner#updateBytesToScan(..)}} updates {{bytesLeft}} based on following comparison
{code}   if (lastScanTime < currentPeriodStart) {
      bytesLeft += len;
    }{code}

But in {{BlockPoolSliceScanner#assignInitialVerificationTimes()}} {{bytesLeft}} decremented using below comparison
{code}if (now - entry.verificationTime < scanPeriod) {{code}

Hence when the old blocks are deleted {{bytesLeft}} going negative.
new blocks will not be scanned until it becomes positive again.

So in both places verificationtime should be compared against scanperiod.



--
This message was sent by Atlassian JIRA
(v6.2#6252)