[GitHub] [hadoop-ozone] ChenSammi edited a comment on pull request #1295: HDDS-4037. Incorrect container numberOfKeys and usedBytes in SCM after key deletion

[GitBox <gi...@apache.org>]

ChenSammi edited a comment on pull request #1295:
URL: https://github.com/apache/hadoop-ozone/pull/1295#issuecomment-671191882


   > 
   > 
   > @ChenSammi Thanks for fixing this. Can you add a short description on what the root cause is, and how this patch fixes it? That will help make the review easier.
   > 
   
   Sure. Currently there are two issues.     
    1. Container keyCount in datanode side is incorrent.  KeyCount is increased by 1 for every putblock command.  This is true for FilePerChunk container layout while it's incorrect for FilePerBlock layout. With FilePerBlock layout, you will find keyCount value is higher than accutal block number.
   
      2. Container keyCount and bytesUsed hold in SCM side is based on the container report data from each datanode.   AbstractContainerReportHandler#updateContainerStats,  currently SCM only updates the in-memory Container data if the reported container data is bigger.  This's true for OPEN containers. But doesn't stand for CLOSED containers. 
   
    In our cluster, there are a lot of containers, all blocks are deleted. But I can still get the keyCount and BytesUsed data through CLI "sh container info" command.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org