[GitHub] [ozone] Xushaohong commented on a diff in pull request #3779: HDDS-7259. Fix uncounted blocksDeleted in BlockDeletingService

[GitBox <gi...@apache.org>]

Xushaohong commented on code in PR #3779:
URL: https://github.com/apache/ozone/pull/3779#discussion_r980931247


##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/statemachine/background/BlockDeletingService.java:
##########
@@ -556,17 +556,26 @@ private Pair<Integer, Long> deleteTransactions(
             LOG.warn("Missing delete block(Container = " +
                 container.getContainerData().getContainerID() + ", Block = " +
                 blkLong);
+            blocksDeleted++;

Review Comment:
   When SCM sends the DeletedBlocksTransaction to DN, Dn will increase the `numPendingDeletionBlocks `.
   When DN deletes the block, DN then decreases the `numPendingDeletionBlocks `.
   If we do not count these unrecorded as deleted, `numPendingDeletionBlocks ` of container data will stay still and obstruct the blockDeletingService(if we use the default TopN strategy). Cuz `numPendingDeletionBlocks ` is counted into the threshold but it will not reduce. If such invalid pending blocks accumulate and exceed the threshold, no more normal container would be chosen to delete blocks.
   
   
   The unrecorded block should be counted as deleted here. Since Dn now will not act as the client and actively add the unknown blocks to its DB. That's weird.
   
   The blocks that may exist but are not recorded in the RDB could be orphan blocks created by the client crush or met an error(for. example containerNotOpenException) before **putBlock**. I am about to set up a background service to clean up them.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org