[GitHub] [hadoop] jojochuang commented on a diff in pull request #4322: HDFS-16574. Reduces the time it takes once to hold FSNamesystem write lock to remove blocks associated with dead datanodes

[GitBox <gi...@apache.org>]

jojochuang commented on code in PR #4322:
URL: https://github.com/apache/hadoop/pull/4322#discussion_r909043603


##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java:
##########
@@ -1763,7 +1773,21 @@ public BlocksWithLocations getBlocksWithLocations(final DatanodeID datanode,
   void removeBlocksAssociatedTo(final DatanodeDescriptor node) {
     providedStorageMap.removeDatanode(node);
     final Iterator<BlockInfo> it = node.getBlockIterator();
+    int numBlocksRemovedPerLock = 0;
     while(it.hasNext()) {
+      if (numBlocksRemovedPerLock >= numBlocksPerRemove) {
+        namesystem.writeUnlock();

Review Comment:
   How is this code tested? It's usually not wise to release the namenode lock arbitrarily.
   Looking at the code, the iterator is not thread-safe and there is no guarantee the iterator will be able to get to the next one after the lock is re-acquired.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org