[jira] [Created] (HDFS-15354) clearCorruptLazyPersistFiles incrementalBlock removal should be out side write lock

["Uma Maheswara Rao G (Jira)" <ji...@apache.org>]

Uma Maheswara Rao G created HDFS-15354:
------------------------------------------

             Summary: clearCorruptLazyPersistFiles incrementalBlock removal should be out side write lock
                 Key: HDFS-15354
                 URL: https://issues.apache.org/jira/browse/HDFS-15354
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: namenode
    Affects Versions: 3.2.1
            Reporter: Uma Maheswara Rao G
            Assignee: Uma Maheswara Rao G


 In LazyPersistFileScrubber#clearCorruptLazyPersistFiles collecting blocks for removal and also removing them in write lock.

removeBlocks should be moved out of writelock as removeBlocks has incremental deletion logic in which it will acquire write lock and unlock for every block removal.

If there are more corrupt blocks to remove in cluster, it may hold write lock for longer time. 
{code:java}
  for (BlockCollection bc : filesToDelete) {
          LOG.warn("Removing lazyPersist file " + bc.getName() + " with no replicas.");
          BlocksMapUpdateInfo toRemoveBlocks =
              FSDirDeleteOp.deleteInternal(
                  FSNamesystem.this,
                  INodesInPath.fromINode((INodeFile) bc), false);
          changed |= toRemoveBlocks != null;
          if (toRemoveBlocks != null) {
            removeBlocks(toRemoveBlocks); // Incremental deletion of blocks
          }
        }
{code}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org