[jira] [Resolved] (HDFS-16785) Avoid to hold write lock to improve performance when add volume.

["Xiaoqiao He (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xiaoqiao He resolved HDFS-16785.
--------------------------------
    Fix Version/s: 3.4.0
     Hadoop Flags: Reviewed
       Resolution: Fixed

Committed to trunk. Thanks [~xuzq_zander] for your works.

> Avoid to hold write lock to improve performance when add volume. 
> -----------------------------------------------------------------
>
>                 Key: HDFS-16785
>                 URL: https://issues.apache.org/jira/browse/HDFS-16785
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: ZanderXu
>            Assignee: ZanderXu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.4.0
>
>
> When patching the fine-grained locking of datanode, I  found that `addVolume` will hold the write block of the BP lock to scan the new volume to get the blocks. If we try to add one full volume that was fixed offline before, i will hold the write lock for a long time.
> The related code as bellows:
> {code:java}
> for (final NamespaceInfo nsInfo : nsInfos) {
>   String bpid = nsInfo.getBlockPoolID();
>   try (AutoCloseDataSetLock l = lockManager.writeLock(LockLevel.BLOCK_POOl, bpid)) {
>     fsVolume.addBlockPool(bpid, this.conf, this.timer);
>     fsVolume.getVolumeMap(bpid, tempVolumeMap, ramDiskReplicaTracker);
>   } catch (IOException e) {
>     LOG.warn("Caught exception when adding " + fsVolume +
>         ". Will throw later.", e);
>     exceptions.add(e);
>   }
> } {code}
> And I noticed that this lock is added by HDFS-15382, means that this logic is not in lock before. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org