[jira] [Commented] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

["Stephen O'Donnell (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038993#comment-17038993 ] 

Stephen O'Donnell commented on HDFS-15177:
------------------------------------------

Hi [~zhuqi], do you have any jstack traces showing this problem in the datanode? As far as I can tell, the FsDatasetImpl lock is not held during the disk operations - it is only held while the block is removed from the ReplicaMap in memory. Therefore I would like to see what code path appears to be holding the lock too long.

What version of HDFS are you seeing this problem on? Is it a 3.x branch or a 2.x branch?

> Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.
> --------------------------------------------------------------------------------------
>
>                 Key: HDFS-15177
>                 URL: https://issues.apache.org/jira/browse/HDFS-15177
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: zhuqi
>            Assignee: zhuqi
>            Priority: Major
>
> In our cluster, the datanode receive the delete command with too many blocks deletion when we have many blockpools sharing the same datanode and the datanode with about 30 storage dirs, it will cause the FsDatasetImpl lock too much time.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org