You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "zhuqi (Jira)" <ji...@apache.org> on 2020/09/18 03:13:00 UTC
[jira] [Commented] (HDFS-15584) Improve HDFS large deletion cause
namenode lockqueue boom and pending deletion boom.
[ https://issues.apache.org/jira/browse/HDFS-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198093#comment-17198093 ]
zhuqi commented on HDFS-15584:
------------------------------
cc [~sodonnell] , [~hexiaoqiao]
I add the draft patch without unit test.
I add the wait lock time, also i add the threshold to control the wait lock interval. If the lock time of block deletion exceed the threshold of the total lock time, we will wait the lock.
All two we can configure according cluster situation.
If you any advice , thanks.
> Improve HDFS large deletion cause namenode lockqueue boom and pending deletion boom.
> ------------------------------------------------------------------------------------
>
> Key: HDFS-15584
> URL: https://issues.apache.org/jira/browse/HDFS-15584
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 3.4.0
> Reporter: zhuqi
> Assignee: zhuqi
> Priority: Major
> Attachments: HDFS-15584.001.patch
>
>
> In our production cluster, the large deletion will boom the namenode lock queue, also will lead to the boom of pending deletion in invalidate blocks.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org