You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Stephen O'Donnell (Jira)" <ji...@apache.org> on 2021/12/20 12:13:00 UTC
[jira] [Resolved] (HDFS-16386) Reduce DataNode load when FsDatasetAsyncDiskService is working
[ https://issues.apache.org/jira/browse/HDFS-16386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen O'Donnell resolved HDFS-16386.
--------------------------------------
Resolution: Fixed
> Reduce DataNode load when FsDatasetAsyncDiskService is working
> --------------------------------------------------------------
>
> Key: HDFS-16386
> URL: https://issues.apache.org/jira/browse/HDFS-16386
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.9.2
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.2.4, 3.3.3
>
> Attachments: monitor.png
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Our DataNode node has 36 disks. When FsDatasetAsyncDiskService is working, it will cause a high load on the DataNode.
> Here are some monitoring related to memory:
> !monitor.png!
> Since each disk deletes the block asynchronously, and each thread allows 4 threads to work,
> This will cause some troubles to the DataNode, such as increased cpu and increased memory.
> We should appropriately reduce the number of jobs of the total thread so that the DataNode can work better.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org