You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Viraj Jasani (Jira)" <ji...@apache.org> on 2023/01/14 00:18:00 UTC
[jira] [Created] (HDFS-16891) Avoid the overhead of copy-on-write exception list while loading inodes sub sections in parallel
Viraj Jasani created HDFS-16891:
-----------------------------------
Summary: Avoid the overhead of copy-on-write exception list while loading inodes sub sections in parallel
Key: HDFS-16891
URL: https://issues.apache.org/jira/browse/HDFS-16891
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Affects Versions: 3.3.4
Reporter: Viraj Jasani
Assignee: Viraj Jasani
If we enable parallel loading and persisting of inodes from/to fs image, we get the benefit of improved performance. However, while loading sub-sections INODE_DIR_SUB and INODE_SUB, if we encounter any errors, we use copy-on-write list to maintain the list of exceptions. Since our usecase is not to iterate over this list while executor threads are adding new elements to the list, using copy-on-write is bit of an overhead for this usecase.
It would be better to synchronize adding new elements to the list rather than having the list copy all elements over every time new element is added to the list.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org