You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/08/05 14:54:00 UTC
[jira] [Commented] (HDFS-16524) Add configuration to control blocks deletion asynchronous or synchronous
[ https://issues.apache.org/jira/browse/HDFS-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575888#comment-17575888 ]
ASF GitHub Bot commented on HDFS-16524:
---------------------------------------
ted12138 commented on code in PR #4139:
URL: https://github.com/apache/hadoop/pull/4139#discussion_r938898456
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##########
@@ -3410,6 +3410,33 @@ FSPermissionChecker getPermissionChecker()
return dir.getPermissionChecker();
}
+ /**
+ * If blockDeletionAsync enables, blocks will be deleted asynchronously.
+ * If not, incrementally remove the blocks from blockManager
+ * Writelock is dropped and reacquired every BLOCK_DELETION_INCREMENT to
+ * ensure that other waiters on the lock can get in. See HDFS-2938
+ *
+ * @param toDeleteList
+ * a list of blocks that need to be removed from blocksMap
+ */
+ void removeBlocks(List<BlockInfo> toDeleteList) {
+ if (this.blockDeletionAsync) {
+ blockManager.addBLocksToMarkedDeleteQueue(toDeleteList);
+ } else {
+ Iterator<BlockInfo> iter = toDeleteList.iterator();
+ while (iter.hasNext()) {
+ writeLock();
+ try {
+ for (int i = 0; i < blockDeletionIncrement && iter.hasNext(); i++) {
Review Comment:
About "blockDeletionIncrement", in my opinion, why we removed it in HDFS-16554 is that we decided to use delete async to replace the previous method, so the "blockDeletionIncrement" won't be used any more. But when we add a switch to control blocks deletion async, we need to save the previous block deletion method, so we still need it.
> Add configuration to control blocks deletion asynchronous or synchronous
> ------------------------------------------------------------------------
>
> Key: HDFS-16524
> URL: https://issues.apache.org/jira/browse/HDFS-16524
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: leo sun
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When users delete files frequently, deletion is slower than its enqueues. The pending deletions blocks will keep increasing.
> Under this circumstance, deletion synchronous is better.
> So I will add configuration to switch deletion asynchronous or synchronous.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org