You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Lokesh Jain (Jira)" <ji...@apache.org> on 2020/10/27 07:21:00 UTC
[jira] [Commented] (HDDS-4344) Block Deletion Performance
Improvements
[ https://issues.apache.org/jira/browse/HDDS-4344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221206#comment-17221206 ]
Lokesh Jain commented on HDDS-4344:
-----------------------------------
HDDS-4297 gives better deletion speed for deleting a milllion keys.
Without PR - SCM sends total 180 blocks for deletion in a minute. This number does not increase even after increasing configs. Datanode deletion speed was very slow. I monitored a datanode disk space and it deleted less than a GB even after 5 minutes.
With PR - SCM sends 6000 blocks for deletion. Datanode was deleting 1 GB of data every minute.
I used the following configs.
ozone.block.deleting.container.limit.per.interval: "1000"
ozone.key.deleting.limit.per.task: "10000"
> Block Deletion Performance Improvements
> ---------------------------------------
>
> Key: HDDS-4344
> URL: https://issues.apache.org/jira/browse/HDDS-4344
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Lokesh Jain
> Assignee: Lokesh Jain
> Priority: Major
> Attachments: Block Deletion Performance.pdf
>
>
> In cluster deployments it was observed that block deletion can be slow. For example if a user writes a million keys in Ozone, the time it takes for those million keys to be deleted from datanodes can be high. The jira would cover various improvements which can be made for better deletion speeds.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org