You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Ethan Rose (Jira)" <ji...@apache.org> on 2024/04/22 18:16:00 UTC
[jira] [Commented] (HDDS-10731) Avoid send block delete commands to DN if the involved container is closed and empty
[ https://issues.apache.org/jira/browse/HDDS-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17839809#comment-17839809 ]
Ethan Rose commented on HDDS-10731:
-----------------------------------
AFAIK the version of Ozone that this was encountered on was old, before the significant changes to block and container delete like the {{isEmpty}} flag from HDDS-8115 that datanodes use to initiate container deletion. After this fix empty containers should correctly move to the DELETED state. Once this happens, HDDS-7126 will make sure the transactions for those deleted containers get removed. I don't think this version of Ozone had that fix either. [~Sammi] please double check with the current code and let us know if the issue described here is still relevant.
> Avoid send block delete commands to DN if the involved container is closed and empty
> ------------------------------------------------------------------------------------
>
> Key: HDDS-10731
> URL: https://issues.apache.org/jira/browse/HDDS-10731
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Sammi Chen
> Assignee: Sammi Chen
> Priority: Major
>
> During investigate one SCM crash case, we found that the SCM DB deletedBlock table has 900M records while customer said the DN's disk utilization looks normal and didn't have any major space not reclaimed issue.
> {noformat}
> bin/ozone debug ldb --db=/Users/sammi/workspace/ozone-meta/scm.db scan --cf deletedBlocks --count
> 951177554
> {noformat}
> After further checked the deletedBlock table content, there are lots of txs whose target container is already closed and empty. Here is one example,
> Container 1087
> {noformat}
> {
> "state" : "CLOSED",
> "stateEnterTime" : 1706614618.156000000,
> "replicationConfig" : {
> "replicationFactor" : "THREE",
> "replicationType" : "RATIS"
> },
> "usedBytes" : 0,
> "numberOfKeys" : 0,
> "lastUsed" : 1713412552.968000000,
> "owner" : "om2",
> "containerID" : 1087,
> "deleteTransactionId" : 2071,
> "sequenceId" : 0
> }
> {noformat}
> Block delete transactions for this container 1087
> {noformat}
> 1564: {
> "memoizedSize" : -1,
> "unknownFields" : {
> "fields" : { }
> },
> "bitField0_" : 7,
> "txID_" : 1564,
> "containerID_" : 1087,
> "localID_" : [ 111677748019211347, 111677748019211366, 111677748019211404 ],
> "count_" : 0,
> "memoizedIsInitialized" : 1,
> "memoizedSerializedSize" : -1,
> "memoizedHashCode" : 0
> }
> 1834: {
> "memoizedSize" : -1,
> "unknownFields" : {
> "fields" : { }
> },
> "bitField0_" : 7,
> "txID_" : 1834,
> "containerID_" : 1087,
> "localID_" : [ 111677748019211432 ],
> "count_" : 0,
> "memoizedIsInitialized" : 1,
> "memoizedSerializedSize" : -1,
> "memoizedHashCode" : 0
> }
> 2071: {
> "memoizedSize" : -1,
> "unknownFields" : {
> "fields" : { }
> },
> "bitField0_" : 7,
> "txID_" : 2071,
> "containerID_" : 1087,
> "localID_" : [ 111677748019211288 ],
> "count_" : 0,
> "memoizedIsInitialized" : 1,
> "memoizedSerializedSize" : -1,
> "memoizedHashCode" : 0
> }
> {noformat}
> For those kinds of txs, we can avoid send the commands to DN, instead we can directly delete the txs from the DB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org