You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2025/04/04 05:24:00 UTC
[jira] [Updated] (HDDS-12691) Calculation of committed space in Datanode seems incorrect
[ https://issues.apache.org/jira/browse/HDDS-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HDDS-12691:
----------------------------------
Labels: pull-request-available (was: )
> Calculation of committed space in Datanode seems incorrect
> ----------------------------------------------------------
>
> Key: HDDS-12691
> URL: https://issues.apache.org/jira/browse/HDDS-12691
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: Ozone Datanode
> Reporter: Siddhant Sangwan
> Assignee: Siddhant Sangwan
> Priority: Major
> Labels: pull-request-available
>
> The following method is called on every writeChunk request, after writing bytes to the disk to update stats:
> {code}
> public void incrWriteBytes(long bytes) {
> long unused = getMaxSize() - getBytesUsed();
> this.writeBytes.addAndGet(bytes);
> /*
> Increase the cached Used Space in VolumeInfo as it
> maybe not updated, DU or DedicatedDiskSpaceUsage runs
> periodically to update the Used Space in VolumeInfo.
> */
> this.getVolume().incrementUsedSpace(bytes);
> // only if container size < max size
> if (committedSpace && unused > 0) {
> //with this write, container size might breach max size
> long decrement = Math.min(bytes, unused);
> this.getVolume().incCommittedBytes(0 - decrement);
> }
> }
> {code}
> For a container, 5GB (default) is committed when the container is created. Then every writeChunk is supposed to increment usedSpace and decrement committedSpace. Here, the logic of long decrement = Math.min(bytes, unused); seems incorrect to me.
> Consider that the latest writeChunk wrote 256MB into a close-to-full container. Assume that now, unused = getMaxSize() - getBytesUsed() = 100MB. The current logic would decrement the committedSpace by 100MB, whereas it should have been decremented by 256MB.
> Moreover, what about the case where the because of the latest chunk, the used size of the container exceeds the max size (used > max, for example I've seen 13GB containers in actual clusters since the max is just a soft limit), shouldn't we still decrement the committed space by getMaxSize() - (getBytesUsed() - bytes)? That's the remaining amount of committed space that this container was taking up in the volume before the "bytes" sized chunk was written to it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org