You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2025/04/04 05:24:00 UTC

[jira] [Updated] (HDDS-12691) Calculation of committed space in Datanode seems incorrect

     [ https://issues.apache.org/jira/browse/HDDS-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HDDS-12691:
----------------------------------
    Labels: pull-request-available  (was: )

> Calculation of committed space in Datanode seems incorrect
> ----------------------------------------------------------
>
>                 Key: HDDS-12691
>                 URL: https://issues.apache.org/jira/browse/HDDS-12691
>             Project: Apache Ozone
>          Issue Type: Sub-task
>          Components: Ozone Datanode
>            Reporter: Siddhant Sangwan
>            Assignee: Siddhant Sangwan
>            Priority: Major
>              Labels: pull-request-available
>
> The following method is called on every writeChunk request, after writing bytes to the disk to update stats:
> {code}
>   public void incrWriteBytes(long bytes) {
>     long unused = getMaxSize() - getBytesUsed();
>     this.writeBytes.addAndGet(bytes);
>     /*
>        Increase the cached Used Space in VolumeInfo as it
>        maybe not updated, DU or DedicatedDiskSpaceUsage runs
>        periodically to update the Used Space in VolumeInfo.
>      */
>     this.getVolume().incrementUsedSpace(bytes);
>     // only if container size < max size
>     if (committedSpace && unused > 0) {
>       //with this write, container size might breach max size
>       long decrement = Math.min(bytes, unused);
>       this.getVolume().incCommittedBytes(0 - decrement);
>     }
>   }
> {code}
> For a container, 5GB (default) is committed when the container is created. Then every writeChunk is supposed to increment usedSpace and decrement committedSpace. Here, the logic of long decrement = Math.min(bytes, unused); seems incorrect to me.
> Consider that the latest writeChunk wrote 256MB into a close-to-full container. Assume that now, unused = getMaxSize() - getBytesUsed() = 100MB. The current logic would decrement the committedSpace by 100MB, whereas it should have been decremented by 256MB.
> Moreover, what about the case where the because of the latest chunk, the used size of the container exceeds the max size (used > max, for example I've seen 13GB containers in actual clusters since the max is just a soft limit), shouldn't we still decrement the committed space by getMaxSize() - (getBytesUsed() - bytes)? That's the remaining amount of committed space that this container was taking up in the volume before the "bytes" sized chunk was written to it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org