You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Stephen O'Donnell (Jira)" <ji...@apache.org> on 2022/02/28 16:08:00 UTC

[jira] [Assigned] (HDDS-6384) EC: Ensure EC container usage is updated correctly when handling reports

     [ https://issues.apache.org/jira/browse/HDDS-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stephen O'Donnell reassigned HDDS-6384:
---------------------------------------

    Assignee: Stephen O'Donnell

> EC: Ensure EC container usage is updated correctly when handling reports
> ------------------------------------------------------------------------
>
>                 Key: HDDS-6384
>                 URL: https://issues.apache.org/jira/browse/HDDS-6384
>             Project: Apache Ozone
>          Issue Type: Sub-task
>          Components: SCM
>            Reporter: Stephen O'Donnell
>            Assignee: Stephen O'Donnell
>            Priority: Major
>
> The current container report handling logic in AbstractContainerReportHandler does update the container usage and key count correctly in SCM for EC containers. The current logic says:
> {code:java}
> for (ContainerReplica r : otherReplicas) {
>   // Open containers are generally growing in key count and size, the
>   // overall size should be the min of all reported replicas.
>   if (containerInfo.getState().equals(HddsProtos.LifeCycleState.OPEN)) {
>     usedBytes = Math.min(usedBytes, r.getBytesUsed());
>     keyCount = Math.min(keyCount, r.getKeyCount());
>   } else {
>     // Containers which are not open can only shrink in size, so use the
>     // largest values reported.
>     usedBytes = Math.max(usedBytes, r.getBytesUsed());
>     keyCount = Math.max(keyCount, r.getKeyCount());
>   }
> } {code}
> For EC Containers, the bytes used will be different across all the replicas due to partial stripes. The first data replica and the parity replicas should have the max size, but other data replicas are likely to be smaller. Therefore using the min as above will not work.
> We should either use the max value for open containers, or the min of the first data and parity group, and ignore the usage in the other replicas.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org