You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/08/19 16:17:00 UTC
[jira] [Updated] (HDDS-4131) Container report should update
container key count and bytes used if they differ in SCM
[ https://issues.apache.org/jira/browse/HDDS-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HDDS-4131:
---------------------------------
Labels: pull-request-available (was: )
> Container report should update container key count and bytes used if they differ in SCM
> ---------------------------------------------------------------------------------------
>
> Key: HDDS-4131
> URL: https://issues.apache.org/jira/browse/HDDS-4131
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: SCM
> Affects Versions: 0.7.0
> Reporter: Stephen O'Donnell
> Assignee: Stephen O'Donnell
> Priority: Major
> Labels: pull-request-available
>
> In HDDS-4037 it was noted that when blocks are deleted from closed containers, the bytesUsed and Key Count metrics on the SCM container are not updated correctly.
> These stats should be updated via the container reports issued by the DNs to SCM periodically. However, in `AbstractContainerReportHandler#updateContainerStats`, the code assumes the values are always increasing and it will not update them if they are decreasing:
> {code}
> private void updateContainerStats(final ContainerID containerId,
> final ContainerReplicaProto replicaProto)
> throws ContainerNotFoundException {
> if (isHealthy(replicaProto::getState)) {
> final ContainerInfo containerInfo = containerManager
> .getContainer(containerId);
> if (containerInfo.getSequenceId() <
> replicaProto.getBlockCommitSequenceId()) {
> containerInfo.updateSequenceId(
> replicaProto.getBlockCommitSequenceId());
> }
> if (containerInfo.getUsedBytes() < replicaProto.getUsed()) {
> containerInfo.setUsedBytes(replicaProto.getUsed());
> }
> if (containerInfo.getNumberOfKeys() < replicaProto.getKeyCount()) {
> containerInfo.setNumberOfKeys(replicaProto.getKeyCount());
> }
> }
> }
> {code}
> In HDDS-4037 a change was made to the Replication Manager, so it updates the stats. However I don't believe that is the correct place to perform this check, and the issue is caused by the logic shared above.
> In this Jira, I have removed the changes to Replication Manager in HDDS-4037 (but retained the other changes), ensuring the problem statistics are only updated via the containers reports if they are different in SCM from what is reported.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org