You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Attila Doroszlai (Jira)" <ji...@apache.org> on 2025/03/28 11:01:00 UTC
[jira] [Commented] (HDDS-12709) Intermittent failure in Balancer acceptance test
[ https://issues.apache.org/jira/browse/HDDS-12709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17939212#comment-17939212 ]
Attila Doroszlai commented on HDDS-12709:
-----------------------------------------
Importing several containers each with 20MB, but default container size is 100MB, and we reserve 2x during import. So concurrent import fails:
{code}
2025-03-28 10:20:58,460 [459ba096-e723-49c4-8f09-d0b16f4c2651-ReplicationContainerReader-1] INFO replication.SendContainerRequestHandler: Accepting container 5
2025-03-28 10:20:58,807 [459ba096-e723-49c4-8f09-d0b16f4c2651-ReplicationContainerReader-2] INFO replication.SendContainerRequestHandler: Container 5 is downloaded with size 20997632, starting to import.
2025-03-28 10:20:59,104 [459ba096-e723-49c4-8f09-d0b16f4c2651-ReplicationContainerReader-2] INFO replication.SendContainerRequestHandler: Container 5 is replicated successfully
2025-03-28 10:21:12,207 [459ba096-e723-49c4-8f09-d0b16f4c2651-ReplicationContainerReader-4] INFO replication.SendContainerRequestHandler: Accepting container 2
2025-03-28 10:21:12,369 [459ba096-e723-49c4-8f09-d0b16f4c2651-ReplicationContainerReader-8] INFO replication.SendContainerRequestHandler: Accepting container 3
2025-03-28 10:21:12,410 [459ba096-e723-49c4-8f09-d0b16f4c2651-ReplicationContainerReader-1] INFO replication.SendContainerRequestHandler: Accepting container 4
2025-03-28 10:21:12,690 [459ba096-e723-49c4-8f09-d0b16f4c2651-ReplicationContainerReader-4] INFO replication.SendContainerRequestHandler: Container 2 is downloaded with size 29426176, starting to import.
2025-03-28 10:21:12,756 [459ba096-e723-49c4-8f09-d0b16f4c2651-ReplicationContainerReader-7] INFO volume.CapacityVolumeChoosingPolicy: No volumes have enough space for a new container. Most available space: 154062848 bytes; required space: 209715200, volumes: [{ dir=/data/hdds/hdds type=DISK id=DS-e9e757ec-e4a9-42dc-833e-6cf2f2dfb4cb capacity=1073634449 used=20979712 available=888066048 minFree=104857600 committed=629145600 }]
2025-03-28 10:21:12,759 [459ba096-e723-49c4-8f09-d0b16f4c2651-ReplicationContainerReader-7] WARN replication.SendContainerRequestHandler: Error receiving container 1 at 0
org.apache.hadoop.util.DiskChecker$DiskOutOfSpaceException: No volumes have enough space for a new container. Most available space: 154062848 bytes
at org.apache.hadoop.ozone.container.common.volume.VolumeChoosingUtil.throwDiskOutOfSpace(VolumeChoosingUtil.java:38)
at org.apache.hadoop.ozone.container.common.volume.CapacityVolumeChoosingPolicy.chooseVolume(CapacityVolumeChoosingPolicy.java:66)
at org.apache.hadoop.ozone.container.replication.ContainerImporter.chooseNextVolume(ContainerImporter.java:153)
at org.apache.hadoop.ozone.container.replication.SendContainerRequestHandler.onNext(SendContainerRequestHandler.java:89)
{code}
> Intermittent failure in Balancer acceptance test
> ------------------------------------------------
>
> Key: HDDS-12709
> URL: https://issues.apache.org/jira/browse/HDDS-12709
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: test
> Reporter: Attila Doroszlai
> Assignee: Attila Doroszlai
> Priority: Major
>
> {code}
> Verify Container Balancer for RATIS/EC containers | FAIL |
> Keyword 'Run Balancer Verbose History Status' failed after retrying for 40 seconds. The last error was: 'ContainerBalancer is Running.
> {code}
> {code}
> Balancing duration: 43s
> ...
> Size scheduled to move 102 MB
> Moved data size 82 MB
> Scheduled to move containers 5
> Already moved containers 4
> {code}
> Balancer is progressing, but not finished in 40 seconds. I guess the timeout needs to be increased.
> Probably related to HDDS-12235.
> CC [~ashishkr], [~Sammi], [~siddhant], [~sumitagrawl]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org