You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Sammi Chen (Jira)" <ji...@apache.org> on 2020/07/04 09:17:00 UTC
[jira] [Updated] (HDDS-3921) IllegalArgumentException triggered in
SCMContainerPlacementRackAware.chooseDatanodes
[ https://issues.apache.org/jira/browse/HDDS-3921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sammi Chen updated HDDS-3921:
-----------------------------
Description:
The root cause is existing replicas plus replicas in replicationInFlight meet the crossing rack requriement, for this case, we should check
if misReplicated again.
2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.SCMCommonPlacementPolicy: currentRackCount 1, replicas 3, requiredRacks 2, numRacks 4
2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.container.ReplicationManager: misReplicated = true, replicas size = 1
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: Handling underreplicated container: 250499
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: replicas of container {}#250499
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 9.179.142.198 CLOSED
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: deletionInFlight of container {}#250499
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: replicationInFlight of container {}#250499
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 9.180.19.147
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 100.76.61.31
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: source of container {}#250499
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 9.179.142.198
2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.container.ReplicationManager: Container #250499 expected replica count 3, but found 3, delta 0.
2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.SCMCommonPlacementPolicy: currentRackCount 2, replicas 3, requiredRacks 2, numRacks 4
2020-07-04 16:25:01,496 [ReplicationMonitor] WARN org.apache.hadoop.hdds.scm.container.ReplicationManager: Process container #250499 error:
java.lang.IllegalArgumentException
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
at org.apache.hadoop.hdds.scm.container.placement.algorithms.SCMContainerPlacementRackAware.chooseDatanodes(SCMContainerPlacementRackAware.java:101)
at org.apache.hadoop.hdds.scm.container.ReplicationManager.handleUnderReplicatedContainer(ReplicationManager.java:578)
at org.apache.hadoop.hdds.scm.container.ReplicationManager.processContainer(ReplicationManager.java:331)
at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1082)
at org.apache.hadoop.hdds.scm.container.ReplicationManager.run(ReplicationManager.java:238)
at java.lang.Thread.run(Thread.java:748)
was:
2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.SCMCommonPlacementPolicy: currentRackCount 1, replicas 3, requiredRacks 2, numRacks 4
2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.container.ReplicationManager: misReplicated = true, replicas size = 1
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: Handling underreplicated container: 250499
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: replicas of container {}#250499
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 9.179.142.198 CLOSED
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: deletionInFlight of container {}#250499
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: replicationInFlight of container {}#250499
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 9.180.19.147
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 100.76.61.31
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: source of container {}#250499
2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 9.179.142.198
2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.container.ReplicationManager: Container #250499 expected replica count 3, but found 3, delta 0.
2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.SCMCommonPlacementPolicy: currentRackCount 2, replicas 3, requiredRacks 2, numRacks 4
2020-07-04 16:25:01,496 [ReplicationMonitor] WARN org.apache.hadoop.hdds.scm.container.ReplicationManager: Process container #250499 error:
java.lang.IllegalArgumentException
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
at org.apache.hadoop.hdds.scm.container.placement.algorithms.SCMContainerPlacementRackAware.chooseDatanodes(SCMContainerPlacementRackAware.java:101)
at org.apache.hadoop.hdds.scm.container.ReplicationManager.handleUnderReplicatedContainer(ReplicationManager.java:578)
at org.apache.hadoop.hdds.scm.container.ReplicationManager.processContainer(ReplicationManager.java:331)
at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1082)
at org.apache.hadoop.hdds.scm.container.ReplicationManager.run(ReplicationManager.java:238)
at java.lang.Thread.run(Thread.java:748)
> IllegalArgumentException triggered in SCMContainerPlacementRackAware.chooseDatanodes
> ------------------------------------------------------------------------------------
>
> Key: HDDS-3921
> URL: https://issues.apache.org/jira/browse/HDDS-3921
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Sammi Chen
> Assignee: Sammi Chen
> Priority: Major
>
> The root cause is existing replicas plus replicas in replicationInFlight meet the crossing rack requriement, for this case, we should check
> if misReplicated again.
> 2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.SCMCommonPlacementPolicy: currentRackCount 1, replicas 3, requiredRacks 2, numRacks 4
> 2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.container.ReplicationManager: misReplicated = true, replicas size = 1
> 2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: Handling underreplicated container: 250499
> 2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: replicas of container {}#250499
> 2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 9.179.142.198 CLOSED
> 2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: deletionInFlight of container {}#250499
> 2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: replicationInFlight of container {}#250499
> 2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 9.180.19.147
> 2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 100.76.61.31
> 2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: source of container {}#250499
> 2020-07-04 16:25:01,496 [ReplicationMonitor] DEBUG org.apache.hadoop.hdds.scm.container.ReplicationManager: 9.179.142.198
> 2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.container.ReplicationManager: Container #250499 expected replica count 3, but found 3, delta 0.
> 2020-07-04 16:25:01,496 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.SCMCommonPlacementPolicy: currentRackCount 2, replicas 3, requiredRacks 2, numRacks 4
> 2020-07-04 16:25:01,496 [ReplicationMonitor] WARN org.apache.hadoop.hdds.scm.container.ReplicationManager: Process container #250499 error:
> java.lang.IllegalArgumentException
> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
> at org.apache.hadoop.hdds.scm.container.placement.algorithms.SCMContainerPlacementRackAware.chooseDatanodes(SCMContainerPlacementRackAware.java:101)
> at org.apache.hadoop.hdds.scm.container.ReplicationManager.handleUnderReplicatedContainer(ReplicationManager.java:578)
> at org.apache.hadoop.hdds.scm.container.ReplicationManager.processContainer(ReplicationManager.java:331)
> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
> at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1082)
> at org.apache.hadoop.hdds.scm.container.ReplicationManager.run(ReplicationManager.java:238)
> at java.lang.Thread.run(Thread.java:748)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org