You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/02/09 06:57:16 UTC
[GitHub] [pulsar] lhotari commented on a change in pull request #9393: Fix testBrokerSelectionForAntiAffinityGroup by increasing OverloadedThreshold
lhotari commented on a change in pull request #9393:
URL: https://github.com/apache/pulsar/pull/9393#discussion_r572635313
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LinuxBrokerHostUsageImpl.java
##########
@@ -75,18 +75,17 @@ public LinuxBrokerHostUsageImpl(int hostUsageCheckIntervalMin,
this.lastCollection = 0L;
this.usage = new SystemResourceUsage();
this.overrideBrokerNicSpeedGbps = overrideBrokerNicSpeedGbps;
- executorService.scheduleAtFixedRate(this::calculateBrokerHostUsage, 0,
- hostUsageCheckIntervalMin, TimeUnit.MINUTES);
boolean isCGroupsEnabled = false;
try {
isCGroupsEnabled = Files.exists(Paths.get(CGROUPS_CPU_USAGE_PATH));
} catch (Exception e) {
log.warn("Failed to check cgroup CPU usage file: {}", e.getMessage());
}
-
this.isCGroupsEnabled = isCGroupsEnabled;
- calculateBrokerHostUsage();
+
+ executorService.scheduleAtFixedRate(this::calculateBrokerHostUsage, 0,
Review comment:
@michaeljmarshall I was also assuming some race condition and I made an experiment on top of your changes where I replaced the `Files.readAllBytes` usage and made `ResourceUsage` immutable: https://github.com/apache/pulsar/compare/master...lhotari:issue-6368-flaky-test . However, the same problem still reproduces. CPU shows as Infinity in the log lines:
```
06:52:40.333 [TestNG-method=testBrokerSelectionForAntiAffinityGroup-1] INFO org.apache.pulsar.broker.loadbala
nce.impl.ModularLoadManagerImpl - 2 brokers being considered for assignment of tenant-98033a04-3a4d-4e5d-9203-
3147486f2673/use/ns1/0x00000000_0xffffffff
06:52:40.333 [TestNG-method=testBrokerSelectionForAntiAffinityGroup-1] WARN org.apache.pulsar.broker.loadbala
nce.impl.LeastLongTermMessageRate - Broker http://localhost:35061 is overloaded: max usage=Infinity
06:52:40.333 [TestNG-method=testBrokerSelectionForAntiAffinityGroup-1] WARN org.apache.pulsar.broker.loadbala
nce.impl.LeastLongTermMessageRate - Broker localhost:35061 is overloaded: CPU: Infinity%, MEMORY: 16.867092%,
DIRECT MEMORY: 2.4597175%, BANDWIDTH IN: NaN%, BANDWIDTH OUT: NaN%
06:52:40.337 [TestNG-method=testBrokerSelectionForAntiAffinityGroup-1] INFO org.apache.pulsar.broker.loadbala
nce.impl.ModularLoadManagerImpl - 1 brokers being considered for assignment of tenant-98033a04-3a4d-4e5d-9203-
3147486f2673/use/ns2/0x00000000_0xffffffff
06:52:40.337 [TestNG-method=testBrokerSelectionForAntiAffinityGroup-1] WARN org.apache.pulsar.broker.loadbala
nce.impl.LeastLongTermMessageRate - Broker http://localhost:35061 is overloaded: max usage=Infinity
06:52:40.337 [TestNG-method=testBrokerSelectionForAntiAffinityGroup-1] WARN org.apache.pulsar.broker.loadbala
nce.impl.LeastLongTermMessageRate - Broker localhost:35061 is overloaded: CPU: Infinity%, MEMORY: 16.867092%,
DIRECT MEMORY: 2.4597175%, BANDWIDTH IN: NaN%, BANDWIDTH OUT: NaN%
06:52:40.338 [TestNG-method=testBrokerSelectionForAntiAffinityGroup-1] WARN org.apache.pulsar.broker.loadbala
nce.impl.LeastLongTermMessageRate - Broker http://localhost:35061 is overloaded: max usage=Infinity
06:52:40.338 [TestNG-method=testBrokerSelectionForAntiAffinityGroup-1] WARN org.apache.pulsar.broker.loadbala
nce.impl.LeastLongTermMessageRate - Broker localhost:35061 is overloaded: CPU: Infinity%, MEMORY: 16.867092%,
DIRECT MEMORY: 2.4597175%, BANDWIDTH IN: NaN%, BANDWIDTH OUT: NaN%
!!!!!!!!! FAILURE-- [TestClass name=class org.apache.pulsar.broker.loadbalance.AntiAffinityNamespaceGroupTest]
.testBrokerSelectionForAntiAffinityGroup([])-------
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org