You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/07/22 10:44:30 UTC
[GitHub] [pulsar] Nicklee007 opened a new pull request, #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Nicklee007 opened a new pull request, #16743:
URL: https://github.com/apache/pulsar/pull/16743
### Motivation
Avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight.
When loadManager leader running a period of time, `brokerAvgResourceUsageWithWeight` will contain almost all brokers, the judge will cause we selected broker exceed candidates, it's out of expectation.
### Modifications
1. check if `candidates` contain the broker before add the broker to `bestBrokers`.
### Documentation
- [X] `doc-not-needed`
(Please explain why)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1193313917
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1200652529
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1193306724
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1207779133
Removed the release/2.10.2 and release/2.9.4 labels since we don't have LeastResourceUsageWithWeight on branch-2.10 and branch-2.9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1193125261
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1194007184
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1193311833
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Demogorgon314 commented on a diff in pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Demogorgon314 commented on code in PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#discussion_r928273654
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LeastResourceUsageWithWeight.java:
##########
@@ -139,7 +139,8 @@ public Optional<String> selectBroker(Set<String> candidates, BundleData bundleTo
final double diffThreshold =
conf.getLoadBalancerAverageResourceUsageDifferenceThresholdPercentage() / 100.0;
brokerAvgResourceUsageWithWeight.forEach((broker, avgResUsage) -> {
- if (avgResUsage + diffThreshold <= avgUsage) {
+ if ((avgResUsage + diffThreshold <= avgUsage)
+ && candidates.contains(broker)) {
bestBrokers.add(broker);
}
});
Review Comment:
We can change it to be more simple, right?
```java
for (String broker : candidates) {
Double avgResUsage = brokerAvgResourceUsageWithWeight.getOrDefault(broker, Double.MAX_VALUE);
if ((avgResUsage + diffThreshold <= avgUsage)) {
bestBrokers.add(broker);
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on a diff in pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on code in PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#discussion_r928517423
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LeastResourceUsageWithWeight.java:
##########
@@ -139,7 +139,8 @@ public Optional<String> selectBroker(Set<String> candidates, BundleData bundleTo
final double diffThreshold =
conf.getLoadBalancerAverageResourceUsageDifferenceThresholdPercentage() / 100.0;
brokerAvgResourceUsageWithWeight.forEach((broker, avgResUsage) -> {
- if (avgResUsage + diffThreshold <= avgUsage) {
+ if ((avgResUsage + diffThreshold <= avgUsage)
+ && candidates.contains(broker)) {
bestBrokers.add(broker);
}
});
Review Comment:
you're right, `candidates` is smaller than `brokerAvgResourceUsageWithWeight`, changed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1193042028
> Nice catch! Could you add a unit test to protect this change?
+1 Please add test to avoid regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1193960627
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaozhangmin merged pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
gaozhangmin merged PR #16743:
URL: https://github.com/apache/pulsar/pull/16743
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
HQebupt commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1193047332
Nice catch. +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1193289357
> Nice catch! Could you add a unit test to protect this change?
@codelipenghui @shibd add unit test,PTAL~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1193928928
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1198940731
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1194955417
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1193301450
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Nicklee007 commented on pull request #16743: [fix][broker]avoid the bestBroker choice broker exceed candidates when selectBroker in LeastResourceUsageWithWeight
Posted by GitBox <gi...@apache.org>.
Nicklee007 commented on PR #16743:
URL: https://github.com/apache/pulsar/pull/16743#issuecomment-1194023884
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org