You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "Demogorgon314 (via GitHub)" <gi...@apache.org> on 2023/03/02 01:57:16 UTC
[GitHub] [pulsar] Demogorgon314 opened a new pull request, #19613: [improve][broker] PIP-192: Add large topic count filter
Demogorgon314 opened a new pull request, #19613:
URL: https://github.com/apache/pulsar/pull/19613
Master Issue: https://github.com/apache/pulsar/issues/16691
### Motivation
We will start raising PRs to implement PIP-192, https://github.com/apache/pulsar/issues/16691
### Modifications
This PR added a filter that can filter brokers based on the number of topics.
### TODO
The broker load data might be delayed, so the max topic check might not be accurate.
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] heesung-sn commented on a diff in pull request #19613: [improve][broker] PIP-192: Add large topic count filter
Posted by "heesung-sn (via GitHub)" <gi...@apache.org>.
heesung-sn commented on code in PR #19613:
URL: https://github.com/apache/pulsar/pull/19613#discussion_r1119150532
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/reporter/BrokerLoadDataReporter.java:
##########
@@ -77,12 +79,18 @@ public BrokerLoadData generateLoadData() {
final var pulsarStats = pulsar.getBrokerService().getPulsarStats();
synchronized (pulsarStats) {
var brokerStats = pulsarStats.getBrokerStats();
+ var bundleStats = pulsarStats.getBundleStats();
+ var topics = 0;
+ for (Map.Entry<String, NamespaceBundleStats> bundleStatsEntry : bundleStats.entrySet()) {
Review Comment:
Can we define `topics` in `BrokerStats` and pre-aggregate this topic counts in `updateStats`?
```
public synchronized void updateStats(
ConcurrentOpenHashMap<String, ConcurrentOpenHashMap<String, ConcurrentOpenHashMap<String, Topic>>>
topicsMap) {
...
bundles.forEach((bundle, topics) -> {
NamespaceBundleStats currentBundleStats = bundleStats.computeIfAbsent(bundle,
k -> new NamespaceBundleStats());
currentBundleStats.reset();
currentBundleStats.topics = topics.size();
brokerStats.topics += topics.size(); //pre-aggregate
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- merged pull request #19613: [improve][broker] PIP-192: Add large topic count filter
Posted by "Technoboy- (via GitHub)" <gi...@apache.org>.
Technoboy- merged PR #19613:
URL: https://github.com/apache/pulsar/pull/19613
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- closed pull request #19613: [improve][broker] PIP-192: Add large topic count filter
Posted by "Technoboy- (via GitHub)" <gi...@apache.org>.
Technoboy- closed pull request #19613: [improve][broker] PIP-192: Add large topic count filter
URL: https://github.com/apache/pulsar/pull/19613
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Demogorgon314 commented on a diff in pull request #19613: [improve][broker] PIP-192: Add large topic count filter
Posted by "Demogorgon314 (via GitHub)" <gi...@apache.org>.
Demogorgon314 commented on code in PR #19613:
URL: https://github.com/apache/pulsar/pull/19613#discussion_r1119504325
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/reporter/BrokerLoadDataReporter.java:
##########
@@ -77,12 +79,18 @@ public BrokerLoadData generateLoadData() {
final var pulsarStats = pulsar.getBrokerService().getPulsarStats();
synchronized (pulsarStats) {
var brokerStats = pulsarStats.getBrokerStats();
+ var bundleStats = pulsarStats.getBundleStats();
+ var topics = 0;
+ for (Map.Entry<String, NamespaceBundleStats> bundleStatsEntry : bundleStats.entrySet()) {
Review Comment:
Yes, pre-aggregate will be better. Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org