You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/09/22 15:49:50 UTC
[GitHub] [pulsar] lhotari opened a new pull request #12137: [Broker] Change the default number of namespace bundles to 32
lhotari opened a new pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137
### Motivation
Currently, the default number of namespace bundles is 4. This leads to cases where a lot of topics can reside in just 1 or 2 brokers in 3 broker cluster.
Side note:
A good rule of thumb rule for defining the number of shards is to have at least 10x more shards than there are nodes. In Pulsar a "namespace bundle" behaves like a shard. In addition to this, some sharding algorithms might provide a better distribution when the number of shards is a power of 2.
From these rules, you could get that a 3 broker node cluster should use 32 bundles, a 4-6 node cluster 64 bundles and 7-12 node cluster should use 128 bundles.
This might not be the best advice, but I haven't seen better rule of thumbs rules.
### Modifications
Change the defaultNumberOfNamespaceBundles to 32.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari removed a comment on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari removed a comment on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925285755
> user can always enable auto bundle unloading in loadbalancer. and default-bundle =4 is not defined for test cluster with 1 namespace but it considers a normal cluster that is serving multiple namespaces with considerable traffic. again, higher number of bundles comes with cost specially for znodes and default behavior unnecessarily will add more bundles for users who are using default conf file.
I have a test cluster on GCP/GKE where the auto bundle unloading never happens. It's using the default settings which contains
```
loadBalancerBrokerOverloadedThresholdPercentage=85
loadBalancerAutoUnloadSplitBundlesEnabled=true
loadBalancerNamespaceBundleMaxSessions=1000
```
The test load is created with this type of pulsar-perf command (using 2.7.3 pulsar-perf since current [2.8.0 contains a bug that is fixed in master](https://github.com/apache/pulsar/pull/11824), the bug causes partitions to stop consuming in pulsar-perf).
```
# create a 100 partition topic
pulsar-admin topics create-partitioned-topic -p 100 persistent://public/default/perftest
# produce messages
PULSAR_EXTRA_OPTS="-Xms1500M -Xmx1500M -XX:MaxDirectMemorySize=768M" /pulsar/bin/pulsar-perf produce -u pulsar+ssl://pulsar-broker:6651/ -ioThreads 4 -s 512 -r 50000 persistent://public/default/perftest
# consume messages
PULSAR_EXTRA_OPTS="-Xms1500M -Xmx1500M -XX:MaxDirectMemorySize=768M" /pulsar/bin/pulsar-perf consume -u pulsar+ssl://pulsar-broker:6651/ -s sub persistent://public/default/perftest
```
(these are run on a pod which has 2CPU + 4GB RAM requested/limits, using pulsar-perf from Pulsar 2.7.3 docker image)
The traffic can go on for hours and the CPU load will be very low. Since the threshold never happens.
In this case, the GKE cluster has a nodepool with 6 n2-standard-32 VMs (32 cores, 128GB RAM). Storage for bookies in on SSD.
The impact of this is that one of the brokers will never get any real load on it.
After changing to a namespace with 32 bundles, the load was evenly distributed. This is the reason why I think that `defaultNumberOfNamespaceBundles=4` is not a good default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-932034800
I'm closing this since the `public/default` is currently created without using the `defaultNumberOfNamespaceBundles` at all. It's currently hard code to 16 as explained in https://github.com/apache/pulsar/pull/12137#issuecomment-925695116 . Therefore there isn't a real need to change `defaultNumberOfNamespaceBundles` since `public/default` has 16 in every case.
I had done the analysis of the problem incorrectly and the actual problem that load didn't get balanced was caused by the behavior of the Pulsar load balancer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
sijie commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-926870223
Should this be considered for a PIP since we are changing a default value that can potentially impact users that would create a lot of namespaces?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] rdhabalia commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
rdhabalia commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925167716
`defaultNumberOfNamespaceBundles` is a configurable param and it can be configured with different values for different use cases. Having large number of bundles come with its cost and it should not be large enough by default for every namespace. We still have kept 4 by default which should be more than enough. if specific usecase require 16/32/64 bundles then user should update configuration file in the system and we should avoid changing default values based on specific user's usecase,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925250699
> `defaultNumberOfNamespaceBundles` is a configurable param and it can be configured with different values for different use cases. Having large number of bundles come with its cost and it should not be large enough by default for every namespace. We still have kept 4 by default which should be more than enough. if specific usecase require 16/32/64 bundles then user should update configuration file in the system and we should avoid changing default values based on specific user's usecase,
That's true that it's configurable, but the default value 4 leads often to a case where a broker in a 3 broker cluster doesn't get bundles assigned and is completely idling while the other 2 brokers take all of the load. I can provide a repro for this case if that helps describing the problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari closed pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari closed pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari edited a comment on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925285825
> user can always enable auto bundle unloading in loadbalancer. and default-bundle =4 is not defined for test cluster with 1 namespace but it considers a normal cluster that is serving multiple namespaces with considerable traffic. again, higher number of bundles comes with cost specially for znodes and default behavior unnecessarily will add more bundles for users who are using default conf file.
I have a test cluster on GCP/GKE where the auto bundle unloading never happens and one of the broker pods is completely idling (all 4 bundles get assigned to 2 of the 3 broker pods).
It's using the default settings which contains
```
loadBalancerBrokerOverloadedThresholdPercentage=85
loadBalancerAutoUnloadSplitBundlesEnabled=true
loadBalancerNamespaceBundleMaxSessions=1000
```
The test load is created with this type of pulsar-perf command (using 2.7.3 pulsar-perf since current [2.8.0 contains a bug that is fixed in master](https://github.com/apache/pulsar/pull/11824), the bug causes partitions to stop consuming in pulsar-perf).
```
# create a 100 partition topic
pulsar-admin topics create-partitioned-topic -p 100 persistent://public/default/perftest
# produce messages
PULSAR_EXTRA_OPTS="-Xms1500M -Xmx1500M -XX:MaxDirectMemorySize=768M" /pulsar/bin/pulsar-perf produce -u pulsar+ssl://pulsar-broker:6651/ -ioThreads 4 -s 512 -r 50000 persistent://public/default/perftest
# consume messages
PULSAR_EXTRA_OPTS="-Xms1500M -Xmx1500M -XX:MaxDirectMemorySize=768M" /pulsar/bin/pulsar-perf consume -u pulsar+ssl://pulsar-broker:6651/ -s sub persistent://public/default/perftest
```
(these are run on a pod which has 2CPU + 4GB RAM requested/limits, using pulsar-perf from Pulsar 2.7.3 docker image)
The traffic can go on for hours and the CPU load will be very low. Since the threshold never gets exceeded, the auto unload and splitting of the bundle never happens.
In this case, the GKE cluster has a nodepool with 6 n2-standard-32 VMs (32 cores, 128GB RAM). Storage for bookies in on SSD.
The impact of this is that one of the brokers will never get any real load on it.
After changing to a namespace with 32 bundles, the load was evenly distributed. This is the reason why I think that `defaultNumberOfNamespaceBundles=4` is not a good default.
The bundle splitting has an open bug #5510 which is another reason to use a higher number of default namespace bundles.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925285755
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
merlimat commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-926713000
> I now checked with pulsar-admin namespaces bundles public/default and there's 44 for public/default in this perf testing environment.
Yes, they should be getting split when the load increases, still it takes some time to converge to a reasonable number
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari edited a comment on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925285825
> user can always enable auto bundle unloading in loadbalancer. and default-bundle =4 is not defined for test cluster with 1 namespace but it considers a normal cluster that is serving multiple namespaces with considerable traffic. again, higher number of bundles comes with cost specially for znodes and default behavior unnecessarily will add more bundles for users who are using default conf file.
I have a test cluster on GCP/GKE where the auto bundle unloading never happens and one of the broker pods is completely idling.
It's using the default settings which contains
```
loadBalancerBrokerOverloadedThresholdPercentage=85
loadBalancerAutoUnloadSplitBundlesEnabled=true
loadBalancerNamespaceBundleMaxSessions=1000
```
The test load is created with this type of pulsar-perf command (using 2.7.3 pulsar-perf since current [2.8.0 contains a bug that is fixed in master](https://github.com/apache/pulsar/pull/11824), the bug causes partitions to stop consuming in pulsar-perf).
```
# create a 100 partition topic
pulsar-admin topics create-partitioned-topic -p 100 persistent://public/default/perftest
# produce messages
PULSAR_EXTRA_OPTS="-Xms1500M -Xmx1500M -XX:MaxDirectMemorySize=768M" /pulsar/bin/pulsar-perf produce -u pulsar+ssl://pulsar-broker:6651/ -ioThreads 4 -s 512 -r 50000 persistent://public/default/perftest
# consume messages
PULSAR_EXTRA_OPTS="-Xms1500M -Xmx1500M -XX:MaxDirectMemorySize=768M" /pulsar/bin/pulsar-perf consume -u pulsar+ssl://pulsar-broker:6651/ -s sub persistent://public/default/perftest
```
(these are run on a pod which has 2CPU + 4GB RAM requested/limits, using pulsar-perf from Pulsar 2.7.3 docker image)
The traffic can go on for hours and the CPU load will be very low. Since the threshold never gets exceeded, the auto unload and splitting of the bundle never happens.
In this case, the GKE cluster has a nodepool with 6 n2-standard-32 VMs (32 cores, 128GB RAM). Storage for bookies in on SSD.
The impact of this is that one of the brokers will never get any real load on it.
After changing to a namespace with 32 bundles, the load was evenly distributed. This is the reason why I think that `defaultNumberOfNamespaceBundles=4` is not a good default.
The bundle splitting has an open bug #5510 which is another reason to use a higher number of default namespace bundles.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari edited a comment on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925285825
> user can always enable auto bundle unloading in loadbalancer. and default-bundle =4 is not defined for test cluster with 1 namespace but it considers a normal cluster that is serving multiple namespaces with considerable traffic. again, higher number of bundles comes with cost specially for znodes and default behavior unnecessarily will add more bundles for users who are using default conf file.
I have a test cluster on GCP/GKE where the auto bundle unloading never happens. It's using the default settings which contains
```
loadBalancerBrokerOverloadedThresholdPercentage=85
loadBalancerAutoUnloadSplitBundlesEnabled=true
loadBalancerNamespaceBundleMaxSessions=1000
```
The test load is created with this type of pulsar-perf command (using 2.7.3 pulsar-perf since current [2.8.0 contains a bug that is fixed in master](https://github.com/apache/pulsar/pull/11824), the bug causes partitions to stop consuming in pulsar-perf).
```
# create a 100 partition topic
pulsar-admin topics create-partitioned-topic -p 100 persistent://public/default/perftest
# produce messages
PULSAR_EXTRA_OPTS="-Xms1500M -Xmx1500M -XX:MaxDirectMemorySize=768M" /pulsar/bin/pulsar-perf produce -u pulsar+ssl://pulsar-broker:6651/ -ioThreads 4 -s 512 -r 50000 persistent://public/default/perftest
# consume messages
PULSAR_EXTRA_OPTS="-Xms1500M -Xmx1500M -XX:MaxDirectMemorySize=768M" /pulsar/bin/pulsar-perf consume -u pulsar+ssl://pulsar-broker:6651/ -s sub persistent://public/default/perftest
```
(these are run on a pod which has 2CPU + 4GB RAM requested/limits, using pulsar-perf from Pulsar 2.7.3 docker image)
The traffic can go on for hours and the CPU load will be very low. Since the threshold never gets exceeded, the auto unload and splitting of the bundle never happens.
In this case, the GKE cluster has a nodepool with 6 n2-standard-32 VMs (32 cores, 128GB RAM). Storage for bookies in on SSD.
The impact of this is that one of the brokers will never get any real load on it.
After changing to a namespace with 32 bundles, the load was evenly distributed. This is the reason why I think that `defaultNumberOfNamespaceBundles=4` is not a good default.
The bundle splitting has an open bug #5510 which is another reason to use a higher number of default namespace bundles.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925682501
> I agree that 16 or 32 are better defaults that would avoid many surprises out of the box. That has been the config I have set in prod for the past many years.
@merlimat Is 32 fine as a default, or should I change it to 16 in this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari closed pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari closed pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari edited a comment on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925285825
> user can always enable auto bundle unloading in loadbalancer. and default-bundle =4 is not defined for test cluster with 1 namespace but it considers a normal cluster that is serving multiple namespaces with considerable traffic. again, higher number of bundles comes with cost specially for znodes and default behavior unnecessarily will add more bundles for users who are using default conf file.
I have a test cluster on GCP/GKE where the auto bundle unloading never happens. It's using the default settings which contains
```
loadBalancerBrokerOverloadedThresholdPercentage=85
loadBalancerAutoUnloadSplitBundlesEnabled=true
loadBalancerNamespaceBundleMaxSessions=1000
```
The test load is created with this type of pulsar-perf command (using 2.7.3 pulsar-perf since current [2.8.0 contains a bug that is fixed in master](https://github.com/apache/pulsar/pull/11824), the bug causes partitions to stop consuming in pulsar-perf).
```
# create a 100 partition topic
pulsar-admin topics create-partitioned-topic -p 100 persistent://public/default/perftest
# produce messages
PULSAR_EXTRA_OPTS="-Xms1500M -Xmx1500M -XX:MaxDirectMemorySize=768M" /pulsar/bin/pulsar-perf produce -u pulsar+ssl://pulsar-broker:6651/ -ioThreads 4 -s 512 -r 50000 persistent://public/default/perftest
# consume messages
PULSAR_EXTRA_OPTS="-Xms1500M -Xmx1500M -XX:MaxDirectMemorySize=768M" /pulsar/bin/pulsar-perf consume -u pulsar+ssl://pulsar-broker:6651/ -s sub persistent://public/default/perftest
```
(these are run on a pod which has 2CPU + 4GB RAM requested/limits, using pulsar-perf from Pulsar 2.7.3 docker image)
The traffic can go on for hours and the CPU load will be very low. Since the threshold never gets exceeded, the auto unload and splitting of the bundle never happens.
In this case, the GKE cluster has a nodepool with 6 n2-standard-32 VMs (32 cores, 128GB RAM). Storage for bookies in on SSD.
The impact of this is that one of the brokers will never get any real load on it.
After changing to a namespace with 32 bundles, the load was evenly distributed. This is the reason why I think that `defaultNumberOfNamespaceBundles=4` is not a good default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925695116
I now realised that public/default gets always created with 16 bundles, regardless of what the configuration of defaultNumberOfNamespaceBundles is.
Bundles is hard coded to 16 when public/default gets created in cluster metadata setup:
https://github.com/apache/pulsar/blob/9905a9debd82ccb2f408faf32db2858166a4fff7/pulsar-broker/src/main/java/org/apache/pulsar/PulsarClusterMetadataSetup.java#L304
this gets called from this location:
https://github.com/apache/pulsar/blob/9905a9debd82ccb2f408faf32db2858166a4fff7/pulsar-broker/src/main/java/org/apache/pulsar/PulsarClusterMetadataSetup.java#L265-L266
The root cause of the problem where one of the servers was idling must have been caused by something else than the default bundles of 4 since public/default has 16 by default. I now checked with `pulsar-admin namespaces bundles public/default` and there's 44 for public/default in this perf testing environment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
merlimat commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925302604
I agree that 16 or 32 are better defaults that would avoid many surprises out of the box. That has been the config I have set in prod for the past many years.
The case of having a huge namespaces with limited load on each is far less common and it can always be adjusted in the config, where it poses a problem.
The default should be a good setting for ~80% of users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-926818757
Note also that the `LoadManager` implementation (and in the case the `LoadManager` is the `ModularLoadManagerWrapper`, the `LoadSheddingStrategy` implementation) will affect bundle distribution across brokers. In some implementations, a broker's bundles will not be unloaded if it is not overloaded, even though there might be "underloaded" brokers in the cluster. (I've already discussed this with @lhotari in separate channels. Posting here for any other readers that might want come across this PR.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] rdhabalia commented on pull request #12137: [Broker] Change the default number of namespace bundles to 32
Posted by GitBox <gi...@apache.org>.
rdhabalia commented on pull request #12137:
URL: https://github.com/apache/pulsar/pull/12137#issuecomment-925260150
user can always enable auto bundle unloading in loadbalancer. and default-bundle =4 is not defined for test cluster with 1 namespace but it considers a normal cluster that is serving multiple namespaces with considerable traffic. again, higher number of bundles comes with cost specially for znodes and default behavior unnecessarily will add more bundles for users who are using default conf file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org