You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/06/02 15:53:43 UTC
[GitHub] [pulsar] frankjkelly opened a new issue, #15905: [Doc] Overload Shedder documentation - need to add documentation on the individual weights
frankjkelly opened a new issue, #15905:
URL: https://github.com/apache/pulsar/issues/15905
### What issue do you find in Pulsar docs?
In https://pulsar.apache.org/docs/next/administration-load-balance#thresholdshedder
1) LocalBrokerData#getMaxResourceUsageWithWeight should be a link or in different font
2) "The weights for each resource are configurable." - it would be great if the weights were listed or perhaps a link to the relevant section of the reference configuration here https://pulsar.apache.org/docs/reference-configuration/#broker
3) It would be nice to know when to prioritize one weight over another - just some general guidelines
4) "If a broker's usage is greater than the average usage per broker plus the loadBalancerBrokerThresholdShedderPercentage"
Is that percentage an absolute percentage?
e.g. say the % usage is 5% / 1% / 1% for three brokers - comparing 5 vs 1 is a 5x difference or 500% comparing on a relative basis but a 4% on an absolute basis. Can it be clarified which percentage that is? Perhaps an example would help?
### What is your suggestion?
See above
### Do you have any references?
No
### Would you like to fix this issue?
Sorry - some of it requires more detailed clarification from someone who knows the load balancer algos in detail.
### Note
- [X] I have researched my question.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly commented on issue #15905: [Doc] Threshold Shedder documentation - need to add documentation on the individual weights
Posted by GitBox <gi...@apache.org>.
frankjkelly commented on issue #15905:
URL: https://github.com/apache/pulsar/issues/15905#issuecomment-1158999729
@Demogorgon314 thanks the example is super helpful and perhaps could be added to the documentation?
As regards the weights - isn't the purpose of the documentation so that I don't have to read the code / configuration files?
Also what are the range of values to set to weights? 0 to 1.0, 1.0 to 100.0?
I also realized another question - when I read the code I was surprised to see when the resource weighting is employed here
https://github.com/apache/pulsar/blob/a812f29bab93a5e841a9b3c3190fca343fab6e71/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ThresholdShedder.java#L160-L164
The the `max` of the `resource * resourceWeight` is used to determine overload
https://github.com/apache/pulsar/blob/a2559923877b005e4eec797214d1bcd9fb378df7/pulsar-common/src/main/java/org/apache/pulsar/policies/data/loadbalancer/LocalBrokerData.java#L253-L255
Usually when I see a "weight" it is not a winner-take-all approach but some scoring that takes each resource & weight into account. In addition the calculation results are not normalized on a scale of 0 to 1 or 0 to 100 - the problem without normalizing
For example I could set loadBalancerCPUResourceWeight to 1000.0 and then the threshold would have to be scaled appropriately.
I see a divide by `100` here https://github.com/apache/pulsar/blob/a2559923877b005e4eec797214d1bcd9fb378df7/pulsar-common/src/main/java/org/apache/pulsar/policies/data/loadbalancer/LocalBrokerData.java#L253-L255
but without bounds on the weights that's not going to normalize things.
So even when I look at the config files I still need to read the code to figure out how those weights will be used.
Anyway I am excited to use this feature and appreciate your insight and help.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet closed issue #15905: [Doc] Threshold Shedder documentation - need to add documentation on the individual weights
Posted by GitBox <gi...@apache.org>.
Anonymitaet closed issue #15905: [Doc] Threshold Shedder documentation - need to add documentation on the individual weights
URL: https://github.com/apache/pulsar/issues/15905
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Demogorgon314 commented on issue #15905: [Doc] Threshold Shedder documentation - need to add documentation on the individual weights
Posted by GitBox <gi...@apache.org>.
Demogorgon314 commented on issue #15905:
URL: https://github.com/apache/pulsar/issues/15905#issuecomment-1162716885
> Also what are the range of values to set to weights? 0 to 1.0, 1.0 to 100.0?
The range is 0 to 1.0. I agree with you, and it is a little confusing for users. We might need to explain in the doc.
> I could set loadBalancerCPUResourceWeight to 1000.0 and then the threshold would have to be scaled appropriately.I see a divide by `100` here
That is a good catch! We should limit the max weight to 1.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] frankjkelly commented on issue #15905: [Doc] Threshold Shedder documentation - need to add documentation on the individual weights
Posted by GitBox <gi...@apache.org>.
frankjkelly commented on issue #15905:
URL: https://github.com/apache/pulsar/issues/15905#issuecomment-1181929088
Awesome - thanks so much @Anonymitaet !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on issue #15905: [Doc] Threshold Shedder documentation - need to add documentation on the individual weights
Posted by GitBox <gi...@apache.org>.
momo-jun commented on issue #15905:
URL: https://github.com/apache/pulsar/issues/15905#issuecomment-1162752667
@hangc0276, are there any general guidelines for when to prioritize one weight over another?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Demogorgon314 commented on issue #15905: [Doc] Threshold Shedder documentation - need to add documentation on the individual weights
Posted by GitBox <gi...@apache.org>.
Demogorgon314 commented on issue #15905:
URL: https://github.com/apache/pulsar/issues/15905#issuecomment-1158581682
> "The weights for each resource are configurable." - it would be great if the weights were listed or perhaps a link to the relevant section of the reference configuration here https://pulsar.apache.org/docs/reference-configuration/#broker
You can see these configurations.
```
# The BandWithIn usage weight when calculating new resource usage.
# It only takes effect in the ThresholdShedder strategy.
loadBalancerBandwithInResourceWeight=1.0
# The BandWithOut usage weight when calculating new resource usage.
# It only takes effect in the ThresholdShedder strategy.
loadBalancerBandwithOutResourceWeight=1.0
# The CPU usage weight when calculating new resource usage.
# It only takes effect in the ThresholdShedder strategy.
loadBalancerCPUResourceWeight=1.0
# The heap memory usage weight when calculating new resource usage.
# It only takes effect in the ThresholdShedder strategy.
loadBalancerMemoryResourceWeight=1.0
# The direct memory usage weight when calculating new resource usage.
# It only takes effect in the ThresholdShedder strategy.
loadBalancerDirectMemoryResourceWeight=1.0
```
> "If a broker's usage is greater than the average usage per broker plus the loadBalancerBrokerThresholdShedderPercentage"
Is that percentage an absolute percentage?
It should be an absolute percentage.
Example:
Assume we have three brokers, the broker1 avg broker usage is 40%, the broker2 avg broker usage is 10%, the broker3 avg broker usage is 10%, then the cluster avg usage should be ((40% + 10% + 10%) / 3) = 20%.
If we set `loadBalancerBrokerThresholdShedderPercentage=1`, then the broker1's some bundles will get unloaded, because broker1 broker avg usage is greater than cluster avg usage plus loadBalancerBrokerThresholdShedderPercentage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on issue #15905: [Doc] Threshold Shedder documentation - need to add documentation on the individual weights
Posted by GitBox <gi...@apache.org>.
momo-jun commented on issue #15905:
URL: https://github.com/apache/pulsar/issues/15905#issuecomment-1158415729
@Demogorgon314 can you take a look at this issue? Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org