You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "wchevreuil (via GitHub)" <gi...@apache.org> on 2026/05/11 20:26:21 UTC
[PR] HBASE-30135: Improve CacheAwareLoadBalancer to simulate low cache ratio regions as cached in candidate servers with enough cache space [hbase]
wchevreuil opened a new pull request, #8221:
URL: https://github.com/apache/hbase/pull/8221
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-30135: Improve CacheAwareLoadBalancer to simulate low cache ratio regions as cached in candidate servers with enough cache space [hbase]
Posted by "petersomogyi (via GitHub)" <gi...@apache.org>.
petersomogyi commented on code in PR #8221:
URL: https://github.com/apache/hbase/pull/8221#discussion_r3252758867
##########
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/CacheAwareLoadBalancer.java:
##########
@@ -88,10 +90,11 @@ public class CacheAwareLoadBalancer extends StochasticLoadBalancer {
private Long sleepTime;
private Configuration configuration;
- public enum GeneratorFunctionType {
- LOAD,
- CACHE_RATIO
- }
+ private float lowCacheRatioThreshold;
+ private float potentialCacheRatioAfterMove;
+ private float minFreeCacheSpaceFactor;
+
+ private BigDecimal simulatedRatio = new BigDecimal(0);
Review Comment:
Thanks for the explanation!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-30135: Improve CacheAwareLoadBalancer to simulate low cache ratio regions as cached in candidate servers with enough cache space [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil merged PR #8221:
URL: https://github.com/apache/hbase/pull/8221
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-30135: Improve CacheAwareLoadBalancer to simulate low cache ratio regions as cached in candidate servers with enough cache space [hbase]
Posted by "petersomogyi (via GitHub)" <gi...@apache.org>.
petersomogyi commented on code in PR #8221:
URL: https://github.com/apache/hbase/pull/8221#discussion_r3248576963
##########
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/CacheAwareLoadBalancer.java:
##########
@@ -88,10 +90,11 @@ public class CacheAwareLoadBalancer extends StochasticLoadBalancer {
private Long sleepTime;
private Configuration configuration;
- public enum GeneratorFunctionType {
- LOAD,
- CACHE_RATIO
- }
+ private float lowCacheRatioThreshold;
+ private float potentialCacheRatioAfterMove;
+ private float minFreeCacheSpaceFactor;
+
+ private BigDecimal simulatedRatio = new BigDecimal(0);
Review Comment:
In other places you used `BigDecimal.ZERO` for initialization.
##########
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/CacheAwareLoadBalancer.java:
##########
@@ -88,10 +90,11 @@ public class CacheAwareLoadBalancer extends StochasticLoadBalancer {
private Long sleepTime;
private Configuration configuration;
- public enum GeneratorFunctionType {
- LOAD,
- CACHE_RATIO
- }
+ private float lowCacheRatioThreshold;
+ private float potentialCacheRatioAfterMove;
+ private float minFreeCacheSpaceFactor;
+
+ private BigDecimal simulatedRatio = new BigDecimal(0);
Review Comment:
This `simulatedRatio` field is accessed from multiple places. Can it cause a race issue?
* CacheAwareCandidateGenerator.generate()
* CacheAwareSkewnessCandidateGenerator.pickRandomRegions()
* CacheAwareCostFunction.regionMoved()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-30135: Improve CacheAwareLoadBalancer to simulate low cache ratio regions as cached in candidate servers with enough cache space [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil commented on code in PR #8221:
URL: https://github.com/apache/hbase/pull/8221#discussion_r3248930878
##########
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/CacheAwareLoadBalancer.java:
##########
@@ -88,10 +90,11 @@ public class CacheAwareLoadBalancer extends StochasticLoadBalancer {
private Long sleepTime;
private Configuration configuration;
- public enum GeneratorFunctionType {
- LOAD,
- CACHE_RATIO
- }
+ private float lowCacheRatioThreshold;
+ private float potentialCacheRatioAfterMove;
+ private float minFreeCacheSpaceFactor;
+
+ private BigDecimal simulatedRatio = new BigDecimal(0);
Review Comment:
It's not thread safe for sure, but as of now, balancer is a single thread only operation (no multiple parallel balancers run at the same time).
Unfortunately, we need to keep this global variable in order to rollback the cache ratio increment if the plan isn't accepted.
In StochasticLoadBalancer.balanceTable we do:
1) Pick a generator and cost function at random;
2) Call generate() on the generator so that we set that variable to 0 and get a move plan;
3) Simulate the plan, which calls regionMoved(). If the chosen cost function was the CacheAwareCostFunction, we will calculate the impact of that plan on the overall cache ratio. This impact is what we save simulatedRatio field;
4) Calculate the total cost with this plan. If it's lower than the lowest cost so far, keep it. Otherwise, roll it back by using the same cost function but inverting the to/from servers order. If it's the CacheAwareCostFunction, regionMoved would be called and it would enter the branch that reverts the value from the cacheRatio.
5) Iterate again. Reset the variable on step 2;
Without this, we would need to always recalculate all individual regions/servers ratios on every new plan simulation (every iteration of the StochasticLoadBalancer loop), which is costly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org