[jira] [Commented] (HBASE-30300) CacheAwareLoadBalancer disproportionally favouring cache ratio over skewness

["Hudson (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-30300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18101287#comment-18101287 ] 

Hudson commented on HBASE-30300:
--------------------------------

Results for branch branch-2
	[build #1462 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1462/]: (x) *{color:red}-1 overall{color}*
----
details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1462/General_20Nightly_20Build_20Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1462/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1462/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1462/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1462/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]


> CacheAwareLoadBalancer disproportionally favouring cache ratio over skewness
> ----------------------------------------------------------------------------
>
>                 Key: HBASE-30300
>                 URL: https://issues.apache.org/jira/browse/HBASE-30300
>             Project: HBase
>          Issue Type: Bug
>          Components: Balancer
>    Affects Versions: 3.0.0, 2.7.0, 2.6.6
>            Reporter: Wellington Chevreuil
>            Assignee: Wellington Chevreuil
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0-alpha-1, 2.7.0, 2.6.7, 3.0.1
>
>
> There’s a problem in the current CacheAwareLoadBalancer implementation where it tends to disproportionally favour moves based on cache ratio, over skewness.
> This happens even when cache usage is at capacity, and moving low cached regions has close to no benefits (because the moved region wouldn't be cached anyways, due to no cache space available).
> Scanning the current CacheAwareLoadBalancer code with Claude Opus 4.6 identified the following issues behind this unexpected behaviour:
> 1 ) Generator weight mismatch in CacheAwareRegionSkewnessCostFunction and CacheAwareCostFunction updateWeight(): Was writing to LoadCandidateGenerator.class instead of the related implementations, causing the generators to be picked randomly.
> 2) Skewness generator wasting iterations on old-server map: CacheAwareSkewnessCandidateGenerator.pickRandomRegions() was draining regionCacheRatioOnOldServerMap before doing skew-based moves. This could worsen skew by moving regions to already-overloaded servers. 
> 3) Throttle blocking skew moves for low-cache regions: throttle() is adding sleep delays for regions that had near-zero cache ratio on their source server (nothing to lose by moving quickly). Could be skipped when cacheRatioOnSource < lowCacheRatioThreshold.
> 4) moveRegionToOldServer() disrupting well-cached regions: if a region is already well-cached where it is, don't move it back to the "old" server based on potentially stale historical data.
> 5) Prefetch overhead not subtracted from reported free space: getServerBlockCacheFreeBytes() doesn't take into account acceptableFactor, when calculating cache space available on potential destination servers. This can cause moves that would then fail to prefetch in the target server.
> 6) Cumulative free space tracking across simulated moves: during plan simulation, free space tracking is not updated correctly, and the the solver could stack multiple regions onto one server without realizing it was filling up.
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)