You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ray Mattingly (Jira)" <ji...@apache.org> on 2024/12/13 23:23:00 UTC
[jira] [Updated] (HBASE-28513) Secondary replica balancing squashes all other cost considerations
[ https://issues.apache.org/jira/browse/HBASE-28513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ray Mattingly updated HBASE-28513:
----------------------------------
Description:
I have a larger write up available [here|https://docs.google.com/document/d/1jA8Ghs86v7b-53j5DcsdbPnOXxbHjewkIBFi1E4S1pY/edit?usp=sharing].
Secondary replica balancing squashes all other cost considerations.
Basically there are a few cost functions with relatively huge default multipliers. For example `PrimaryRegionCountSkewCostFunction` has a default multiplier of 100,000. Meanwhile things like StoreFileCostFunction have a multiplier of 5. Having any multiplier of 100k, while others are single digit, basically makes the latter category totally irrelevant from balancer considerations.
I understand that it's critical to distribute a region's replicas across multiple hosts/racks, but I don't think we should do this at the expense of all other balancer considerations.
For example, maybe we could have two types of balancer considerations: costs (as we do now), and conditionals (for the more discrete considerations, like ">1 replica of the same region should not exist on a single host"). This would allow us to prioritize replica distribution _and_ maintain consideration for things like storefile balance.
was:
I have a larger write up available [here|https://gist.github.com/rmdmattingly/9db0885f9977234e56325beff566e749].
Basically there are a few cost functions with relatively huge default multipliers. For example `PrimaryRegionCountSkewCostFunction` has a default multiplier of 100,000. Meanwhile things like StoreFileCostFunction have a multiplier of 5. Having any multiplier of 100k, while others are single digit, basically makes the latter category totally irrelevant from balancer considerations.
I understand that it's critical to distribute a region's replicas across multiple hosts/racks, but I don't think we should do this at the expense of all other balancer considerations.
For example, maybe we could have two types of balancer considerations: costs (as we do now), and conditionals (for the more discrete considerations, like ">1 replica of the same region should not exist on a single host"). This would allow us to prioritize replica distribution _and_ maintain consideration for things like storefile balance.
> Secondary replica balancing squashes all other cost considerations
> ------------------------------------------------------------------
>
> Key: HBASE-28513
> URL: https://issues.apache.org/jira/browse/HBASE-28513
> Project: HBase
> Issue Type: Improvement
> Reporter: Ray Mattingly
> Priority: Major
>
> I have a larger write up available [here|https://docs.google.com/document/d/1jA8Ghs86v7b-53j5DcsdbPnOXxbHjewkIBFi1E4S1pY/edit?usp=sharing].
> Secondary replica balancing squashes all other cost considerations.
> Basically there are a few cost functions with relatively huge default multipliers. For example `PrimaryRegionCountSkewCostFunction` has a default multiplier of 100,000. Meanwhile things like StoreFileCostFunction have a multiplier of 5. Having any multiplier of 100k, while others are single digit, basically makes the latter category totally irrelevant from balancer considerations.
> I understand that it's critical to distribute a region's replicas across multiple hosts/racks, but I don't think we should do this at the expense of all other balancer considerations.
> For example, maybe we could have two types of balancer considerations: costs (as we do now), and conditionals (for the more discrete considerations, like ">1 replica of the same region should not exist on a single host"). This would allow us to prioritize replica distribution _and_ maintain consideration for things like storefile balance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)