[jira] [Commented] (KUDU-886) Cluster load balancing

["Todd Lipcon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KUDU-886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437163#comment-15437163 ] 

Todd Lipcon commented on KUDU-886:
----------------------------------

One interesting metric for write-heavy workloads is "apply pool utilization": sum up the total task time of apply tasks, divide by the max number of threads in the pool. Once this reaches 100%, the apply pool starts queueing and write latencies start heading sky high on that node if it's a leader. Alternatively we can just use the queue time/length itself, but that's kind of a lagging indicator, whereas load percentage is a leading one that may allow us to balance before the problem gets out of hand.

Note that shifting leaders off of a host with high apply pool utilization won't resolve the utilization issue, but it will at least reduce user-facing latency (so long as the other two nodes hosting that tablet dont have high utilization)

> Cluster load balancing
> ----------------------
>
>                 Key: KUDU-886
>                 URL: https://issues.apache.org/jira/browse/KUDU-886
>             Project: Kudu
>          Issue Type: New Feature
>          Components: master
>    Affects Versions: GA
>            Reporter: Todd Lipcon
>
> We should add some load balancing support for GA:
> - move leaders to evenly spread RPC load.
> - eventually move tablets to even out disk space or load.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)