[jira] [Commented] (HBASE-27496) Optionally limit the amount of plans executed in the Normalizer

["Hudson (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-27496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637568#comment-17637568 ] 

Hudson commented on HBASE-27496:
--------------------------------

Results for branch branch-2
	[build #686 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/686/]: (/) *{color:green}+1 overall{color}*
----
details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/686/General_20Nightly_20Build_20Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/686/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/686/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/686/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Optionally limit the amount of plans executed in the Normalizer
> ---------------------------------------------------------------
>
>                 Key: HBASE-27496
>                 URL: https://issues.apache.org/jira/browse/HBASE-27496
>             Project: HBase
>          Issue Type: Improvement
>          Components: Normalizer
>            Reporter: Charles Connell
>            Priority: Minor
>             Fix For: 3.0.0-alpha-4, 2.5.3
>
>
> My company (Hubspot) is starting to use {{{}SimpleRegionNormalizer{}}}. We turn the normalizer switch on for 30 minutes each day, when our database traffic is at a low point. We're using the {{hbase.normalizer.throughput.max_bytes_per_sec}} setting to create a rate limit. I've found that while the {{SimpleRegionNormalizer}} only produces new plans for 30 minutes each day, the plans often take many hours to execute. This leds to region splits, merges, and moves occurring in our HBase clusters during hours we'd prefer them not to.
> I propose a new setting {{{}hbase.normalizer.plans_size_limit.mb{}}}.
> This will allow HBase administrators to limit the number of plans processed by a run of {{{}RegionNormalizerWorker{}}}, by forcing it to stop processing new plans once the cumulative region size limits are exceeded. This will give you a way to limit approximately how long it takes to execute the plans. Because the current limit to execute plans is primarily determined by a per-byte rate limit, I propose that the new settings also work on a similar basis. This will make it feasible to reason about how your rate limit and your size limits interact.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)