[jira] [Commented] (HBASE-28385) Quota estimates are too optimistic for large scans

["Hudson (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-28385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17827115#comment-17827115 ] 

Hudson commented on HBASE-28385:
--------------------------------

Results for branch branch-2.6
	[build #76 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/76/]: (x) *{color:red}-1 overall{color}*
----
details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/76/General_20Nightly_20Build_20Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/76/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/75/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/76//console].


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Quota estimates are too optimistic for large scans
> --------------------------------------------------
>
>                 Key: HBASE-28385
>                 URL: https://issues.apache.org/jira/browse/HBASE-28385
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Ray Mattingly
>            Assignee: Ray Mattingly
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 2.6.0, 3.0.0-beta-2
>
>
> Let's say you're running a table scan with a throttle of 100MB/sec per RegionServer. Ideally your scans are going to pull down large results, often containing hundreds or thousands of blocks.
> You will estimate each scan as costing a single block of read capacity, and if your quota is already exhausted then the server will evaluate the backoff required for your estimated consumption (1 block) to be available. This will often be ~1ms, causing your retries to basically be immediate.
> Obviously it will routinely take much longer than 1ms for 100MB of IO to become available in the given configuration, so your retries will be destined to fail. At worst this can cause a saturation of your server's RPC layer, and at best this causes erroneous exhaustion of the client's retries.
> We should find a way to make these estimates a bit smarter for large scans.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)