Re: [PR] HBASE-27687: support consumption of `block bytes scanned` in operation quota [hbase]

["rmdmattingly (via GitHub)" <gi...@apache.org>]

rmdmattingly commented on code in PR #5654:
URL: https://github.com/apache/hbase/pull/5654#discussion_r1475316978


##########
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/DefaultOperationQuota.java:
##########
@@ -132,16 +146,28 @@ public void addMutation(final Mutation mutation) {
     operationSize[OperationType.MUTATE.ordinal()] += QuotaUtil.calculateMutationSize(mutation);
   }
 
+  @Override
+  public void addBlockBytesScanned(long blockBytesScanned) {
+    this.blockBytesScanned += blockBytesScanned;
+  }
+
   /**
    * Update estimate quota(read/write size/capacityUnits) which will be consumed
    * @param numWrites the number of write requests
    * @param numReads  the number of read requests
    * @param numScans  the number of scan requests
    */
   protected void updateEstimateConsumeQuota(int numWrites, int numReads, int numScans) {
-    writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);
-    readConsumed = estimateConsume(OperationType.GET, numReads, 100);
-    readConsumed += estimateConsume(OperationType.SCAN, numScans, 1000);
+    if (useBlockBytesScanned) {
+      writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);
+      // assume 1 block required for reads. this is probably a low estimate, which is okay
+      readConsumed = numReads > 0 ? blockSizeBytes : 0;
+      readConsumed += numScans > 0 ? blockSizeBytes : 0;

Review Comment:
   After the testing mentioned in https://github.com/apache/hbase/pull/5654#issue-2101355167, I feel more confident that these estimates are adequate. We verified that both multiget and filtered scan workloads get throttled as one would expect.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org