[GitHub] [hbase] saintstack commented on a change in pull request #3532: HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned

[GitBox <gi...@apache.org>]

saintstack commented on a change in pull request #3532:
URL: https://github.com/apache/hbase/pull/3532#discussion_r685673437



##########
File path: hbase-protocol/src/main/protobuf/Client.proto
##########
@@ -91,6 +91,8 @@ message Get {
   optional Consistency consistency = 12 [default = STRONG];
   repeated ColumnFamilyTimeRange cf_time_range = 13;
   optional bool load_column_families_on_demand = 14; /* DO NOT add defaults to load_column_families_on_demand. */
+
+  optional uint64 max_result_size = 15;

Review comment:
       FYI, these protos we want to let go of eventually. They are for use of downstreamers, not for internal hbase use.... internally we use the shaded stuff.  So, its fine adding this here but probably better not adding it.

##########
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java
##########
@@ -339,6 +340,21 @@ public Get setFilter(Filter filter) {
     return this;
   }
 
+  /**
+   * Set the maximum result size. The default is -1; this means that no specific
+   * maximum result size will be set for this Get.
+   *
+   * If set to a value greater than zero, the server may respond with a Result where
+   * {@link Result#mayHaveMoreCellsInRow()} is true. The user is required to handle
+   * this case.

Review comment:
       What do you do when mayHaveMoreCellsInRow is true @bbeaudreault  ? How do you use this boolean in prod (if you don't mind me asking...)

##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##########
@@ -7518,11 +7516,35 @@ void prepareGet(final Get get) throws IOException {
 
   @Override
   public List<Cell> get(Get get, boolean withCoprocessor) throws IOException {
-    return get(get, withCoprocessor, HConstants.NO_NONCE, HConstants.NO_NONCE);
+    return getInternal(get, withCoprocessor, HConstants.NO_NONCE, HConstants.NO_NONCE).getFirst();
   }
 
-  private List<Cell> get(Get get, boolean withCoprocessor, long nonceGroup, long nonce)
-      throws IOException {
+  private Result get(Get get, boolean withCoprocessor, long nonceGroup, long nonce)
+    throws IOException {
+    Pair<List<Cell>, ScannerContext> result = getInternal(get, withCoprocessor, nonceGroup, nonce);
+    boolean stale = this.getRegionInfo().getReplicaId() != 0;
+
+    return Result.create(
+      result.getFirst(),
+      get.isCheckExistenceOnly() ? !result.getFirst().isEmpty() : null,
+      stale,
+      result.getSecond().mayHaveMoreCellsInRow());
+  }
+
+  private Pair<List<Cell>, ScannerContext> getInternal(Get get, boolean withCoprocessor, long nonceGroup, long nonce)
+    throws IOException {
+    ScannerContext scannerContext = ScannerContext.newBuilder()

Review comment:
       Every Get will now carry a ScannerContext where previous it was usually null? (IIRC)  Do you think this will cost Bryan? Will there be  other benefits having a ScannerContext on every Get?

##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##########
@@ -3864,8 +3865,7 @@ public void prepareMiniBatchOperations(MiniBatchOperationInProgress<Mutation> mi
             Result result;
             if (returnResults) {
               // convert duplicate increment/append to get
-              List<Cell> results = region.get(toGet(mutation), false, nonceGroup, nonce);
-              result = Result.create(results);
+              result = region.get(toGet(mutation), false, nonceGroup, nonce);

Review comment:
       I'm missing something here.... region.get returns  List<Cell> ? Not a Result?

##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##########
@@ -3864,8 +3865,7 @@ public void prepareMiniBatchOperations(MiniBatchOperationInProgress<Mutation> mi
             Result result;
             if (returnResults) {
               // convert duplicate increment/append to get
-              List<Cell> results = region.get(toGet(mutation), false, nonceGroup, nonce);
-              result = Result.create(results);
+              result = region.get(toGet(mutation), false, nonceGroup, nonce);

Review comment:
       nvm.. I see.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org