[GitHub] [druid] jihoonson commented on a change in pull request #10685: Subtotal queries give incorrect results if query has a limit spec

[GitBox <gi...@apache.org>]

jihoonson commented on a change in pull request #10685:
URL: https://github.com/apache/druid/pull/10685#discussion_r551659244



##########
File path: processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/ByteBufferMinMaxOffsetHeap.java
##########
@@ -59,6 +60,35 @@ public ByteBufferMinMaxOffsetHeap(
     this.heapIndexUpdater = heapIndexUpdater;
   }
 
+  public ByteBufferMinMaxOffsetHeap copy()
+  {
+    LimitedBufferHashGrouper.BufferGrouperOffsetHeapIndexUpdater updater =
+        Optional
+            .ofNullable(heapIndexUpdater)
+            .map(LimitedBufferHashGrouper.BufferGrouperOffsetHeapIndexUpdater::copy)
+            .orElse(null);
+
+    // deep copy buf
+    ByteBuffer buffer = ByteBuffer.allocateDirect(buf.capacity());

Review comment:
       GroupBy queries use both processing buffers and merge buffers. The former is used when you compute per-segment results while the later is for all other purposes (merging per-segment results, computing subtotals, etc). Especially the broker only uses the merge buffers to process subtotals and subqueries. 
   
   The merge buffers are maintained in a `BlockingPool` to manage the memory usage in brokers and historicals. This is important to avoid query failures due to OOM errors. Here, you should not allocate memory directly, but get one from the merge buffer pool. Check out [how the merge buffer is currently acquired](https://github.com/apache/druid/blob/master/processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/GroupByRowProcessor.java#L117-L126). 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org