[GitHub] Ben-Zvi commented on a change in pull request #1409: DRILL-6644: Don't reserve space for incoming probe batches unnecessarily during the build phase.

[GitBox <gi...@apache.org>]

Ben-Zvi commented on a change in pull request #1409: DRILL-6644: Don't reserve space for incoming probe batches unnecessarily during the build phase.
URL: https://github.com/apache/drill/pull/1409#discussion_r206370742
 
 

 ##########
 File path: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinMemoryCalculatorImpl.java
 ##########
 @@ -391,7 +391,7 @@ private void calculateMemoryUsage()
         // probe batch we sniffed.
         // TODO when batch sizing project is complete we won't have to sniff probe batches since
         // they will have a well defined size.
-        reservedMemory = incompletePartitionsBatchSizes + maxBuildBatchSize + maxProbeBatchSize;
+        reservedMemory = incompletePartitionsBatchSizes + maxBuildBatchSize + probeSizePredictor.getBatchSize();
 
 Review comment:
   While experimenting, I noticed that the size of the incoming probe-side batch is **not** charged to the hash-Join allocator's available memory (basically I set the available memory to zero before the next() call, and never got an OOM). So is this addition needed ?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services