[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #5514: [BUG] fix memory limit failure and optimize memory usage in join stage

[GitBox <gi...@apache.org>]

EmmyMiao87 commented on a change in pull request #5514:
URL: https://github.com/apache/incubator-doris/pull/5514#discussion_r594982587



##########
File path: be/src/exec/hash_join_node_ir.cpp
##########
@@ -118,7 +119,10 @@ int HashJoinNode::process_probe_batch(RowBatch* out_batch, RowBatch* probe_batch
             if (UNLIKELY(_probe_batch_pos == probe_rows)) {
                 goto end;
             }
-
+            if (++_probe_counter % _release_context_counter == 0) {
+                ExprContext::free_local_allocations(_probe_expr_ctxs);
+                ExprContext::free_local_allocations(_build_expr_ctxs);

Review comment:
       Is it okay to release the build memory directly here?

##########
File path: be/src/exec/hash_join_node.h
##########
@@ -174,6 +175,13 @@ class HashJoinNode : public ExecNode {
     // This is only used for debugging and outputting the left child rows before
     // doing the join.
     std::string get_probe_row_output_string(TupleRow* probe_row);
+
+    // _release_context_counter should be power of 2
+    // GCC will optimize the modulo operation to &(release_context_counter - 1)

Review comment:
       ```suggestion
       // GCC will optimize the module operation to &(release_context_counter - 1)
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org