You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/02/02 21:16:32 UTC

[GitHub] [arrow-datafusion] alamb commented on a change in pull request #1730: optimize hash join by reducing critical zone

alamb commented on a change in pull request #1730:
URL: https://github.com/apache/arrow-datafusion/pull/1730#discussion_r798018435



##########
File path: datafusion/src/physical_plan/hash_join.rs
##########
@@ -333,8 +334,10 @@ impl ExecutionPlan for HashJoinExec {
                                 concat_batches(&self.left.schema(), &batches, num_rows)?;
 
                             let left_side = Arc::new((hashmap, single_batch));
-
-                            *build_side = Some(left_side.clone());
+                            {

Review comment:
       Looking at the code, it seems like by moving the acquisition of the lock down, here multiple partitions could potentially race to build the `build_side` hash map. 
   
   So I don't fully understand how this change is increasing performance. I need to think about this more 🤔 
   
   cc @Dandandan 
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org