[GitHub] [arrow-datafusion] metesynnada commented on a diff in pull request #6679: Improve performance/memory usage of HashJoin datastructure (5-15% improvement on selected TPC-H queries)

["metesynnada (via GitHub)" <gi...@apache.org>]

metesynnada commented on code in PR #6679:
URL: https://github.com/apache/arrow-datafusion/pull/6679#discussion_r1233938719


##########
datafusion/core/src/physical_plan/joins/hash_join.rs:
##########
@@ -563,16 +563,24 @@ pub fn update_hash(
     // insert hashes to key of the hashmap
     for (row, hash_value) in hash_values.iter().enumerate() {
         let item = hash_map
-            .0
+            .map
             .get_mut(*hash_value, |(hash, _)| *hash_value == *hash);
-        if let Some((_, indices)) = item {
-            indices.push((row + offset) as u64);
+        if let Some((_, index)) = item {
+            // Already exists: add index to next array
+            let prev_index = *index;
+            // Store new value inside hashmap
+            *index = (row + offset + 1) as u64;

Review Comment:
   Is it possible to hold the chain start in hashmap, instead of end of the chain? Is there any particular reason for this?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org