[GitHub] [arrow] alamb commented on a change in pull request #9116: ARROW-11156: [Rust][DataFusion] Create hashes vectorized in hash join [WIP]

[GitBox <gi...@apache.org>]

alamb commented on a change in pull request #9116:
URL: https://github.com/apache/arrow/pull/9116#discussion_r554584104



##########
File path: rust/datafusion/src/physical_plan/hash_join.rs
##########
@@ -55,13 +56,12 @@ use crate::error::{DataFusionError, Result};
 
 use super::{ExecutionPlan, Partitioning, RecordBatchStream, SendableRecordBatchStream};
 use crate::physical_plan::coalesce_batches::concat_batches;
-use ahash::RandomState;
 use log::debug;
 
-// Maps ["on" value] -> [list of indices with this key's value]
-// E.g. [1, 2] -> [(0, 3), (1, 6), (0, 8)] indicates that (column1, column2) = [1, 2] is true
-// for rows 3 and 8 from batch 0 and row 6 from batch 1.
-type JoinHashMap = HashMap<Vec<u8>, Vec<u64>, RandomState>;
+// Maps a `u64` hash value based on the left ["on" values] to a list of indices with this key's value.
+// E.g. [1, 2] -> [3, 6, 8] indicates that the column values map to rows 3, 6 and 8
+// As the key is a hash value, we need to check possible hash collisions in the probe stage

Review comment:
       👍 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org