[GitHub] [arrow-datafusion] tustvold commented on a diff in pull request #7358: Specialize Avg and Sum (#6842)

["tustvold (via GitHub)" <gi...@apache.org>]

tustvold commented on code in PR #7358:
URL: https://github.com/apache/arrow-datafusion/pull/7358#discussion_r1301853347


##########
datafusion/physical-expr/src/aggregate/sum_distinct.rs:
##########
@@ -106,29 +116,56 @@ impl PartialEq<dyn Any> for DistinctSum {
     }
 }
 
-#[derive(Debug)]
-struct DistinctSumAccumulator {
-    hash_values: HashSet<ScalarValue, RandomState>,
+/// A wrapper around a type to provide hash for floats
+#[derive(Copy, Clone)]
+struct Hashable<T>(T);
+
+impl<T: ToByteSlice> std::hash::Hash for Hashable<T> {
+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
+        self.0.to_byte_slice().hash(state)

Review Comment:
   This does use ahash, it overrides the BuildHasher used by the HashSet. I think we could definitely do something better here, but in the absence of benchmarks I'm keen to go with simple and we can always optimise it later down the line. Regardless this should be significantly faster than the prior approach



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org