[GitHub] [arrow] nevi-me commented on a change in pull request #8664: ARROW-10588: [Rust] Safe and parallel bit operations for Arrow

[GitBox <gi...@apache.org>]

nevi-me commented on a change in pull request #8664:
URL: https://github.com/apache/arrow/pull/8664#discussion_r530293315



##########
File path: rust/arrow/Cargo.toml
##########
@@ -48,11 +49,12 @@ lazy_static = "1.4"
 packed_simd = { version = "0.3.4", optional = true, package = "packed_simd_2" }
 chrono = "0.4"
 flatbuffers = "0.6"
+bitvec = "0.19"
 hex = "0.4"
 prettytable-rs = { version = "0.8.0", optional = true }
 
 [features]
-default = []
+default = ["simd"]

Review comment:
       we probably shouldn't change the default to include simd, as we'd like the default features to allow users to compile with stable.

##########
File path: rust/arrow/src/compute/kernels/aggregate.rs
##########
@@ -141,32 +146,41 @@ where
 
     match array.data().null_buffer() {
         None => {
-            let sum = data.iter().fold(T::default_value(), |accumulator, value| {
-                accumulator + *value
-            });
+            let total = data
+                .par_iter()

Review comment:
       It's interesting that this is yielding better results. I would have thought that `rayon` being introduced at thsi level, would incur enough overhead to slow the kernels down. I've previously applied parallelism at an `Array` level instead.

##########
File path: rust/arrow/src/compute/kernels/aggregate.rs
##########
@@ -180,9 +194,12 @@ where
 ///
 /// Returns `None` if the array is empty or only contains null values.
 #[cfg(all(any(target_arch = "x86", target_arch = "x86_64"), feature = "simd"))]
-pub fn sum<T: ArrowNumericType>(array: &PrimitiveArray<T>) -> Option<T::Native>
+pub fn sum<T>(array: &PrimitiveArray<T>) -> Option<T::Native>
 where
-    T::Native: Add<Output = T::Native>,
+    T: ArrowNumericType,
+    // T::Native: Add<Output = T::Native> + Sum + Sum<T::Simd>,

Review comment:
       nit: should the comment be removed?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org