You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/11/16 19:41:26 UTC
[GitHub] [arrow] jhorstmann opened a new pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
jhorstmann opened a new pull request #8685:
URL: https://github.com/apache/arrow/pull/8685
This refactors the simd aggregation to a reusable trait and adds implementations for min and max.
Some tests were failing with the simd feature active because the different order of additions rounded to a slightly different result. I reused the comparison function that @vertexclique implemented in his recent PR.
Microbenchmarks show a 9x-12x improvement:
```
$ cargo bench --features simd --bench aggregate_kernels
Compiling arrow v3.0.0-SNAPSHOT (/home/joernhorstmann/Source/github/apache/arrow/rust/arrow)
Finished bench [optimized] target(s) in 47.73s
Running /home/joernhorstmann/Source/github/apache/arrow/rust/target/release/deps/aggregate_kernels-717937ec43706892
Gnuplot not found, using plotters backend
sum 512 time: [75.840 ns 75.870 ns 75.903 ns]
change: [+0.3060% +0.3962% +0.4914%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 18 outliers among 100 measurements (18.00%)
3 (3.00%) low severe
2 (2.00%) low mild
7 (7.00%) high mild
6 (6.00%) high severe
min 512 time: [79.360 ns 79.422 ns 79.498 ns]
change: [-89.496% -89.483% -89.468%] (p = 0.00 < 0.05)
Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
4 (4.00%) high mild
12 (12.00%) high severe
sum nulls 512 time: [136.84 ns 136.93 ns 137.04 ns]
change: [+1.5542% +2.1189% +2.7055%] (p = 0.00 < 0.05)
Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
1 (1.00%) low severe
1 (1.00%) low mild
3 (3.00%) high mild
10 (10.00%) high severe
min nulls 512 time: [177.30 ns 177.38 ns 177.47 ns]
change: [-92.219% -92.206% -92.194%] (p = 0.00 < 0.05)
Performance has improved.
Found 17 outliers among 100 measurements (17.00%)
1 (1.00%) low mild
5 (5.00%) high mild
11 (11.00%) high severe
```
A datafusion benchmark using a global aggregation `SELECT MIN(f64), MAX(f64) FROM t` also shows a 2x improvement between running with and without feature.
```
Running /home/joernhorstmann/Source/github/apache/arrow/rust/target/release/deps/aggregate_query_sql-034b2ab6143485ec
Gnuplot not found, using plotters backend
aggregate_query_no_group_by_min_max_f64
time: [373.77 us 383.51 us 395.09 us]
change: [-59.420% -56.845% -53.563%] (p = 0.00 < 0.05)
Performance has improved.
Found 18 outliers among 100 measurements (18.00%)
2 (2.00%) low mild
1 (1.00%) high mild
15 (15.00%) high severe
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorgecarleitao commented on pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#issuecomment-739465567
Hi @jhorstmann , just to double check: are you blocked and need help, or can we review it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#issuecomment-728284606
https://issues.apache.org/jira/browse/ARROW-10216
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jhorstmann commented on pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jhorstmann commented on pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#issuecomment-739527274
Hi @jorgecarleitao , I did not have much time to work on this the last two weeks, but have now found a solution for handling NaN values consistently. Benchmarks need to be updated still, but I think it is ready for review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] vertexclique commented on a change in pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
vertexclique commented on a change in pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#discussion_r525066900
##########
File path: rust/arrow/src/datatypes.rs
##########
@@ -771,6 +784,21 @@ macro_rules! make_numeric_type {
fn write(simd_result: Self::Simd, slice: &mut [Self::Native]) {
unsafe { simd_result.write_to_slice_unaligned_unchecked(slice) };
}
+
+ #[inline]
+ fn mask_and(left: Self::SimdMask, right: Self::SimdMask) -> Self::SimdMask {
+ left & right
+ }
+
+ #[inline]
+ fn identity_for_min_op() -> Self::Native {
+ $max_value
+ }
+
+ #[inline]
+ fn identity_for_max_op() -> Self::Native {
+ $min_value
+ }
Review comment:
> Or do you mean there is an existing trait function in num::num that provides those constants for all numeric types?
yes: https://docs.rs/num/0.3.1/num/trait.Bounded.html
That will be useful in many places where we manually figuring out bounds.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] vertexclique commented on pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
vertexclique commented on pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#issuecomment-728367158
This looks nice, I will check this out tomorrow!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jhorstmann commented on pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jhorstmann commented on pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#issuecomment-731343546
Both scalar and simd implementations have issues involving null values, I added `ignore` tests with comments. The handling of NaN is also not consistent with the sort kernel, which follows the postgres model of sorting NaN as larger than any other number (including infinity). Solving those issues could be quite tricky and degrade the performance again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Dandandan commented on pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
Dandandan commented on pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#issuecomment-740997894
@jhorstmann
I wonder if we should support a standardized ordering like IEEE 754 instead?
There is a (unstable) implementation in rust std for it.
https://doc.rust-lang.org/beta/std/primitive.f64.html#method.total_cmp
Sounds something we could have in the source for now (it is only a few lines), I think it might also simplify some code like the current sort kernels? What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorgecarleitao closed pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jorgecarleitao closed pull request #8685:
URL: https://github.com/apache/arrow/pull/8685
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb commented on pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#issuecomment-740237607
I am checking it out
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jhorstmann commented on a change in pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jhorstmann commented on a change in pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#discussion_r525065747
##########
File path: rust/arrow/src/datatypes.rs
##########
@@ -771,6 +784,21 @@ macro_rules! make_numeric_type {
fn write(simd_result: Self::Simd, slice: &mut [Self::Native]) {
unsafe { simd_result.write_to_slice_unaligned_unchecked(slice) };
}
+
+ #[inline]
+ fn mask_and(left: Self::SimdMask, right: Self::SimdMask) -> Self::SimdMask {
+ left & right
+ }
+
+ #[inline]
+ fn identity_for_min_op() -> Self::Native {
+ $max_value
+ }
+
+ #[inline]
+ fn identity_for_max_op() -> Self::Native {
+ $min_value
+ }
Review comment:
I'm using those in https://github.com/apache/arrow/pull/8685/files#diff-45705886920f322ea90b662e46ed728e11c40402cedbe42d5718fe73d63c69abR273
Or do you mean there is an existing trait function in num::num that provides those constants for all numeric types?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jhorstmann commented on pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jhorstmann commented on pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#issuecomment-730012542
I still need to add a few more tests involving infinities and NaNs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on a change in pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#discussion_r537246148
##########
File path: rust/arrow/src/datatypes.rs
##########
@@ -618,7 +628,7 @@ macro_rules! make_numeric_type {
fn mask_from_u64(mask: u64) -> Self::SimdMask {
match Self::lanes() {
8 => {
- let vecidx = i64x8::new(128, 64, 32, 16, 8, 4, 2, 1);
+ let vecidx = i64x8::new(1, 2, 4, 8, 16, 32, 64, 128);
Review comment:
Maybe a line comment before this assignment could help others in understanding these numbers?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jhorstmann commented on a change in pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jhorstmann commented on a change in pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#discussion_r537062626
##########
File path: rust/arrow/src/datatypes.rs
##########
@@ -771,6 +784,21 @@ macro_rules! make_numeric_type {
fn write(simd_result: Self::Simd, slice: &mut [Self::Native]) {
unsafe { simd_result.write_to_slice_unaligned_unchecked(slice) };
}
+
+ #[inline]
+ fn mask_and(left: Self::SimdMask, right: Self::SimdMask) -> Self::SimdMask {
+ left & right
+ }
+
+ #[inline]
+ fn identity_for_min_op() -> Self::Native {
+ $max_value
+ }
+
+ #[inline]
+ fn identity_for_max_op() -> Self::Native {
+ $min_value
+ }
Review comment:
Refactored so the implementation no longer needs those bounds
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] vertexclique commented on a change in pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
vertexclique commented on a change in pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#discussion_r525009848
##########
File path: rust/arrow/src/datatypes.rs
##########
@@ -587,7 +600,7 @@ where
pub trait ArrowNumericType: ArrowPrimitiveType {}
macro_rules! make_numeric_type {
- ($impl_ty:ty, $native_ty:ty, $simd_ty:ident, $simd_mask_ty:ident) => {
+ ($impl_ty:ty, $native_ty:ty, $simd_ty:ident, $simd_mask_ty:ident, $min_value:expr, $max_value:expr) => {
Review comment:
Macro doesn't need to take min and max value. It can stay as it is except `mask_and`.
##########
File path: rust/arrow/src/datatypes.rs
##########
@@ -771,6 +784,21 @@ macro_rules! make_numeric_type {
fn write(simd_result: Self::Simd, slice: &mut [Self::Native]) {
unsafe { simd_result.write_to_slice_unaligned_unchecked(slice) };
}
+
+ #[inline]
+ fn mask_and(left: Self::SimdMask, right: Self::SimdMask) -> Self::SimdMask {
+ left & right
+ }
+
+ #[inline]
+ fn identity_for_min_op() -> Self::Native {
+ $max_value
+ }
+
+ #[inline]
+ fn identity_for_max_op() -> Self::Native {
+ $min_value
+ }
Review comment:
These can be removed.
##########
File path: rust/arrow/src/compute/kernels/aggregate.rs
##########
@@ -179,87 +181,258 @@ where
}
}
-/// Returns the sum of values in the array.
-///
-/// Returns `None` if the array is empty or only contains null values.
#[cfg(all(any(target_arch = "x86", target_arch = "x86_64"), feature = "simd"))]
-pub fn sum<T: ArrowNumericType>(array: &PrimitiveArray<T>) -> Option<T::Native>
-where
- T::Native: Add<Output = T::Native>,
-{
- let null_count = array.null_count();
+mod simd {
+ use crate::array::{Array, PrimitiveArray};
+ use crate::datatypes::ArrowNumericType;
+ use std::marker::PhantomData;
+ use std::ops::Add;
+
+ pub(super) trait SimdAggregate<T: ArrowNumericType> {
+ /// Returns the identity value for this aggregation function
+ fn init_accumulator_scalar() -> T::Native;
+
+ /// Returns a vector filled with the identity value for this aggregation function
+ #[inline]
+ fn init_accumulator_chunk() -> T::Simd {
+ T::init(Self::init_accumulator_scalar())
+ }
- if null_count == array.len() {
- return None;
+ /// Updates the accumulator with the values of one chunk
+ fn accumulate_chunk_non_null(accumulator: &mut T::Simd, chunk: T::Simd);
+
+ /// Updates the accumulator with the values of one chunk according to the given vector mask
+ fn accumulate_chunk_nullable(
+ accumulator: &mut T::Simd,
+ chunk: T::Simd,
+ mask: T::SimdMask,
+ );
+
+ /// Updates the accumulator with one value
+ fn accumulate_scalar(accumulator: &mut T::Native, value: T::Native);
+
+ /// Reduces the vector lanes of the accumulator to a single value
+ #[inline]
+ fn reduce(accumulator: T::Simd) -> T::Native {
+ // reduce by first writing to a temporary and then use scalar operations
+ // this should be about the same performance as extracting individual lanes
+ // but allows us to reuse the scalar reduction logic
+ let tmp = &mut [T::default_value(); 64];
+ T::write(accumulator, &mut tmp[0..T::lanes()]);
+
+ let mut reduced = Self::init_accumulator_scalar();
+ tmp[0..T::lanes()]
+ .iter()
+ .for_each(|value| Self::accumulate_scalar(&mut reduced, *value));
+
+ reduced
+ }
}
- let data: &[T::Native] = array.value_slice(0, array.len());
+ pub(super) struct SumAggregate<T: ArrowNumericType> {
+ phantom: PhantomData<T>,
+ }
- let mut vector_sum = T::init(T::default_value());
- let mut remainder_sum = T::default_value();
+ impl<T: ArrowNumericType> SimdAggregate<T> for SumAggregate<T>
+ where
+ T::Native: Add<Output = T::Native>,
+ {
+ fn init_accumulator_scalar() -> T::Native {
+ T::default_value()
+ }
- match array.data().null_buffer() {
- None => {
- let data_chunks = data.chunks_exact(64);
- let remainder = data_chunks.remainder();
+ fn accumulate_chunk_non_null(accumulator: &mut T::Simd, chunk: T::Simd) {
+ *accumulator = *accumulator + chunk;
+ }
- data_chunks.for_each(|chunk| {
- chunk.chunks_exact(T::lanes()).for_each(|chunk| {
- let chunk = T::load(&chunk);
- vector_sum = vector_sum + chunk;
- });
- });
+ fn accumulate_chunk_nullable(
+ accumulator: &mut T::Simd,
+ chunk: T::Simd,
+ vecmask: T::SimdMask,
+ ) {
+ let zero = T::init(T::default_value());
+ let blended = T::mask_select(vecmask, chunk, zero);
- remainder.iter().for_each(|value| {
- remainder_sum = remainder_sum + *value;
- });
+ *accumulator = *accumulator + blended;
}
- Some(buffer) => {
- // process data in chunks of 64 elements since we also get 64 bits of validity information at a time
- let data_chunks = data.chunks_exact(64);
- let remainder = data_chunks.remainder();
- let bit_chunks = buffer.bit_chunks(array.offset(), array.len());
- let remainder_bits = bit_chunks.remainder_bits();
+ fn accumulate_scalar(accumulator: &mut T::Native, value: T::Native) {
+ *accumulator = *accumulator + value
+ }
+ }
+
+ pub(super) struct MinAggregate<T: ArrowNumericType> {
+ phantom: PhantomData<T>,
+ }
+
+ impl<T: ArrowNumericType> SimdAggregate<T> for MinAggregate<T>
+ where
+ T::Native: PartialOrd,
+ {
+ fn init_accumulator_scalar() -> T::Native {
+ T::identity_for_min_op()
+ }
+
+ fn accumulate_chunk_non_null(accumulator: &mut T::Simd, chunk: T::Simd) {
+ let cmp_mask = T::lt(chunk, *accumulator);
+
+ *accumulator = T::mask_select(cmp_mask, chunk, *accumulator);
+ }
+
+ fn accumulate_chunk_nullable(
+ accumulator: &mut T::Simd,
+ chunk: T::Simd,
+ vecmask: T::SimdMask,
+ ) {
+ let cmp_mask = T::lt(chunk, *accumulator);
+ let blend_mask = T::mask_and(vecmask, cmp_mask);
+
+ *accumulator = T::mask_select(blend_mask, chunk, *accumulator);
+ }
+
+ fn accumulate_scalar(accumulator: &mut T::Native, value: T::Native) {
+ if value < *accumulator {
+ *accumulator = value
+ }
+ }
+ }
+
+ pub(super) struct MaxAggregate<T: ArrowNumericType> {
+ phantom: PhantomData<T>,
+ }
+
+ impl<T: ArrowNumericType> SimdAggregate<T> for MaxAggregate<T>
+ where
+ T::Native: PartialOrd,
+ {
+ fn init_accumulator_scalar() -> T::Native {
+ T::identity_for_max_op()
+ }
+
+ fn accumulate_chunk_non_null(accumulator: &mut T::Simd, chunk: T::Simd) {
+ let cmp_mask = T::gt(chunk, *accumulator);
+
+ *accumulator = T::mask_select(cmp_mask, chunk, *accumulator);
+ }
+
+ fn accumulate_chunk_nullable(
+ accumulator: &mut T::Simd,
+ chunk: T::Simd,
+ vecmask: T::SimdMask,
+ ) {
+ let cmp_mask = T::gt(chunk, *accumulator);
+ let blend_mask = T::mask_and(vecmask, cmp_mask);
+
+ *accumulator = T::mask_select(blend_mask, chunk, *accumulator);
+ }
+
+ fn accumulate_scalar(accumulator: &mut T::Native, value: T::Native) {
+ if value > *accumulator {
+ *accumulator = value
+ }
+ }
+ }
+
+ pub(super) fn simd_aggregation<T: ArrowNumericType, A: SimdAggregate<T>>(
+ array: &PrimitiveArray<T>,
+ ) -> Option<T::Native> {
+ let null_count = array.null_count();
+
+ if null_count == array.len() {
+ return None;
+ }
+
+ let data: &[T::Native] = array.value_slice(0, array.len());
+
+ let mut chunk_acc = A::init_accumulator_chunk();
+ let mut rem_acc = A::init_accumulator_scalar();
- data_chunks.zip(bit_chunks).for_each(|(chunk, mut mask)| {
- // split chunks further into slices corresponding to the vector length
- // the compiler is able to unroll this inner loop and remove bounds checks
- // since the outer chunk size (64) is always a multiple of the number of lanes
- chunk.chunks_exact(T::lanes()).for_each(|chunk| {
- let zero = T::init(T::default_value());
- let vecmask = T::mask_from_u64(mask);
- let chunk = T::load(&chunk);
- let blended = T::mask_select(vecmask, chunk, zero);
+ match array.data().null_buffer() {
+ None => {
+ let data_chunks = data.chunks_exact(64);
+ let remainder = data_chunks.remainder();
- vector_sum = vector_sum + blended;
+ data_chunks.for_each(|chunk| {
+ chunk.chunks_exact(T::lanes()).for_each(|chunk| {
+ let chunk = T::load(&chunk);
+ A::accumulate_chunk_non_null(&mut chunk_acc, chunk);
+ });
+ });
- mask = mask >> T::lanes();
+ remainder.iter().for_each(|value| {
+ A::accumulate_scalar(&mut rem_acc, *value);
});
- });
+ }
+ Some(buffer) => {
+ // process data in chunks of 64 elements since we also get 64 bits of validity information at a time
+ let data_chunks = data.chunks_exact(64);
+ let remainder = data_chunks.remainder();
- remainder.iter().enumerate().for_each(|(i, value)| {
- if remainder_bits & (1 << i) != 0 {
- remainder_sum = remainder_sum + *value;
- }
- });
+ let bit_chunks = buffer.bit_chunks(array.offset(), array.len());
+ let remainder_bits = bit_chunks.remainder_bits();
+
+ data_chunks.zip(bit_chunks).for_each(|(chunk, mut mask)| {
+ // split chunks further into slices corresponding to the vector length
+ // the compiler is able to unroll this inner loop and remove bounds checks
+ // since the outer chunk size (64) is always a multiple of the number of lanes
+ chunk.chunks_exact(T::lanes()).for_each(|chunk| {
+ let vecmask = T::mask_from_u64(mask);
+ let chunk = T::load(&chunk);
+
+ A::accumulate_chunk_nullable(&mut chunk_acc, chunk, vecmask);
+
+ mask = mask >> T::lanes();
+ });
+ });
+
+ remainder.iter().enumerate().for_each(|(i, value)| {
+ if remainder_bits & (1 << i) != 0 {
+ A::accumulate_scalar(&mut rem_acc, *value)
+ }
+ });
+ }
}
+
+ let mut total = A::reduce(chunk_acc);
+ A::accumulate_scalar(&mut total, rem_acc);
+
+ Some(total)
}
+}
+
+/// Returns the sum of values in the array.
+///
+/// Returns `None` if the array is empty or only contains null values.
+#[cfg(all(any(target_arch = "x86", target_arch = "x86_64"), feature = "simd"))]
+pub fn sum<T: ArrowNumericType>(array: &PrimitiveArray<T>) -> Option<T::Native>
+where
+ T::Native: Add<Output = T::Native>,
+{
+ use simd::*;
- // calculate horizontal sum of accumulator by writing to a temporary
- // this is probably faster than extracting individual lanes
- // the compiler is free to optimize this to something faster
- let tmp = &mut [T::default_value(); 64];
- T::write(vector_sum, &mut tmp[0..T::lanes()]);
+ simd::simd_aggregation::<T, SumAggregate<T>>(&array)
+}
- let mut total_sum = T::default_value();
- tmp[0..T::lanes()]
- .iter()
- .for_each(|lane| total_sum = total_sum + *lane);
+#[cfg(all(any(target_arch = "x86", target_arch = "x86_64"), feature = "simd"))]
+/// Returns the minimum value in the array, according to the natural order.
+pub fn min<T: ArrowNumericType>(array: &PrimitiveArray<T>) -> Option<T::Native>
+where
+ T::Native: PartialOrd,
+{
+ use simd::*;
- total_sum = total_sum + remainder_sum;
+ simd::simd_aggregation::<T, MinAggregate<T>>(&array)
+}
+
+#[cfg(all(any(target_arch = "x86", target_arch = "x86_64"), feature = "simd"))]
+/// Returns the maximum value in the array, according to the natural order.
+pub fn max<T: ArrowNumericType>(array: &PrimitiveArray<T>) -> Option<T::Native>
+where
+ T::Native: PartialOrd,
Review comment:
```suggestion
T::Native: num::Num + PartialOrd,
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jhorstmann commented on a change in pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jhorstmann commented on a change in pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#discussion_r526469339
##########
File path: rust/arrow/src/datatypes.rs
##########
@@ -618,7 +628,7 @@ macro_rules! make_numeric_type {
fn mask_from_u64(mask: u64) -> Self::SimdMask {
match Self::lanes() {
8 => {
- let vecidx = i64x8::new(128, 64, 32, 16, 8, 4, 2, 1);
+ let vecidx = i64x8::new(1, 2, 4, 8, 16, 32, 64, 128);
Review comment:
And the tests for mask generation were apparently also incorrect.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jhorstmann commented on a change in pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jhorstmann commented on a change in pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#discussion_r526388987
##########
File path: rust/arrow/src/datatypes.rs
##########
@@ -618,7 +628,7 @@ macro_rules! make_numeric_type {
fn mask_from_u64(mask: u64) -> Self::SimdMask {
match Self::lanes() {
8 => {
- let vecidx = i64x8::new(128, 64, 32, 16, 8, 4, 2, 1);
+ let vecidx = i64x8::new(1, 2, 4, 8, 16, 32, 64, 128);
Review comment:
Tests for min/max exposed that these constants were actually in the wrong order. This didn't turn up in the sum tests because there all invalid elements were already set to 0. I think when I initially implemented this, I copied the initialization from code that used intel avx intrinsics which take the parameters in the opposite order :facepalm:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jhorstmann commented on pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jhorstmann commented on pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#issuecomment-741606137
@Dandandan that sounds like good idea as it is a bit more standardized. The current ordering was choosen initially for the sort kernels, as it is the same ordering that Postgres implements. The main difference seems to be the handling of negative NaN. If we decide on a different ordering we should keep sort and min/max consistent.
One nice thing about the current impl for min/max is that all numeric types are handled the same, with the compiler optimizing away the `is_nan` function for integer types.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jhorstmann commented on a change in pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels
Posted by GitBox <gi...@apache.org>.
jhorstmann commented on a change in pull request #8685:
URL: https://github.com/apache/arrow/pull/8685#discussion_r525196649
##########
File path: rust/arrow/src/datatypes.rs
##########
@@ -771,6 +784,21 @@ macro_rules! make_numeric_type {
fn write(simd_result: Self::Simd, slice: &mut [Self::Native]) {
unsafe { simd_result.write_to_slice_unaligned_unchecked(slice) };
}
+
+ #[inline]
+ fn mask_and(left: Self::SimdMask, right: Self::SimdMask) -> Self::SimdMask {
+ left & right
+ }
+
+ #[inline]
+ fn identity_for_min_op() -> Self::Native {
+ $max_value
+ }
+
+ #[inline]
+ fn identity_for_max_op() -> Self::Native {
+ $min_value
+ }
Review comment:
"smallest **finite** number this type can represent" -> I have to think about or write some tests to see whether that works for floats.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org