You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "neilconway (via GitHub)" <gi...@apache.org> on 2026/03/22 16:10:52 UTC
[PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
neilconway opened a new pull request, #21101:
URL: https://github.com/apache/datafusion/pull/21101
## Which issue does this PR close?
- Closes #21100.
## Rationale for this change
In the current implementation, we construct a `PrimitiveArray` for each row, feed it to the Arrow `min` / `max` kernel, and then collect the resulting `ScalarValue`s in a `Vec`. We then construct a final `PrimitiveArray` for the result via `ScalarValue::iter_to_array` of the `Vec`.
We can do better for ListArrays of primitive types. First, we can iterate directly over the flat values buffer of the `ListArray` for the batch and compute the min/max from each row's slice directly. Second, Arrow's `min` / `max` kernels have a reasonable amount of per-call overhead; for small arrays, it is more efficient to compute the min/max ourselves via direct iteration.
Benchmarks (1000 row batch, arrays of int64 values, M4 Max):
- no_nulls / list_size=10: 42.0 µs → 3.2 µs (13.1x faster)
- no_nulls / list_size=100: 52.0 µs → 17.9 µs (2.9x faster)
- no_nulls / list_size=1000: 144.3 µs → 107.5 µs (1.3x faster)
- nulls / list_size=10: 48.8 µs → 7.6 µs (6.4x faster)
- nulls / list_size=100: 97.2 µs → 70.8 µs (1.4x faster)
- nulls / list_size=1000: 654.0 µs → 633.1 µs (1.03x faster)
## What changes are included in this PR?
* Add benchmark for `array_max`
* Expand SLT test coverage
* Implement optimization
## Are these changes tested?
Yes.
## Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "neilconway (via GitHub)" <gi...@apache.org>.
neilconway commented on PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#issuecomment-4106763514
On an M4 Max, it looks like the crossover point between direct iteration and using the Arrow kernel is 32-40 list elements:
```
┌───────────┬──────────┬──────────┬─────────────────────┐
│ List size │ Scalar │ Kernel │ Kernel vs Scalar │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 8 │ 54.8 µs │ 172.7 µs │ scalar 3.2x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 16 │ 105.3 µs │ 188.1 µs │ scalar 1.8x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 32 │ 232.5 µs │ 253.2 µs │ scalar 1.09x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 48 │ 362.6 µs │ 329.6 µs │ kernel 1.10x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 64 │ 492.8 µs │ 444.2 µs │ kernel 1.11x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 96 │ 761.7 µs │ 589.0 µs │ kernel 1.29x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 128 │ 1.032 ms │ 782.0 µs │ kernel 1.32x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 256 │ 2.076 ms │ 1.428 ms │ kernel 1.45x faster │
├───────────┼──────────┼──────────┼─────────────────────┤
│ 512 │ 4.138 ms │ 2.728 ms │ kernel 1.52x faster │
└───────────┴──────────┴──────────┴─────────────────────┘
```
So I lowered the iteration -> kernel switchover threshold to 32.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#issuecomment-4108160509
Sure @neilconway . Thank you for trying the approaches. When we tried to improve cast ops in comet (string to integer), taking out the if condition and implementing 3 separate functions for various eval modes ( ANSI , Try and Legacy modes) helped with the performance. Let me find that PR up and link it here for reference for reference
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "neilconway (via GitHub)" <gi...@apache.org>.
neilconway commented on PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#issuecomment-4106523265
We could add a similar fastpath for arrays of strings, although maybe not worth it because `array_min` / max on arrays of strings is not particularly common?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on code in PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#discussion_r2973004438
##########
datafusion/sqllogictest/test_files/array.slt:
##########
@@ -1551,6 +1551,44 @@ NULL
query error DataFusion error: Error during planning: 'array_max' does not support zero arguments
select array_max();
+# array_max over multiple rows (exercises the offsets-based iteration)
+query I
+select array_max(column1) from (values
+ (make_array(1, 5, 3)),
+ (make_array(10, 2, 8)),
+ (NULL),
+ (make_array(NULL, 7, NULL)),
+ (make_array(100))
+) as t(column1);
+----
+5
+10
+NULL
+7
+100
Review Comment:
nit : perhaps we could also validate the result's datatype ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#issuecomment-4121229339
Thanks @neilconway and @coderfender and @Dandandan
I thought @coderfender 's comments were good suggestions -- maybe we can make a follow on PR to add them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "neilconway (via GitHub)" <gi...@apache.org>.
neilconway commented on PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#issuecomment-4107367618
@coderfender Thanks for the feedback!
I quickly checked 1 and 3 and they don't yield any improvement; I'd suspect the compiler will hoist loop-invariant branches like this out of the loop. The threshold check should be similar: it should be branch-predicted effectively.
Lmk if you disagree!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#discussion_r2973028445
##########
datafusion/functions-nested/src/min_max.rs:
##########
@@ -198,20 +202,120 @@ impl ScalarUDFImpl for ArrayMin {
fn array_min_inner(args: &[ArrayRef]) -> Result<ArrayRef> {
let [array] = take_function_args("array_min", args)?;
match array.data_type() {
- List(_) => array_min_max_helper(as_list_array(array)?, min_batch),
- LargeList(_) => array_min_max_helper(as_large_list_array(array)?, min_batch),
+ List(_) => array_min_max_helper(as_list_array(array)?, true),
+ LargeList(_) => array_min_max_helper(as_large_list_array(array)?, true),
arg_type => exec_err!("array_min does not support type: {arg_type}"),
}
}
fn array_min_max_helper<O: OffsetSizeTrait>(
array: &GenericListArray<O>,
- agg_fn: fn(&ArrayRef) -> Result<ScalarValue>,
+ is_min: bool,
) -> Result<ArrayRef> {
+ // Try the primitive fast path first
+ if let Some(result) = try_primitive_array_min_max(array, is_min) {
+ return result;
+ }
+
+ // Fallback: per-row ScalarValue path for non-primitive types
+ let agg_fn = if is_min { min_batch } else { max_batch };
let null_value = ScalarValue::try_from(array.value_type())?;
let result_vec: Vec<ScalarValue> = array
.iter()
.map(|arr| arr.as_ref().map_or_else(|| Ok(null_value.clone()), agg_fn))
.try_collect()?;
ScalarValue::iter_to_array(result_vec)
}
+
+/// Dispatches to a typed primitive min/max implementation, or returns `None` if
+/// the element type is not a primitive.
+fn try_primitive_array_min_max<O: OffsetSizeTrait>(
+ list_array: &GenericListArray<O>,
+ is_min: bool,
+) -> Option<Result<ArrayRef>> {
+ macro_rules! helper {
+ ($t:ty) => {
+ return Some(primitive_array_min_max::<O, $t>(list_array, is_min))
+ };
+ }
+ downcast_primitive! {
+ list_array.value_type() => (helper),
+ _ => {}
+ }
+ None
+}
+
+/// Threshold to switch from direct iteration to using `min` / `max` kernel from
+/// `arrow::compute`. The latter has enough per-invocation overhead that direct
+/// iteration is faster for small lists.
+const ARROW_COMPUTE_THRESHOLD: usize = 32;
+
+/// Computes min or max for each row of a primitive ListArray.
+fn primitive_array_min_max<O: OffsetSizeTrait, T: ArrowPrimitiveType>(
+ list_array: &GenericListArray<O>,
+ is_min: bool,
+) -> Result<ArrayRef> {
+ let values_array = list_array.values().as_primitive::<T>();
+ let values_slice = values_array.values();
+ let values_nulls = values_array.nulls();
+ let mut result_builder = PrimitiveBuilder::<T>::with_capacity(list_array.len())
+ .with_data_type(values_array.data_type().clone());
+
+ for (row, w) in list_array.offsets().windows(2).enumerate() {
+ let row_result = if list_array.is_null(row) {
+ None
+ } else {
+ let start = w[0].as_usize();
+ let end = w[1].as_usize();
+ let len = end - start;
+
+ match len {
+ 0 => None,
+ _ if len < ARROW_COMPUTE_THRESHOLD => {
+ scalar_min_max::<T>(values_slice, values_nulls, start, end, is_min)
+ }
+ _ => {
+ let slice = values_array.slice(start, len);
+ if is_min {
+ arrow::compute::min::<T>(&slice)
+ } else {
+ arrow::compute::max::<T>(&slice)
+ }
+ }
+ }
+ };
+
+ result_builder.append_option(row_result);
+ }
+
+ Ok(Arc::new(result_builder.finish()) as ArrayRef)
+}
+
+/// Computes min or max for a single list row by directly scanning a slice of
+/// the flat values buffer.
+#[inline]
+fn scalar_min_max<T: ArrowPrimitiveType>(
+ values_slice: &[T::Native],
+ values_nulls: Option<&arrow::buffer::NullBuffer>,
+ start: usize,
+ end: usize,
+ is_min: bool,
+) -> Option<T::Native> {
+ let mut best: Option<T::Native> = None;
+ for (i, &val) in values_slice[start..end].iter().enumerate() {
Review Comment:
I think for this small loop it makes sense to optimize for non-null case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #21101:
URL: https://github.com/apache/datafusion/pull/21101
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#discussion_r2973028445
##########
datafusion/functions-nested/src/min_max.rs:
##########
@@ -198,20 +202,120 @@ impl ScalarUDFImpl for ArrayMin {
fn array_min_inner(args: &[ArrayRef]) -> Result<ArrayRef> {
let [array] = take_function_args("array_min", args)?;
match array.data_type() {
- List(_) => array_min_max_helper(as_list_array(array)?, min_batch),
- LargeList(_) => array_min_max_helper(as_large_list_array(array)?, min_batch),
+ List(_) => array_min_max_helper(as_list_array(array)?, true),
+ LargeList(_) => array_min_max_helper(as_large_list_array(array)?, true),
arg_type => exec_err!("array_min does not support type: {arg_type}"),
}
}
fn array_min_max_helper<O: OffsetSizeTrait>(
array: &GenericListArray<O>,
- agg_fn: fn(&ArrayRef) -> Result<ScalarValue>,
+ is_min: bool,
) -> Result<ArrayRef> {
+ // Try the primitive fast path first
+ if let Some(result) = try_primitive_array_min_max(array, is_min) {
+ return result;
+ }
+
+ // Fallback: per-row ScalarValue path for non-primitive types
+ let agg_fn = if is_min { min_batch } else { max_batch };
let null_value = ScalarValue::try_from(array.value_type())?;
let result_vec: Vec<ScalarValue> = array
.iter()
.map(|arr| arr.as_ref().map_or_else(|| Ok(null_value.clone()), agg_fn))
.try_collect()?;
ScalarValue::iter_to_array(result_vec)
}
+
+/// Dispatches to a typed primitive min/max implementation, or returns `None` if
+/// the element type is not a primitive.
+fn try_primitive_array_min_max<O: OffsetSizeTrait>(
+ list_array: &GenericListArray<O>,
+ is_min: bool,
+) -> Option<Result<ArrayRef>> {
+ macro_rules! helper {
+ ($t:ty) => {
+ return Some(primitive_array_min_max::<O, $t>(list_array, is_min))
+ };
+ }
+ downcast_primitive! {
+ list_array.value_type() => (helper),
+ _ => {}
+ }
+ None
+}
+
+/// Threshold to switch from direct iteration to using `min` / `max` kernel from
+/// `arrow::compute`. The latter has enough per-invocation overhead that direct
+/// iteration is faster for small lists.
+const ARROW_COMPUTE_THRESHOLD: usize = 32;
+
+/// Computes min or max for each row of a primitive ListArray.
+fn primitive_array_min_max<O: OffsetSizeTrait, T: ArrowPrimitiveType>(
+ list_array: &GenericListArray<O>,
+ is_min: bool,
+) -> Result<ArrayRef> {
+ let values_array = list_array.values().as_primitive::<T>();
+ let values_slice = values_array.values();
+ let values_nulls = values_array.nulls();
+ let mut result_builder = PrimitiveBuilder::<T>::with_capacity(list_array.len())
+ .with_data_type(values_array.data_type().clone());
+
+ for (row, w) in list_array.offsets().windows(2).enumerate() {
+ let row_result = if list_array.is_null(row) {
+ None
+ } else {
+ let start = w[0].as_usize();
+ let end = w[1].as_usize();
+ let len = end - start;
+
+ match len {
+ 0 => None,
+ _ if len < ARROW_COMPUTE_THRESHOLD => {
+ scalar_min_max::<T>(values_slice, values_nulls, start, end, is_min)
+ }
+ _ => {
+ let slice = values_array.slice(start, len);
+ if is_min {
+ arrow::compute::min::<T>(&slice)
+ } else {
+ arrow::compute::max::<T>(&slice)
+ }
+ }
+ }
+ };
+
+ result_builder.append_option(row_result);
+ }
+
+ Ok(Arc::new(result_builder.finish()) as ArrayRef)
+}
+
+/// Computes min or max for a single list row by directly scanning a slice of
+/// the flat values buffer.
+#[inline]
+fn scalar_min_max<T: ArrowPrimitiveType>(
+ values_slice: &[T::Native],
+ values_nulls: Option<&arrow::buffer::NullBuffer>,
+ start: usize,
+ end: usize,
+ is_min: bool,
+) -> Option<T::Native> {
+ let mut best: Option<T::Native> = None;
+ for (i, &val) in values_slice[start..end].iter().enumerate() {
Review Comment:
I think for this small loop it makes sense to optimize for non-null case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on code in PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#discussion_r2973004438
##########
datafusion/sqllogictest/test_files/array.slt:
##########
@@ -1551,6 +1551,44 @@ NULL
query error DataFusion error: Error during planning: 'array_max' does not support zero arguments
select array_max();
+# array_max over multiple rows (exercises the offsets-based iteration)
+query I
+select array_max(column1) from (values
+ (make_array(1, 5, 3)),
+ (make_array(10, 2, 8)),
+ (NULL),
+ (make_array(NULL, 7, NULL)),
+ (make_array(100))
+) as t(column1);
+----
+5
+10
+NULL
+7
+100
Review Comment:
nit : perhaps we could also validate the resultant datatype (not that we would have different datatypes but I still think it is a good practice to have that additional check with the data type) :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on code in PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#discussion_r2973007258
##########
datafusion/sqllogictest/test_files/array.slt:
##########
@@ -1641,6 +1679,83 @@ NULL
query error DataFusion error: Error during planning: 'array_min' does not support zero arguments
select array_min();
+# array_min over multiple rows (exercises the offsets-based iteration)
+query I
+select array_min(column1) from (values
+ (make_array(1, 5, 3)),
+ (make_array(10, 2, 8)),
+ (NULL),
+ (make_array(NULL, 7, NULL)),
+ (make_array(100))
+) as t(column1);
+----
+1
+2
+NULL
+7
+100
+
+# array_min with NaN values (NaN should not be returned as min)
+query R
+select array_min(make_array(1.0, 'NaN'::double, 3.0));
+----
+1
+
+query R
+select array_min(make_array('NaN'::double, 'NaN'::double));
+----
+NaN
+
+query R
Review Comment:
nit : may be we could check + inf / - inf as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on code in PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#discussion_r2973008564
##########
datafusion/sqllogictest/test_files/array.slt:
##########
@@ -1641,6 +1679,83 @@ NULL
query error DataFusion error: Error during planning: 'array_min' does not support zero arguments
select array_min();
+# array_min over multiple rows (exercises the offsets-based iteration)
+query I
+select array_min(column1) from (values
+ (make_array(1, 5, 3)),
+ (make_array(10, 2, 8)),
+ (NULL),
+ (make_array(NULL, 7, NULL)),
+ (make_array(100))
+) as t(column1);
+----
+1
+2
+NULL
+7
+100
+
+# array_min with NaN values (NaN should not be returned as min)
+query R
+select array_min(make_array(1.0, 'NaN'::double, 3.0));
+----
+1
+
+query R
+select array_min(make_array('NaN'::double, 'NaN'::double));
+----
+NaN
+
+query R
+select array_min(make_array('NaN'::double, NULL));
+----
+NaN
+
+# array_min with Int32 (exercises a different primitive type than Int64)
+query I
+select array_min(arrow_cast(make_array(10, -5, 3), 'List(Int32)'));
Review Comment:
nice 👍🏽
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: Optimize `array_min`, `array_max` for arrays of primitive types [datafusion]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on PR #21101:
URL: https://github.com/apache/datafusion/pull/21101#issuecomment-4107028698
These are great numbers ! @neilconway . Could we perhaps also remove if conditions as well and see if those help out. Example :
1. Separate implementation for non null arrays ( to prevent if loop cycles inside the inner function)
2. Hot looping`ARROW_COMPUTE_THRESHOLD` if calls
3.min/max check (separate max vs min impl)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org