You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "mightsleep (via GitHub)" <gi...@apache.org> on 2026/10/14 16:27:54 UTC
[PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
mightsleep opened a new pull request, #11559:
URL: https://github.com/apache/arrow-rs/pull/11559
# Which issue does this PR close?
- Part of #11437.
# Rationale for this change
`zip` with a scalar on one side extends `MutableArrayData` once per scalar row and once per run of array rows, 6 to 7 ns a row on random masks. Callers include DataFusion's Spark `%` and `pmod`, `unnest` and `union_extract`.
# What changes are included in this PR?
For a primitive array and a scalar of the same type:
- Values are a branch-free select per row over blocks of 64 rows. Mask words that are all set or all clear copy or fill. 16 and 32 byte values are 2 or 4 `i64` lanes.
- The lane masks are computed into an array before the select. Computed inside it, LLVM emits a branch per row, which mispredicts on random masks.
- Nulls in closed form. With a null scalar the array's values buffer is shared.
Other types keep the `MutableArrayData` path.
# Are these changes tested?
`test_zip_primitive_with_scalar_matches_general_path` compares against `zip_impl` for 8 types (including `Decimal256` and `Interval(MonthDayNano)`), with offsets, nulls in the array and the mask, a null and a valid scalar, on either side.
ns per row, 8192 rows, a pool of 512 random masks so the branch predictor cannot learn them:
| type | main | this PR |
|---|---|---|
| i8 | 6.0 to 7.3 | 0.32 |
| i32 | 6.3 to 7.4 | 0.30 |
| f64 | 6.7 to 7.6 | 0.38 |
| Decimal128 | 6.3 to 7.4 | 0.66 |
| i32, null scalar | 9 to 10 | 0.05 |
`zip_kernels` (i32) on GitHub runners (Neoverse-N2, Apple M1, EPYC 9V74 default and x86-64-v3), main / this PR: 22 to 60x on random masks, 1.07 to 1.35x when every row comes from the array.
# Are there any significant user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6186457442
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6186136950)
**Target:** zip_kernels
Comparing zip-scalar-blend (deecbb71446efd61c0963b5f5802731b894edc10) to 4dc6bc2 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/4dc6bc25f351dd0b8474339ca342d488723f1368..deecbb71446efd61c0963b5f5802731b894edc10)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
env:
BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array
shards: 1
```
</details>
<details><summary>Details</summary>
```
group base changed
----- ---- -------
zip_8192_from_i32/array_vs_non_null_scalar/10pct_true 30.47 77.7±0.10µs ? ?/sec 1.00 2.5±0.01µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/1pct_true 41.44 71.9±0.30µs ? ?/sec 1.00 1734.7±6.50ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls 30.28 81.8±0.16µs ? ?/sec 1.00 2.7±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_true 30.47 78.1±0.20µs ? ?/sec 1.00 2.6±0.01µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/90pct_true 8.72 22.4±0.25µs ? ?/sec 1.00 2.6±0.01µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/99pct_true 2.26 4.2±0.03µs ? ?/sec 1.00 1860.7±4.72ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_false 59.08 70.7±0.13µs ? ?/sec 1.00 1196.8±9.10ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_true 1.25 1533.3±5.02ns ? ?/sec 1.00 1224.9±7.28ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/true_then_false 28.54 36.2±0.07µs ? ?/sec 1.00 1267.9±7.48ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/10pct_true 9.06 24.0±0.26µs ? ?/sec 1.00 2.6±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/1pct_true 3.05 5.6±0.02µs ? ?/sec 1.00 1828.1±6.64ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls 18.89 51.5±0.31µs ? ?/sec 1.00 2.7±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_true 32.10 85.1±0.20µs ? ?/sec 1.00 2.7±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/90pct_true 30.86 82.6±0.10µs ? ?/sec 1.00 2.7±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/99pct_true 38.83 76.3±0.13µs ? ?/sec 1.00 1964.3±7.44ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_false 1.17 1590.6±7.09ns ? ?/sec 1.00 1360.4±7.39ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_true 57.84 74.7±0.26µs ? ?/sec 1.00 1291.0±9.18ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/true_then_false 29.20 39.2±0.15µs ? ?/sec 1.00 1342.4±7.96ns ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/10pct_true 1.01 95.3±0.54µs ? ?/sec 1.00 94.2±0.32µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/1pct_true 1.01 89.6±0.67µs ? ?/sec 1.00 88.7±0.42µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/50pct_nulls 1.01 99.2±0.48µs ? ?/sec 1.00 97.8±0.29µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/50pct_true 1.04 94.7±0.50µs ? ?/sec 1.00 91.4±0.30µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/90pct_true 1.01 33.1±0.28µs ? ?/sec 1.00 32.8±0.27µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/99pct_true 1.00 11.9±0.04µs ? ?/sec 1.00 11.9±0.04µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/all_false 1.01 88.5±0.62µs ? ?/sec 1.00 87.6±0.38µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/all_true 1.01 8.7±0.02µs ? ?/sec 1.00 8.6±0.01µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/true_then_false 1.01 48.6±0.34µs ? ?/sec 1.00 48.3±0.28µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/10pct_true 1.01 36.7±0.25µs ? ?/sec 1.00 36.5±0.25µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/1pct_true 1.00 15.2±0.04µs ? ?/sec 1.00 15.1±0.04µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/50pct_nulls 1.03 69.2±0.42µs ? ?/sec 1.00 66.9±0.32µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/50pct_true 1.04 106.1±0.47µs ? ?/sec 1.00 102.2±0.32µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/90pct_true 1.01 102.8±0.50µs ? ?/sec 1.00 101.9±0.36µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/99pct_true 1.01 96.5±0.56µs ? ?/sec 1.00 95.7±0.37µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/all_false 1.01 8.9±0.08µs ? ?/sec 1.00 8.8±0.01µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/all_true 1.01 95.1±0.57µs ? ?/sec 1.00 94.1±0.39µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/true_then_false 1.02 54.0±0.32µs ? ?/sec 1.00 53.0±0.22µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/10pct_true 1.01 118.5±0.59µs ? ?/sec 1.00 117.9±0.61µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/1pct_true 1.01 110.9±0.72µs ? ?/sec 1.00 110.1±0.63µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/50pct_nulls 1.01 122.5±0.50µs ? ?/sec 1.00 121.7±0.51µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/50pct_true 1.02 111.6±0.42µs ? ?/sec 1.00 109.3±0.36µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/90pct_true 1.01 38.5±0.27µs ? ?/sec 1.00 38.0±0.24µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/99pct_true 1.00 14.4±0.04µs ? ?/sec 1.00 14.5±0.03µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/all_false 1.01 109.6±0.63µs ? ?/sec 1.00 109.0±0.62µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/all_true 1.00 8.8±0.01µs ? ?/sec 1.00 8.8±0.01µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/true_then_false 1.01 60.5±0.40µs ? ?/sec 1.00 59.7±0.38µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/10pct_true 1.00 41.3±0.33µs ? ?/sec 1.02 42.2±0.23µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/1pct_true 1.00 15.9±0.05µs ? ?/sec 1.02 16.3±0.05µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/50pct_nulls 1.02 77.0±0.40µs ? ?/sec 1.00 75.8±0.41µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/50pct_true 1.02 124.0±0.52µs ? ?/sec 1.00 121.5±0.40µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/90pct_true 1.00 130.5±0.62µs ? ?/sec 1.00 130.3±0.56µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/99pct_true 1.01 119.8±0.57µs ? ?/sec 1.00 118.6±0.51µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/all_false 1.00 8.9±0.03µs ? ?/sec 1.00 8.8±0.01µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/all_true 1.01 117.1±0.57µs ? ?/sec 1.00 116.4±0.49µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/true_then_false 1.00 65.5±0.30µs ? ?/sec 1.00 65.7±0.25µs ? ?/sec
```
</details>
<details><summary>Per-runner information</summary>
<details><summary>zip_kernels — shard 1/1</summary>
**Node:** gk3-benchmark-cluster-nap-671s0p7k-bef850df-xlxs
**Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
**uname:**
```text
Linux bench-c6186136950-3433-c2x8g 6.12.94+ #1 SMP Fri Aug 21 08:00:16 UTC 2026 aarch64 GNU/Linux
```
**BENCH_COMMAND:**
```sh
cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
```
<details><summary>CPU Details (lscpu)</summary>
```text
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Resource Usage</summary>
**base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 525.1s |
| Peak memory | 14.3 MiB |
| Avg memory | 13.6 MiB |
| CPU user | 519.4s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
**branch**
| Metric | Value |
|--------|-------|
| Wall time | 530.1s |
| Peak memory | 14.7 MiB |
| Avg memory | 14.1 MiB |
| CPU user | 526.6s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
</details>
</details>
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6186136950:finish:890393730d3f4feb709d0eee12eb8e53 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383519765
🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383519229)
**Target:** zip_kernels
**Sharding factor: 1** (1 workers).
Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
env:
BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
RUSTFLAGS: -C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6
shards: 1
```
</details>
Results will be posted when all workers finish.
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6383519229:start:67d0738b1ec27686058081579ff388af -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383510708
```
env:
BENCH_FILTER: "from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)"
RUSTFLAGS: "-C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6"
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mightsleep (via GitHub)" <gi...@apache.org>.
mightsleep commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4363377028
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return Ok(output);
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+ 16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+ 32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+ _ => return None,
+ }
+ } else {
+ array.buffers()[0].slice_with_length(array.offset() * width, len * width)
+ };
+
+ let data = ArrayData::builder(array.data_type().clone())
+ .len(len)
+ .add_buffer(values)
+ .nulls(nulls.map(NullBuffer::new));
+ Some(make_array(
+ data.build().expect("the layout of the input array"),
+ ))
+}
+
+/// An integer lane that [`blend`] selects with.
+trait BlendLane:
+ ArrowNativeType + BitAnd<Output = Self> + BitOr<Output = Self> + Not<Output = Self>
+{
+ /// `mask` (0 or -1) at this width.
+ fn from_mask(mask: i32) -> Self;
+}
+
+macro_rules! blend_lane {
+ ($($t:ty),*) => {
+ $(impl BlendLane for $t {
+ #[inline]
+ fn from_mask(mask: i32) -> Self {
+ mask as $t
+ }
+ })*
+ };
+}
+blend_lane!(i8, i16, i64);
+
+impl BlendLane for i32 {
+ #[inline]
+ fn from_mask(mask: i32) -> Self {
+ mask
+ }
+}
+
+/// For each bit of `word`, an `i32` with all bits set if it is set, else clear.
+#[inline]
+fn lane_masks(word: u64) -> [i32; 64] {
+ // Every lane of a byte tests a constant bit of the same value: an AND and a compare of
+ // 32-bit lanes, which SSE2 and NEON have. Shifting the word by a different count per lane
+ // has no SIMD form before AVX2, and SSE2 has no 64-bit compare.
Review Comment:
my bad, thanks for checking. I had verified this loop for branches, not for how the masks are computed, and LLVM turns the compare into a shift into the sign bit, which SSE2 emulates with pmuludq. Took your wording.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "Jefffrey (via GitHub) via github" <gi...@arrow.apache.org>.
Jefffrey commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4481428036
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,174 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return output;
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<Result<ArrayRef, ArrowError>> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+ 16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+ 32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+ _ => return None,
+ }
+ } else {
+ array.buffers()[0].slice_with_length(array.offset() * width, len * width)
Review Comment:
for the null scalar case, we could actually extend this to work for all datatypes (except runendencoded and unions) since we're just replacing the null buffer of the array; though we can do this in a followup PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mightsleep (via GitHub)" <gi...@apache.org>.
mightsleep commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4363354673
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return Ok(output);
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
Review Comment:
Thanks for digging into this and for filing #11862. I'd like to keep this PR about speed. The fast path dispatches on width only, and an unaligned buffer falls back to zip_impl through the generic alignment check in rows, so the behavior for it doesn't change here. Handling DayTime alignment in each kernel with a fast path would spread the same workaround around, while the fix in MutableArrayData, as #10171 did for MonthDayNano, covers zip, merge and every other kernel that copies through it at once. I can pick up #11862 once one of my open PRs lands, unless someone gets to it first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6434141714
```
zip_8192_from_decimal128/array_vs_non_null_scalar/true_then_false 12.86 37.5±0.14µs ? ?/sec 1.00 2.9±0.02µs ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/10pct_true 183.32 105.8±0.38µs ? ?/sec 1.00 577.0±3.43ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/1pct_true 170.61 98.4±0.36µs ? ?/sec 1.00 576.7±3.37ns ? ?/sec
```
oh wow 🤯
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mightsleep (via GitHub)" <gi...@apache.org>.
mightsleep commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6349928266
Rebased on #11625
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4361268730
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return Ok(output);
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
Review Comment:
What happens here for an `Interval(DayTime)` array whose values buffer is only 4-byte aligned? arrow-rs requires a fixed-width buffer to be aligned only to `align_of` of its native type ([`new_fixed_width`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-data/src/data.rs#L2058-L2063)), and [`IntervalDayTime`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-buffer/src/interval.rs#L348-L355) is two `i32`s, so its alignment is 4. The Arrow format only [recommends 8- or 64-byte alignment](https://github.com/apache/arrow/blob/373d64c02cb23663b63c5853099fca43a46c39fc/docs/source/format/Columnar.rst?plain=1#L264-L273) in memory.
For such a buffer (I built one by slicing a `Buffer` 4 bytes in), `blend::<i64, 1>` returns `None`, and the fallback panics in `Buffer::typed_data`, because `MutableArrayData` [extends `DayTime` as `i64`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-data/src/transform/mod.rs#L259-L263). That panic also happens on `main` without this PR, so it needs its own fix.
For this PR, what do you think about blending `DayTime` as two `i32` lanes? The aligned case would then never fall back, and as I read it, the alignment check in `rows` would no longer fail for any type that reaches `blend`. Could the test also add `Interval(DayTime)` to `data_types`, plus one case with a 4-byte-aligned buffer? `random_primitive` always produces aligned buffers. This needs `IntervalUnit` in the top-level `arrow_schema` import.
```suggestion
8 if *array.data_type() == DataType::Interval(IntervalUnit::DayTime) => {
// arrow-rs aligns a `DayTime` buffer only to `align_of::<IntervalDayTime>()`, 4
blend::<i32, 2>(&from_array, &array, &scalar)?
}
8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383714117
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383511996)
**Target:** zip_kernels
Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
env:
BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
RUSTFLAGS: -C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6
resources:
arch: amd64
shards: 1
```
</details>
<details><summary>Details</summary>
```
group base changed
----- ---- -------
zip_8192_from_decimal128/array_vs_non_null_scalar/10pct_true 8.39 73.4±1.91µs ? ?/sec 1.00 8.8±0.13µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/1pct_true 9.73 61.5±1.80µs ? ?/sec 1.00 6.3±0.08µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_nulls 9.71 85.1±2.37µs ? ?/sec 1.00 8.8±0.16µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_true 10.30 90.5±1.54µs ? ?/sec 1.00 8.8±0.14µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/90pct_true 3.25 28.1±0.46µs ? ?/sec 1.00 8.6±0.14µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/99pct_true 1.18 8.1±0.17µs ? ?/sec 1.00 6.9±0.13µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/all_false 22.12 60.4±0.92µs ? ?/sec 1.00 2.7±0.05µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/all_true 1.06 5.6±0.04µs ? ?/sec 1.00 5.3±0.08µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/true_then_false 7.96 33.0±0.53µs ? ?/sec 1.00 4.1±0.06µs ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/10pct_true 176.17 92.9±1.39µs ? ?/sec 1.00 527.1±8.94ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/1pct_true 154.13 81.9±1.06µs ? ?/sec 1.00 531.3±10.80ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/50pct_nulls 177.48 103.1±2.02µs ? ?/sec 1.00 580.8±13.25ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/50pct_true 183.86 96.3±1.57µs ? ?/sec 1.00 523.8±10.25ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/90pct_true 56.56 29.9±0.52µs ? ?/sec 1.00 529.4±10.98ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/99pct_true 15.66 8.3±0.10µs ? ?/sec 1.00 528.6±11.18ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/all_false 152.09 80.0±1.42µs ? ?/sec 1.00 525.8±9.43ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/all_true 10.93 5.7±0.04µs ? ?/sec 1.00 521.8±7.71ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/true_then_false 81.30 43.0±0.72µs ? ?/sec 1.00 529.2±13.58ns ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/10pct_true 3.91 34.1±0.51µs ? ?/sec 1.00 8.7±0.16µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/1pct_true 1.83 12.9±0.14µs ? ?/sec 1.00 7.1±0.15µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_nulls 6.95 61.4±0.72µs ? ?/sec 1.00 8.8±0.17µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_true 11.29 98.3±1.54µs ? ?/sec 1.00 8.7±0.11µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/90pct_true 8.97 78.9±1.07µs ? ?/sec 1.00 8.8±0.15µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/99pct_true 10.49 67.5±1.19µs ? ?/sec 1.00 6.4±0.09µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/all_false 1.08 5.8±0.04µs ? ?/sec 1.00 5.3±0.16µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/all_true 23.27 64.9±1.36µs ? ?/sec 1.00 2.8±0.04µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/true_then_false 9.09 37.5±0.92µs ? ?/sec 1.00 4.1±0.05µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/10pct_true 20.26 71.0±0.92µs ? ?/sec 1.00 3.5±0.06µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/1pct_true 24.91 62.2±1.49µs ? ?/sec 1.00 2.5±0.05µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls 22.37 80.9±1.31µs ? ?/sec 1.00 3.6±0.06µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_true 24.60 85.6±1.27µs ? ?/sec 1.00 3.5±0.05µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/90pct_true 7.75 27.2±0.52µs ? ?/sec 1.00 3.5±0.06µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/99pct_true 1.78 4.7±0.05µs ? ?/sec 1.00 2.6±0.05µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_false 61.65 60.8±0.86µs ? ?/sec 1.00 987.0±21.69ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_true 1.10 1809.6±24.72ns ? ?/sec 1.00 1644.9±28.64ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/true_then_false 22.39 31.3±0.43µs ? ?/sec 1.00 1396.8±55.86ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/10pct_true 167.86 89.2±1.34µs ? ?/sec 1.00 531.7±10.53ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/1pct_true 156.29 83.2±1.08µs ? ?/sec 1.00 532.3±9.06ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/50pct_nulls 164.97 96.3±2.26µs ? ?/sec 1.00 583.9±7.88ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/50pct_true 180.66 96.0±1.50µs ? ?/sec 1.00 531.5±7.92ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/90pct_true 54.69 29.2±0.39µs ? ?/sec 1.00 534.6±7.03ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/99pct_true 9.53 5.1±0.08µs ? ?/sec 1.00 531.8±10.07ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/all_false 154.66 81.9±1.57µs ? ?/sec 1.00 529.7±6.98ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/all_true 3.56 1875.6±30.34ns ? ?/sec 1.00 527.0±9.79ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/true_then_false 79.80 41.8±0.77µs ? ?/sec 1.00 524.2±8.12ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/10pct_true 8.43 30.4±0.44µs ? ?/sec 1.00 3.6±0.06µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/1pct_true 2.54 6.8±0.10µs ? ?/sec 1.00 2.7±0.14µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls 16.46 59.1±1.10µs ? ?/sec 1.00 3.6±0.06µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_true 25.42 92.4±1.24µs ? ?/sec 1.00 3.6±0.10µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/90pct_true 20.84 74.5±1.17µs ? ?/sec 1.00 3.6±0.06µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/99pct_true 24.92 63.8±1.00µs ? ?/sec 1.00 2.6±0.05µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_false 1.13 1926.3±34.08ns ? ?/sec 1.00 1704.3±25.36ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_true 59.23 62.2±1.14µs ? ?/sec 1.00 1049.8±17.05ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/true_then_false 23.62 32.7±0.39µs ? ?/sec 1.00 1386.0±52.13ns ? ?/sec
```
</details>
<details><summary>Per-runner information</summary>
<details><summary>zip_kernels — shard 1/1</summary>
**Node:** gk3-benchmark-cluster-nap-10zn6ylp-8d12e677-rxgh
**Instance:** c4-highmem-16 (12 vCPU / 65 GiB)
**uname:**
```text
Linux bench-c6383511996-3823-r7nm2 6.12.105+ #1 SMP Sat Sep 5 07:57:24 UTC 2026 x86_64 GNU/Linux
```
**BENCH_COMMAND:**
```sh
cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
```
<details><summary>CPU Details (lscpu)</summary>
```text
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 52 bits physical, 57 bits virtual
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: GenuineIntel
Model name: INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz
CPU family: 6
Model: 207
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
Stepping: 2
BogoMIPS: 4600.00
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx_vnni avx512_bf16 wbnoinvd arat avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid cldemote movdiri movdir64b fsrm md_clear serialize tsxldtrk amx_bf16 avx512_fp16 amx_tile amx_int8 arch_capabilities
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 384 KiB (8 instances)
L1i cache: 256 KiB (8 instances)
L2 cache: 16 MiB (8 instances)
L3 cache: 260 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Resource Usage</summary>
**base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 530.1s |
| Peak memory | 32.8 MiB |
| Avg memory | 25.8 MiB |
| CPU user | 525.4s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
**branch**
| Metric | Value |
|--------|-------|
| Wall time | 525.1s |
| Peak memory | 32.9 MiB |
| Avg memory | 26.3 MiB |
| CPU user | 521.6s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
</details>
</details>
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6383511996:finish:479f8278ca9a49ebf909a09d7a8b0939 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4361268730
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return Ok(output);
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
Review Comment:
What happens here for an `Interval(DayTime)` array whose values buffer is only 4-byte aligned? arrow-rs requires a fixed-width buffer to be aligned only to `align_of` of its native type ([`new_fixed_width`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-data/src/data.rs#L2058-L2063)), and [`IntervalDayTime`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-buffer/src/interval.rs#L348-L355) is two `i32`s, so its alignment is 4. The Arrow format only [recommends 8- or 64-byte alignment](https://github.com/apache/arrow/blob/373d64c02cb23663b63c5853099fca43a46c39fc/docs/source/format/Columnar.rst?plain=1#L264-L273) in memory.
For such a buffer (I built one by slicing a `Buffer` 4 bytes in), `blend::<i64, 1>` returns `None`, and the fallback panics in `Buffer::typed_data`, because `MutableArrayData` [extends `DayTime` as `i64`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-data/src/transform/mod.rs#L259-L263). That panic also happens on `main` without this PR, so it needs its own fix. Could you file an issue for it?
For this PR, what do you think about blending `DayTime` as two `i32` lanes? The aligned case would then never fall back, and as I read it, the alignment check in `rows` would no longer fail for any type that reaches `blend`. Could the test also add `Interval(DayTime)` to `data_types`, plus one case with a 4-byte-aligned buffer? `random_primitive` always produces aligned buffers. This needs `IntervalUnit` in the top-level `arrow_schema` import.
```suggestion
8 if *array.data_type() == DataType::Interval(IntervalUnit::DayTime) => {
// arrow-rs aligns a `DayTime` buffer only to `align_of::<IntervalDayTime>()`, 4
blend::<i32, 2>(&from_array, &array, &scalar)?
}
8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
```
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return Ok(output);
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+ 16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+ 32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+ _ => return None,
+ }
+ } else {
+ array.buffers()[0].slice_with_length(array.offset() * width, len * width)
+ };
+
+ let data = ArrayData::builder(array.data_type().clone())
+ .len(len)
+ .add_buffer(values)
+ .nulls(nulls.map(NullBuffer::new));
+ Some(make_array(
+ data.build().expect("the layout of the input array"),
+ ))
+}
+
+/// An integer lane that [`blend`] selects with.
+trait BlendLane:
+ ArrowNativeType + BitAnd<Output = Self> + BitOr<Output = Self> + Not<Output = Self>
+{
+ /// `mask` (0 or -1) at this width.
+ fn from_mask(mask: i32) -> Self;
+}
+
+macro_rules! blend_lane {
+ ($($t:ty),*) => {
+ $(impl BlendLane for $t {
+ #[inline]
+ fn from_mask(mask: i32) -> Self {
+ mask as $t
+ }
+ })*
+ };
+}
+blend_lane!(i8, i16, i64);
+
+impl BlendLane for i32 {
+ #[inline]
+ fn from_mask(mask: i32) -> Self {
+ mask
+ }
+}
+
+/// For each bit of `word`, an `i32` with all bits set if it is set, else clear.
+#[inline]
+fn lane_masks(word: u64) -> [i32; 64] {
+ // Every lane of a byte tests a constant bit of the same value: an AND and a compare of
+ // 32-bit lanes, which SSE2 and NEON have. Shifting the word by a different count per lane
+ // has no SIMD form before AVX2, and SSE2 has no 64-bit compare.
Review Comment:
Did you check which instructions this compiles to? On rustc 1.99.1 release, I don't see an AND and a compare on either target. On aarch64, `lane_masks` uses the per-lane variable shift that the comment says has no SIMD form, plus scalar bit extracts:
```
ushl.4s v2, v1, v0 ; shift each lane by its own count
cmlt.4s v2, v2, #0 ; mask = lane < 0
sbfx w11, w8, #5, #1
```
On x86_64 (SSE2), it emulates the per-lane shift with multiplies, plus scalar `shl`/`sar`:
```
pmuludq xmm4, xmm3 # multiply by per-lane powers of two
psrad xmm8, 31 # mask = sign bit
```
Neither listing has a `pcmpeqd`, `cmeq` or `cmtst`. Both are branch-free, which is the property that matters here. Should the comment state that property instead, so that nobody later keeps the "AND and compare" form for a reason that doesn't hold?
```suggestion
// Each mask tests a constant bit of one byte of the word. This compiles without a branch per
// bit on x86_64 (SSE2) and aarch64.
```
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return Ok(output);
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+ 16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+ 32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+ _ => return None,
+ }
+ } else {
+ array.buffers()[0].slice_with_length(array.offset() * width, len * width)
+ };
+
+ let data = ArrayData::builder(array.data_type().clone())
+ .len(len)
+ .add_buffer(values)
+ .nulls(nulls.map(NullBuffer::new));
+ Some(make_array(
+ data.build().expect("the layout of the input array"),
+ ))
Review Comment:
Could this return the error instead of panicking? `zip` already returns a `Result`, and `PrimitiveScalarImpl` in this file propagates its build error ([line 633](https://github.com/apache/arrow-rs/blob/03bef9fd4622d8c78244b265a1653873e038d3de/arrow-select/src/zip.rs#L633)). If the function returns `Option<Result<ArrayRef, ArrowError>>` (line 171), the `?` on `blend` keeps working, and the caller at line 149 becomes `return output;`.
```suggestion
Some(data.build().map(make_array))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mightsleep (via GitHub)" <gi...@apache.org>.
mightsleep commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383181775
Yes hello, added them in a commit before the change: decimal128 (an array against a scalar on either side, and against a null scalar) and array_vs_null_scalar for every type. Could you run the bot on zip_kernels @mbutrovich?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383508283
🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383507725)
**Target:** zip_kernels
**Sharding factor: 1** (1 workers).
Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
env:
BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
shards: 1
```
</details>
Results will be posted when all workers finish.
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6383507725:start:4411e15c9be8f7789b94d24390bd300a -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383720375
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383519229)
**Target:** zip_kernels
Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
env:
BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
RUSTFLAGS: -C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6
shards: 1
```
</details>
<details><summary>Details</summary>
```
group base changed
----- ---- -------
zip_8192_from_decimal128/array_vs_non_null_scalar/10pct_true 10.72 80.9±0.30µs ? ?/sec 1.00 7.5±0.13µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/1pct_true 12.17 72.0±0.23µs ? ?/sec 1.00 5.9±0.03µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_nulls 11.60 88.6±0.33µs ? ?/sec 1.00 7.6±0.17µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_true 11.42 88.5±0.32µs ? ?/sec 1.00 7.7±0.19µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/90pct_true 4.07 30.9±0.35µs ? ?/sec 1.00 7.6±0.22µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/99pct_true 1.34 7.3±0.04µs ? ?/sec 1.00 5.5±0.07µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/all_false 30.79 70.6±0.28µs ? ?/sec 1.00 2.3±0.01µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/all_true 1.02 3.6±0.05µs ? ?/sec 1.00 3.5±0.06µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/true_then_false 12.86 37.5±0.14µs ? ?/sec 1.00 2.9±0.02µs ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/10pct_true 183.32 105.8±0.38µs ? ?/sec 1.00 577.0±3.43ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/1pct_true 170.61 98.4±0.36µs ? ?/sec 1.00 576.7±3.37ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/50pct_nulls 168.75 110.3±0.33µs ? ?/sec 1.00 653.4±3.94ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/50pct_true 178.74 103.0±0.27µs ? ?/sec 1.00 576.2±3.28ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/90pct_true 58.84 33.9±0.09µs ? ?/sec 1.00 576.4±3.33ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/99pct_true 13.34 7.7±0.02µs ? ?/sec 1.00 576.3±3.55ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/all_false 170.55 97.3±0.31µs ? ?/sec 1.00 570.5±2.96ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/all_true 6.37 3.6±0.05µs ? ?/sec 1.00 570.7±2.77ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/true_then_false 88.98 51.1±0.24µs ? ?/sec 1.00 574.1±3.08ns ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/10pct_true 4.84 37.1±0.42µs ? ?/sec 1.00 7.7±0.23µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/1pct_true 2.39 13.0±0.13µs ? ?/sec 1.00 5.5±0.05µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_nulls 8.69 66.4±0.74µs ? ?/sec 1.00 7.6±0.16µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_true 13.22 101.4±0.78µs ? ?/sec 1.00 7.7±0.24µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/90pct_true 11.38 88.8±0.28µs ? ?/sec 1.00 7.8±0.25µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/99pct_true 13.06 78.3±0.17µs ? ?/sec 1.00 6.0±0.02µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/all_false 1.07 3.8±0.06µs ? ?/sec 1.00 3.6±0.05µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/all_true 32.50 76.4±0.19µs ? ?/sec 1.00 2.4±0.01µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/true_then_false 14.12 42.4±0.19µs ? ?/sec 1.00 3.0±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/10pct_true 29.18 78.8±0.21µs ? ?/sec 1.00 2.7±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/1pct_true 33.24 71.7±0.27µs ? ?/sec 1.00 2.2±0.01µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls 31.11 86.9±0.11µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_true 31.83 86.0±0.30µs ? ?/sec 1.00 2.7±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/90pct_true 10.39 28.1±0.34µs ? ?/sec 1.00 2.7±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/99pct_true 2.39 5.1±0.03µs ? ?/sec 1.00 2.1±0.01µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_false 58.91 70.4±0.40µs ? ?/sec 1.00 1195.7±3.11ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_true 1.14 1714.4±9.54ns ? ?/sec 1.00 1502.3±4.41ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/true_then_false 26.67 36.2±0.12µs ? ?/sec 1.00 1357.9±4.53ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/10pct_true 176.41 103.6±0.29µs ? ?/sec 1.00 587.2±5.68ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/1pct_true 167.14 98.1±0.28µs ? ?/sec 1.00 587.2±5.71ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/50pct_nulls 163.62 107.5±0.22µs ? ?/sec 1.00 657.1±2.85ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/50pct_true 172.18 99.8±0.13µs ? ?/sec 1.00 579.4±3.06ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/90pct_true 52.26 30.7±0.05µs ? ?/sec 1.00 586.9±5.85ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/99pct_true 9.45 5.5±0.01µs ? ?/sec 1.00 579.9±3.07ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/all_false 168.79 97.0±0.28µs ? ?/sec 1.00 574.8±2.58ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/all_true 3.15 1806.9±5.63ns ? ?/sec 1.00 574.3±2.55ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/true_then_false 86.36 49.6±0.15µs ? ?/sec 1.00 574.3±2.62ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/10pct_true 11.33 31.4±0.35µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/1pct_true 3.34 7.5±0.03µs ? ?/sec 1.00 2.3±0.01µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls 21.11 61.1±0.48µs ? ?/sec 1.00 2.9±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_true 34.69 96.0±0.16µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/90pct_true 30.46 85.1±0.12µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/99pct_true 33.85 76.2±0.24µs ? ?/sec 1.00 2.3±0.01µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_false 1.15 1826.7±5.11ns ? ?/sec 1.00 1589.1±10.15ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_true 59.72 74.3±0.13µs ? ?/sec 1.00 1244.4±6.41ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/true_then_false 26.81 39.0±0.12µs ? ?/sec 1.00 1456.3±6.57ns ? ?/sec
```
</details>
<details><summary>Per-runner information</summary>
<details><summary>zip_kernels — shard 1/1</summary>
**Node:** gk3-benchmark-cluster-nap-2afsrt1r-6e04f20f-xqf5
**Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
**uname:**
```text
Linux bench-c6383519229-3824-qjqzd 6.12.105+ #1 SMP Sat Sep 5 07:54:11 UTC 2026 aarch64 GNU/Linux
```
**BENCH_COMMAND:**
```sh
cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
```
<details><summary>CPU Details (lscpu)</summary>
```text
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Resource Usage</summary>
**base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 530.1s |
| Peak memory | 34.1 MiB |
| Avg memory | 25.9 MiB |
| CPU user | 523.3s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
**branch**
| Metric | Value |
|--------|-------|
| Wall time | 535.1s |
| Peak memory | 32.3 MiB |
| Avg memory | 24.9 MiB |
| CPU user | 530.6s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
</details>
</details>
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6383519229:finish:0f1e58677ef2d11b365e28ccf179e575 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mightsleep (via GitHub) via github" <gi...@arrow.apache.org>.
mightsleep commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4485722953
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,174 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return output;
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<Result<ArrayRef, ArrowError>> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+ 16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+ 32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+ _ => return None,
+ }
+ } else {
+ array.buffers()[0].slice_with_length(array.offset() * width, len * width)
Review Comment:
yep, will do in a follow-up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mightsleep (via GitHub)" <gi...@apache.org>.
mightsleep commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4363361523
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return Ok(output);
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+ 16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+ 32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+ _ => return None,
+ }
+ } else {
+ array.buffers()[0].slice_with_length(array.offset() * width, len * width)
+ };
+
+ let data = ArrayData::builder(array.data_type().clone())
+ .len(len)
+ .add_buffer(values)
+ .nulls(nulls.map(NullBuffer::new));
+ Some(make_array(
+ data.build().expect("the layout of the input array"),
+ ))
Review Comment:
Makes sense, applied. It can't fail for a layout taken from a valid input, but propagating the error matches PrimitiveScalarImpl in the same file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6367490152
run benchmark zip_kernels
env:
BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383713453
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383509349)
**Target:** zip_kernels
Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
env:
BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
resources:
arch: amd64
shards: 1
```
</details>
<details><summary>Details</summary>
```
group base changed
----- ---- -------
zip_8192_from_decimal128/array_vs_non_null_scalar/10pct_true 11.50 87.7±2.27µs ? ?/sec 1.00 7.6±0.19µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/1pct_true 12.52 73.2±1.65µs ? ?/sec 1.00 5.8±0.10µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_nulls 10.83 86.4±4.01µs ? ?/sec 1.00 8.0±0.43µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_true 13.65 104.0±3.02µs ? ?/sec 1.00 7.6±0.16µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/90pct_true 4.44 34.0±1.14µs ? ?/sec 1.00 7.7±0.22µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/99pct_true 1.34 8.6±0.28µs ? ?/sec 1.00 6.4±0.11µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/all_false 21.70 64.5±3.13µs ? ?/sec 1.00 3.0±0.10µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/all_true 1.26 6.3±0.17µs ? ?/sec 1.00 5.0±0.20µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/true_then_false 9.75 40.5±1.39µs ? ?/sec 1.00 4.2±0.08µs ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/10pct_true 176.44 94.6±1.82µs ? ?/sec 1.00 536.2±9.16ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/1pct_true 157.79 84.4±3.37µs ? ?/sec 1.00 534.9±12.04ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/50pct_nulls 175.73 103.3±4.66µs ? ?/sec 1.00 588.0±17.29ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/50pct_true 214.40 117.0±2.94µs ? ?/sec 1.00 545.5±26.50ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/90pct_true 68.81 36.8±1.04µs ? ?/sec 1.00 535.0±5.58ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/99pct_true 17.31 9.3±0.23µs ? ?/sec 1.00 536.1±7.74ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/all_false 159.26 84.5±2.82µs ? ?/sec 1.00 530.4±3.77ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/all_true 11.62 6.2±0.21µs ? ?/sec 1.00 537.4±25.94ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/true_then_false 84.50 45.0±1.51µs ? ?/sec 1.00 532.2±8.31ns ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/10pct_true 5.41 41.4±1.62µs ? ?/sec 1.00 7.7±0.14µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/1pct_true 2.29 15.2±0.63µs ? ?/sec 1.00 6.6±0.37µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_nulls 9.72 77.1±1.05µs ? ?/sec 1.00 7.9±0.05µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_true 13.03 99.7±3.94µs ? ?/sec 1.00 7.6±0.12µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/90pct_true 10.93 83.7±3.08µs ? ?/sec 1.00 7.7±0.16µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/99pct_true 12.30 72.4±4.02µs ? ?/sec 1.00 5.9±0.08µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/all_false 1.23 6.4±0.16µs ? ?/sec 1.00 5.2±0.10µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/all_true 22.54 67.8±2.26µs ? ?/sec 1.00 3.0±0.05µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/true_then_false 11.16 46.2±1.18µs ? ?/sec 1.00 4.1±0.04µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/10pct_true 18.02 78.4±6.23µs ? ?/sec 1.00 4.3±0.14µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/1pct_true 22.31 67.8±5.24µs ? ?/sec 1.00 3.0±0.07µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls 19.94 88.6±5.69µs ? ?/sec 1.00 4.4±0.11µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_true 20.86 88.8±3.14µs ? ?/sec 1.00 4.3±0.14µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/90pct_true 6.54 28.3±1.16µs ? ?/sec 1.00 4.3±0.14µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/99pct_true 1.55 5.0±0.24µs ? ?/sec 1.00 3.2±0.09µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_false 52.79 65.2±5.03µs ? ?/sec 1.00 1234.9±27.13ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_true 1.09 1903.8±39.32ns ? ?/sec 1.00 1748.0±48.83ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/true_then_false 18.99 33.1±2.51µs ? ?/sec 1.00 1743.9±52.35ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/10pct_true 143.44 94.6±4.33µs ? ?/sec 1.00 659.5±12.25ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/1pct_true 157.36 86.2±2.85µs ? ?/sec 1.00 548.0±15.02ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/50pct_nulls 203.39 120.9±3.73µs ? ?/sec 1.00 594.7±19.63ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/50pct_true 155.84 100.2±4.43µs ? ?/sec 1.00 642.7±15.67ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/90pct_true 46.85 30.2±0.83µs ? ?/sec 1.00 645.2±17.51ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/99pct_true 8.83 5.7±0.50µs ? ?/sec 1.00 645.4±15.37ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/all_false 172.04 92.9±8.26µs ? ?/sec 1.00 540.2±17.44ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/all_true 3.64 2.3±0.06µs ? ?/sec 1.00 634.2±15.52ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/true_then_false 71.45 45.0±2.28µs ? ?/sec 1.00 630.1±18.81ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/10pct_true 9.62 36.3±1.14µs ? ?/sec 1.00 3.8±0.14µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/1pct_true 2.98 8.3±0.20µs ? ?/sec 1.00 2.8±0.07µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls 15.98 72.3±2.37µs ? ?/sec 1.00 4.5±0.10µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_true 30.28 114.8±2.34µs ? ?/sec 1.00 3.8±0.17µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/90pct_true 24.49 91.7±4.48µs ? ?/sec 1.00 3.7±0.10µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/99pct_true 26.97 72.1±6.30µs ? ?/sec 1.00 2.7±0.12µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_false 1.18 2.5±0.09µs ? ?/sec 1.00 2.1±0.05µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_true 51.03 68.0±4.84µs ? ?/sec 1.00 1332.5±32.56ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/true_then_false 26.04 39.8±1.49µs ? ?/sec 1.00 1528.1±39.62ns ? ?/sec
```
</details>
<details><summary>Per-runner information</summary>
<details><summary>zip_kernels — shard 1/1</summary>
**Node:** gk3-benchmark-cluster-nap-10zn6ylp-79de5834-b7fn
**Instance:** c4-highmem-16 (12 vCPU / 65 GiB)
**uname:**
```text
Linux bench-c6383509349-3822-2cwms 6.12.105+ #1 SMP Sat Sep 5 07:57:24 UTC 2026 x86_64 GNU/Linux
```
**BENCH_COMMAND:**
```sh
cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
```
<details><summary>CPU Details (lscpu)</summary>
```text
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 52 bits physical, 57 bits virtual
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: GenuineIntel
Model name: INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz
CPU family: 6
Model: 207
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
Stepping: 2
BogoMIPS: 4600.00
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx_vnni avx512_bf16 wbnoinvd arat avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid cldemote movdiri movdir64b fsrm md_clear serialize tsxldtrk amx_bf16 avx512_fp16 amx_tile amx_int8 arch_capabilities
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 384 KiB (8 instances)
L1i cache: 256 KiB (8 instances)
L2 cache: 16 MiB (8 instances)
L3 cache: 260 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Resource Usage</summary>
**base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 535.1s |
| Peak memory | 31.7 MiB |
| Avg memory | 24.8 MiB |
| CPU user | 532.4s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
**branch**
| Metric | Value |
|--------|-------|
| Wall time | 525.1s |
| Peak memory | 31.7 MiB |
| Avg memory | 24.8 MiB |
| CPU user | 520.6s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
</details>
</details>
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6383509349:finish:b99b3b4829c4e04cbb569a059e4f362c -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6186136950
run benchmark zip_kernels
env:
BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383708566
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383507725)
**Target:** zip_kernels
Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
env:
BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
shards: 1
```
</details>
<details><summary>Details</summary>
```
group base changed
----- ---- -------
zip_8192_from_decimal128/array_vs_non_null_scalar/10pct_true 10.34 82.6±0.76µs ? ?/sec 1.00 8.0±0.10µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/1pct_true 11.60 73.6±0.33µs ? ?/sec 1.00 6.3±0.07µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_nulls 10.91 89.2±0.37µs ? ?/sec 1.00 8.2±0.27µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_true 11.41 91.9±0.58µs ? ?/sec 1.00 8.1±0.17µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/90pct_true 4.20 33.5±0.41µs ? ?/sec 1.00 8.0±0.06µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/99pct_true 1.25 7.6±0.10µs ? ?/sec 1.00 6.1±0.07µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/all_false 31.15 71.8±0.28µs ? ?/sec 1.00 2.3±0.01µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/all_true 1.07 4.6±0.09µs ? ?/sec 1.00 4.3±0.08µs ? ?/sec
zip_8192_from_decimal128/array_vs_non_null_scalar/true_then_false 12.44 39.5±0.17µs ? ?/sec 1.00 3.2±0.06µs ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/10pct_true 187.09 108.5±0.35µs ? ?/sec 1.00 579.8±3.68ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/1pct_true 171.53 99.6±0.36µs ? ?/sec 1.00 580.7±3.85ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/50pct_nulls 171.73 114.4±0.32µs ? ?/sec 1.00 665.9±4.66ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/50pct_true 183.71 106.5±0.30µs ? ?/sec 1.00 579.9±3.82ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/90pct_true 62.60 36.3±0.76µs ? ?/sec 1.00 580.1±3.63ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/99pct_true 13.69 8.0±0.09µs ? ?/sec 1.00 583.3±3.67ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/all_false 170.70 98.4±0.35µs ? ?/sec 1.00 576.7±3.34ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/all_true 6.64 3.8±0.23µs ? ?/sec 1.00 577.9±3.69ns ? ?/sec
zip_8192_from_decimal128/array_vs_null_scalar/true_then_false 92.13 53.1±0.27µs ? ?/sec 1.00 576.3±3.42ns ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/10pct_true 5.01 40.6±0.35µs ? ?/sec 1.00 8.1±0.14µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/1pct_true 2.45 14.8±0.31µs ? ?/sec 1.00 6.0±0.08µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_nulls 9.01 69.7±0.52µs ? ?/sec 1.00 7.7±0.35µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_true 13.14 105.8±0.40µs ? ?/sec 1.00 8.1±0.12µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/90pct_true 11.42 92.1±0.32µs ? ?/sec 1.00 8.1±0.09µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/99pct_true 12.44 79.5±0.45µs ? ?/sec 1.00 6.4±0.05µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/all_false 1.22 4.8±0.10µs ? ?/sec 1.00 3.9±0.22µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/all_true 33.14 78.0±0.33µs ? ?/sec 1.00 2.4±0.01µs ? ?/sec
zip_8192_from_decimal128/non_null_scalar_vs_array/true_then_false 14.83 45.2±0.27µs ? ?/sec 1.00 3.0±0.06µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/10pct_true 28.85 79.9±0.36µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/1pct_true 33.58 72.8±0.33µs ? ?/sec 1.00 2.2±0.01µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls 30.28 87.2±0.24µs ? ?/sec 1.00 2.9±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_true 31.11 86.3±0.36µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/90pct_true 10.20 28.1±0.27µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/99pct_true 2.36 5.1±0.03µs ? ?/sec 1.00 2.2±0.01µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_false 59.24 71.0±0.33µs ? ?/sec 1.00 1198.5±5.87ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_true 1.15 1749.0±20.69ns ? ?/sec 1.00 1519.2±8.10ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/true_then_false 27.48 37.0±0.23µs ? ?/sec 1.00 1346.1±7.02ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/10pct_true 177.17 103.9±0.28µs ? ?/sec 1.00 586.2±7.80ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/1pct_true 167.97 98.5±0.31µs ? ?/sec 1.00 586.3±8.05ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/50pct_nulls 162.26 107.8±0.37µs ? ?/sec 1.00 664.1±4.12ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/50pct_true 171.62 100.6±0.41µs ? ?/sec 1.00 586.0±7.76ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/90pct_true 52.66 30.9±0.09µs ? ?/sec 1.00 585.9±8.09ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/99pct_true 9.32 5.5±0.02µs ? ?/sec 1.00 586.4±8.09ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/all_false 168.68 97.9±0.25µs ? ?/sec 1.00 580.4±7.75ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/all_true 3.06 1787.4±9.07ns ? ?/sec 1.00 583.5±8.63ns ? ?/sec
zip_8192_from_i32/array_vs_null_scalar/true_then_false 86.93 50.4±0.21µs ? ?/sec 1.00 579.9±7.82ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/10pct_true 11.15 31.6±0.33µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/1pct_true 3.38 7.6±0.03µs ? ?/sec 1.00 2.2±0.01µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls 20.89 61.5±0.40µs ? ?/sec 1.00 2.9±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_true 34.14 96.8±0.37µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/90pct_true 30.06 85.8±0.29µs ? ?/sec 1.00 2.9±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/99pct_true 33.91 77.0±0.26µs ? ?/sec 1.00 2.3±0.01µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_false 1.20 1910.0±7.78ns ? ?/sec 1.00 1593.1±6.78ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_true 59.77 74.8±0.21µs ? ?/sec 1.00 1250.8±6.85ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/true_then_false 26.74 39.4±0.18µs ? ?/sec 1.00 1472.2±6.32ns ? ?/sec
```
</details>
<details><summary>Per-runner information</summary>
<details><summary>zip_kernels — shard 1/1</summary>
**Node:** gk3-benchmark-cluster-nap-hr1hsdiv-b6316889-j9b7
**Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
**uname:**
```text
Linux bench-c6383507725-3821-4rhsq 6.12.105+ #1 SMP Sat Sep 5 07:54:11 UTC 2026 aarch64 GNU/Linux
```
**BENCH_COMMAND:**
```sh
cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
```
<details><summary>CPU Details (lscpu)</summary>
```text
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Resource Usage</summary>
**base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 530.1s |
| Peak memory | 33.2 MiB |
| Avg memory | 25.0 MiB |
| CPU user | 527.3s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
**branch**
| Metric | Value |
|--------|-------|
| Wall time | 535.1s |
| Peak memory | 31.1 MiB |
| Avg memory | 23.6 MiB |
| CPU user | 530.6s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
</details>
</details>
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6383507725:finish:eb445b066884d247ad59304b9895dc8b -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383509349
run benchmark zip_kernels
```
env:
BENCH_FILTER: "from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)"
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
resources:
arch: amd64
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6367929060
🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6367490152)
**Target:** zip_kernels
Comparing zip-scalar-blend (03bef9fd4622d8c78244b265a1653873e038d3de) to df4dc3e (merge-base) [diff](https://github.com/apache/arrow-rs/compare/df4dc3ede59bd43b5f71847231212381d068bb72..03bef9fd4622d8c78244b265a1653873e038d3de)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
env:
BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array
shards: 1
```
</details>
<details><summary>Details</summary>
```
group base changed
----- ---- -------
zip_8192_from_i32/array_vs_non_null_scalar/10pct_true 28.38 79.0±0.22µs ? ?/sec 1.00 2.8±0.03µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/1pct_true 32.39 72.0±0.29µs ? ?/sec 1.00 2.2±0.01µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls 29.92 86.5±0.15µs ? ?/sec 1.00 2.9±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/50pct_true 30.72 85.9±0.34µs ? ?/sec 1.00 2.8±0.03µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/90pct_true 10.15 27.9±0.27µs ? ?/sec 1.00 2.7±0.02µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/99pct_true 2.32 5.1±0.03µs ? ?/sec 1.00 2.2±0.01µs ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_false 58.25 70.4±0.29µs ? ?/sec 1.00 1209.3±3.18ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/all_true 1.09 1717.7±5.66ns ? ?/sec 1.00 1579.3±5.19ns ? ?/sec
zip_8192_from_i32/array_vs_non_null_scalar/true_then_false 25.39 36.3±0.14µs ? ?/sec 1.00 1431.0±4.14ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/10pct_true 11.11 31.4±0.29µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/1pct_true 3.31 7.6±0.02µs ? ?/sec 1.00 2.3±0.01µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls 20.88 61.4±0.44µs ? ?/sec 1.00 2.9±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/50pct_true 33.93 96.7±0.26µs ? ?/sec 1.00 2.8±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/90pct_true 29.75 85.2±0.25µs ? ?/sec 1.00 2.9±0.02µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/99pct_true 32.72 76.1±0.26µs ? ?/sec 1.00 2.3±0.01µs ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_false 1.13 1872.4±6.10ns ? ?/sec 1.00 1659.2±4.59ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/all_true 59.03 74.4±0.26µs ? ?/sec 1.00 1260.2±5.62ns ? ?/sec
zip_8192_from_i32/non_null_scalar_vs_array/true_then_false 25.35 39.1±0.15µs ? ?/sec 1.00 1542.6±3.71ns ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/10pct_true 1.00 96.2±0.40µs ? ?/sec 1.00 95.7±0.38µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/1pct_true 1.00 89.2±0.37µs ? ?/sec 1.00 89.1±0.54µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/50pct_nulls 1.00 103.3±0.23µs ? ?/sec 1.00 103.3±0.33µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/50pct_true 1.00 102.3±0.28µs ? ?/sec 1.00 102.6±0.37µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/90pct_true 1.00 40.3±0.30µs ? ?/sec 1.00 40.3±0.32µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/99pct_true 1.00 13.2±0.06µs ? ?/sec 1.01 13.2±0.06µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/all_false 1.00 88.4±0.41µs ? ?/sec 1.00 88.0±0.62µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/all_true 1.00 8.8±0.04µs ? ?/sec 1.01 8.9±0.07µs ? ?/sec
zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/true_then_false 1.01 48.8±0.19µs ? ?/sec 1.00 48.3±0.26µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/10pct_true 1.00 46.0±0.37µs ? ?/sec 1.00 45.9±0.39µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/1pct_true 1.00 18.0±0.05µs ? ?/sec 1.00 17.9±0.05µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/50pct_nulls 1.00 78.4±0.62µs ? ?/sec 1.00 78.2±0.62µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/50pct_true 1.00 116.1±0.37µs ? ?/sec 1.00 116.1±0.28µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/90pct_true 1.00 105.2±0.30µs ? ?/sec 1.00 105.0±0.38µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/99pct_true 1.00 95.9±0.35µs ? ?/sec 1.00 95.5±0.35µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/all_false 1.04 9.4±0.27µs ? ?/sec 1.00 9.0±0.04µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/all_true 1.00 94.2±0.40µs ? ?/sec 1.00 93.9±0.34µs ? ?/sec
zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/true_then_false 1.01 53.6±0.31µs ? ?/sec 1.00 53.3±0.20µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/10pct_true 1.00 119.8±0.54µs ? ?/sec 1.00 119.9±0.66µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/1pct_true 1.00 110.9±0.71µs ? ?/sec 1.00 111.0±0.86µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/50pct_nulls 1.00 126.4±0.39µs ? ?/sec 1.00 126.7±0.48µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/50pct_true 1.00 123.2±0.49µs ? ?/sec 1.00 123.5±0.53µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/90pct_true 1.00 49.2±0.34µs ? ?/sec 1.00 49.3±0.40µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/99pct_true 1.00 16.3±0.04µs ? ?/sec 1.02 16.6±0.04µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/all_false 1.00 109.9±0.62µs ? ?/sec 1.00 109.3±0.83µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/all_true 1.00 8.6±0.02µs ? ?/sec 1.01 8.7±0.03µs ? ?/sec
zip_8192_from_strings (3..20)/array_vs_non_null_scalar/true_then_false 1.00 60.6±0.36µs ? ?/sec 1.00 60.4±0.38µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/10pct_true 1.01 54.6±0.39µs ? ?/sec 1.00 54.3±0.45µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/1pct_true 1.02 20.5±0.06µs ? ?/sec 1.00 20.0±0.04µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/50pct_nulls 1.01 94.2±0.64µs ? ?/sec 1.00 93.7±0.73µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/50pct_true 1.01 139.4±0.35µs ? ?/sec 1.00 138.6±0.47µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/90pct_true 1.01 132.4±0.46µs ? ?/sec 1.00 131.5±0.62µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/99pct_true 1.01 119.0±0.53µs ? ?/sec 1.00 118.3±0.72µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/all_false 1.00 8.7±0.03µs ? ?/sec 1.01 8.8±0.03µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/all_true 1.00 116.0±0.56µs ? ?/sec 1.00 116.5±0.68µs ? ?/sec
zip_8192_from_strings (3..20)/non_null_scalar_vs_array/true_then_false 1.00 65.7±0.29µs ? ?/sec 1.00 65.7±0.34µs ? ?/sec
```
</details>
<details><summary>Per-runner information</summary>
<details><summary>zip_kernels — shard 1/1</summary>
**Node:** gk3-benchmark-cluster-nap-144p8rlm-9d0d7b21-v79b
**Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
**uname:**
```text
Linux bench-c6367490152-3793-v7fsq 6.12.105+ #1 SMP Sat Sep 5 07:54:11 UTC 2026 aarch64 GNU/Linux
```
**BENCH_COMMAND:**
```sh
cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
```
<details><summary>CPU Details (lscpu)</summary>
```text
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Resource Usage</summary>
**base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 525.1s |
| Peak memory | 38.6 MiB |
| Avg memory | 28.6 MiB |
| CPU user | 521.4s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
**branch**
| Metric | Value |
|--------|-------|
| Wall time | 525.1s |
| Peak memory | 38.6 MiB |
| Avg memory | 28.7 MiB |
| CPU user | 521.6s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
</details>
</details>
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6367490152:finish:74eeca4f2bf8f8caf6c151f95396b609 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "Jefffrey (via GitHub) via github" <gi...@arrow.apache.org>.
Jefffrey merged PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383509942
🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383509349)
**Target:** zip_kernels
**Sharding factor: 1** (1 workers).
Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
env:
BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
resources:
arch: amd64
shards: 1
```
</details>
Results will be posted when all workers finish.
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6383509349:start:a537b171920a9813f5ca4fe83c8ee429 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4365060164
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return Ok(output);
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
Review Comment:
Sounds good!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6367492342
🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6367490152)
**Target:** zip_kernels
**Sharding factor: 1** (1 workers).
Comparing zip-scalar-blend (03bef9fd4622d8c78244b265a1653873e038d3de) to df4dc3e (merge-base) [diff](https://github.com/apache/arrow-rs/compare/df4dc3ede59bd43b5f71847231212381d068bb72..03bef9fd4622d8c78244b265a1653873e038d3de)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
env:
BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array
shards: 1
```
</details>
Results will be posted when all workers finish.
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6367490152:start:8d227a232276e9ddfee246398f40930c -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4361462702
##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
));
}
+ if truthy_is_scalar != falsy_is_scalar {
+ let (array, scalar) = if truthy_is_scalar {
+ (falsy, truthy)
+ } else {
+ (truthy, falsy)
+ };
+ if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+ return Ok(output);
+ }
+ }
+
let falsy = falsy.to_data();
let truthy = truthy.to_data();
zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
}
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+ mask: &BooleanArray,
+ array: &dyn Array,
+ scalar: &dyn Array,
+ scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+ let width = array.data_type().primitive_width()?;
+ let array = array.to_data();
+ let scalar = scalar.to_data();
+ let len = array.len();
+
+ // The rows that take the array's value
+ let mut from_array = maybe_prep_null_mask_filter(mask);
+ if scalar_is_truthy {
+ from_array = !&from_array;
+ }
+
+ let scalar_valid = scalar.is_valid(0);
+ let nulls = match (scalar_valid, array.nulls()) {
+ (true, None) => None,
+ (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+ (false, None) => Some(from_array.clone()),
+ (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+ };
+
+ let values = if scalar_valid {
+ match width {
+ 1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+ 2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+ 4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+ 8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
Review Comment:
Filed https://github.com/apache/arrow-rs/issues/11862.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mightsleep (via GitHub) via github" <gi...@arrow.apache.org>.
mightsleep commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6670111217
Yes hello, merged up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6186138278
🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6186136950)
**Target:** zip_kernels
**Sharding factor: 1** (1 workers).
Comparing zip-scalar-blend (deecbb71446efd61c0963b5f5802731b894edc10) to 4dc6bc2 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/4dc6bc25f351dd0b8474339ca342d488723f1368..deecbb71446efd61c0963b5f5802731b894edc10)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
env:
BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array
shards: 1
```
</details>
Results will be posted when all workers finish.
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6186136950:start:7cc9556a794cb9d3bb3b8ed1e621ffcd -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383512513
🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383511996)
**Target:** zip_kernels
**Sharding factor: 1** (1 workers).
Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
<details><summary>Run configuration</summary>
```yaml
run benchmark zip_kernels
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
env:
BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
RUSTFLAGS: -C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6
resources:
arch: amd64
shards: 1
```
</details>
Results will be posted when all workers finish.
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
<!-- benchmark-request:6383511996:start:0e7339e0b52f432c1e07a2b971a7fd6f -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383511996
run benchmark zip_kernels
```
env:
BENCH_FILTER: "from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)"
RUSTFLAGS: "-C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6"
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
resources:
arch: amd64
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383519229
run benchmark zip_kernels
```
env:
BENCH_FILTER: "from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)"
RUSTFLAGS: "-C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6"
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383507725
run benchmark zip_kernels
```
env:
BENCH_FILTER: "from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)"
baseline:
ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]
Posted by "Jefffrey (via GitHub) via github" <gi...@arrow.apache.org>.
Jefffrey commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6671153796
thanks, some great numbers 🚀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org