You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "mightsleep (via GitHub)" <gi...@apache.org> on 2026/10/14 16:27:54 UTC

[PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

mightsleep opened a new pull request, #11559:
URL: https://github.com/apache/arrow-rs/pull/11559

   # Which issue does this PR close?
   
   - Part of #11437.
   
   # Rationale for this change
   
   `zip` with a scalar on one side extends `MutableArrayData` once per scalar row and once per run of array rows, 6 to 7 ns a row on random masks. Callers include DataFusion's Spark `%` and `pmod`, `unnest` and `union_extract`.
   
   # What changes are included in this PR?
   
   For a primitive array and a scalar of the same type:
   
   - Values are a branch-free select per row over blocks of 64 rows. Mask words that are all set or all clear copy or fill. 16 and 32 byte values are 2 or 4 `i64` lanes.
   - The lane masks are computed into an array before the select. Computed inside it, LLVM emits a branch per row, which mispredicts on random masks.
   - Nulls in closed form. With a null scalar the array's values buffer is shared.
   
   Other types keep the `MutableArrayData` path.
   
   # Are these changes tested?
   
   `test_zip_primitive_with_scalar_matches_general_path` compares against `zip_impl` for 8 types (including `Decimal256` and `Interval(MonthDayNano)`), with offsets, nulls in the array and the mask, a null and a valid scalar, on either side.
   
   ns per row, 8192 rows, a pool of 512 random masks so the branch predictor cannot learn them:
   
   | type | main | this PR |
   |---|---|---|
   | i8 | 6.0 to 7.3 | 0.32 |
   | i32 | 6.3 to 7.4 | 0.30 |
   | f64 | 6.7 to 7.6 | 0.38 |
   | Decimal128 | 6.3 to 7.4 | 0.66 |
   | i32, null scalar | 9 to 10 | 0.05 |
   
   `zip_kernels` (i32) on GitHub runners (Neoverse-N2, Apple M1, EPYC 9V74 default and x86-64-v3), main / this PR: 22 to 60x on random masks, 1.07 to 1.35x when every row comes from the array.
   
   # Are there any significant user-facing changes?
   
   No.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6186457442

   🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6186136950)
   
   **Target:** zip&#95;kernels
   
   Comparing zip-scalar-blend (deecbb71446efd61c0963b5f5802731b894edc10) to 4dc6bc2 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/4dc6bc25f351dd0b8474339ca342d488723f1368..deecbb71446efd61c0963b5f5802731b894edc10)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   env:
     BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array
   shards: 1
   
   ```
   
   </details>
   
   <details><summary>Details</summary>
   
   ```
   group                                                                               base                                   changed
   -----                                                                               ----                                   -------
   zip_8192_from_i32/array_vs_non_null_scalar/10pct_true                               30.47    77.7±0.10µs        ? ?/sec    1.00      2.5±0.01µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/1pct_true                                41.44    71.9±0.30µs        ? ?/sec    1.00   1734.7±6.50ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls                              30.28    81.8±0.16µs        ? ?/sec    1.00      2.7±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_true                               30.47    78.1±0.20µs        ? ?/sec    1.00      2.6±0.01µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/90pct_true                               8.72     22.4±0.25µs        ? ?/sec    1.00      2.6±0.01µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/99pct_true                               2.26      4.2±0.03µs        ? ?/sec    1.00   1860.7±4.72ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_false                                59.08    70.7±0.13µs        ? ?/sec    1.00   1196.8±9.10ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_true                                 1.25   1533.3±5.02ns        ? ?/sec    1.00   1224.9±7.28ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/true_then_false                          28.54    36.2±0.07µs        ? ?/sec    1.00   1267.9±7.48ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/10pct_true                               9.06     24.0±0.26µs        ? ?/sec    1.00      2.6±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/1pct_true                                3.05      5.6±0.02µs        ? ?/sec    1.00   1828.1±6.64ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls                              18.89    51.5±0.31µs        ? ?/sec    1.00      2.7±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_true                               32.10    85.1±0.20µs        ? ?/sec    1.00      2.7±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/90pct_true                               30.86    82.6±0.10µs        ? ?/sec    1.00      2.7±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/99pct_true                               38.83    76.3±0.13µs        ? ?/sec    1.00   1964.3±7.44ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_false                                1.17   1590.6±7.09ns        ? ?/sec    1.00   1360.4±7.39ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_true                                 57.84    74.7±0.26µs        ? ?/sec    1.00   1291.0±9.18ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/true_then_false                          29.20    39.2±0.15µs        ? ?/sec    1.00   1342.4±7.96ns        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/10pct_true         1.01     95.3±0.54µs        ? ?/sec    1.00     94.2±0.32µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/1pct_true          1.01     89.6±0.67µs        ? ?/sec    1.00     88.7±0.42µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/50pct_nulls        1.01     99.2±0.48µs        ? ?/sec    1.00     97.8±0.29µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/50pct_true         1.04     94.7±0.50µs        ? ?/sec    1.00     91.4±0.30µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/90pct_true         1.01     33.1±0.28µs        ? ?/sec    1.00     32.8±0.27µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/99pct_true         1.00     11.9±0.04µs        ? ?/sec    1.00     11.9±0.04µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/all_false          1.01     88.5±0.62µs        ? ?/sec    1.00     87.6±0.38µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/all_true           1.01      8.7±0.02µs        ? ?/sec    1.00      8.6±0.01µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/true_then_false    1.01     48.6±0.34µs        ? ?/sec    1.00     48.3±0.28µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/10pct_true         1.01     36.7±0.25µs        ? ?/sec    1.00     36.5±0.25µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/1pct_true          1.00     15.2±0.04µs        ? ?/sec    1.00     15.1±0.04µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/50pct_nulls        1.03     69.2±0.42µs        ? ?/sec    1.00     66.9±0.32µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/50pct_true         1.04    106.1±0.47µs        ? ?/sec    1.00    102.2±0.32µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/90pct_true         1.01    102.8±0.50µs        ? ?/sec    1.00    101.9±0.36µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/99pct_true         1.01     96.5±0.56µs        ? ?/sec    1.00     95.7±0.37µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/all_false          1.01      8.9±0.08µs        ? ?/sec    1.00      8.8±0.01µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/all_true           1.01     95.1±0.57µs        ? ?/sec    1.00     94.1±0.39µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/true_then_false    1.02     54.0±0.32µs        ? ?/sec    1.00     53.0±0.22µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/10pct_true                   1.01    118.5±0.59µs        ? ?/sec    1.00    117.9±0.61µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/1pct_true                    1.01    110.9±0.72µs        ? ?/sec    1.00    110.1±0.63µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/50pct_nulls                  1.01    122.5±0.50µs        ? ?/sec    1.00    121.7±0.51µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/50pct_true                   1.02    111.6±0.42µs        ? ?/sec    1.00    109.3±0.36µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/90pct_true                   1.01     38.5±0.27µs        ? ?/sec    1.00     38.0±0.24µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/99pct_true                   1.00     14.4±0.04µs        ? ?/sec    1.00     14.5±0.03µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/all_false                    1.01    109.6±0.63µs        ? ?/sec    1.00    109.0±0.62µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/all_true                     1.00      8.8±0.01µs        ? ?/sec    1.00      8.8±0.01µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/true_then_false              1.01     60.5±0.40µs        ? ?/sec    1.00     59.7±0.38µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/10pct_true                   1.00     41.3±0.33µs        ? ?/sec    1.02     42.2±0.23µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/1pct_true                    1.00     15.9±0.05µs        ? ?/sec    1.02     16.3±0.05µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/50pct_nulls                  1.02     77.0±0.40µs        ? ?/sec    1.00     75.8±0.41µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/50pct_true                   1.02    124.0±0.52µs        ? ?/sec    1.00    121.5±0.40µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/90pct_true                   1.00    130.5±0.62µs        ? ?/sec    1.00    130.3±0.56µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/99pct_true                   1.01    119.8±0.57µs        ? ?/sec    1.00    118.6±0.51µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/all_false                    1.00      8.9±0.03µs        ? ?/sec    1.00      8.8±0.01µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/all_true                     1.01    117.1±0.57µs        ? ?/sec    1.00    116.4±0.49µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/true_then_false              1.00     65.5±0.30µs        ? ?/sec    1.00     65.7±0.25µs        ? ?/sec
   
   ```
   
   </details>
   
   <details><summary>Per-runner information</summary>
   
   <details><summary>zip&#95;kernels — shard 1/1</summary>
   
   **Node:** gk3-benchmark-cluster-nap-671s0p7k-bef850df-xlxs
   
   **Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
   
   **uname:**
   
   ```text
   Linux bench-c6186136950-3433-c2x8g 6.12.94+ #1 SMP Fri Aug 21 08:00:16 UTC 2026 aarch64 GNU/Linux
   ```
   
   **BENCH_COMMAND:**
   
   ```sh
   cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
   ```
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```text
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 525.1s |
   | Peak memory | 14.3 MiB |
   | Avg memory | 13.6 MiB |
   | CPU user | 519.4s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 530.1s |
   | Peak memory | 14.7 MiB |
   | Avg memory | 14.1 MiB |
   | CPU user | 526.6s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   
   </details>
   
   </details>
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6186136950:finish:890393730d3f4feb709d0eee12eb8e53 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383519765

   🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383519229)
   
   **Target:** zip&#95;kernels
   
   **Sharding factor: 1** (1 workers).
   
   Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   env:
     BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
     RUSTFLAGS: -C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6
   shards: 1
   
   ```
   
   </details>
   
   Results will be posted when all workers finish.
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6383519229:start:67d0738b1ec27686058081579ff388af -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383510708

   ```
   env:
     BENCH_FILTER: "from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)"
     RUSTFLAGS: "-C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6"
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mightsleep (via GitHub)" <gi...@apache.org>.
mightsleep commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4363377028


##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return Ok(output);
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+            16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+            32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+            _ => return None,
+        }
+    } else {
+        array.buffers()[0].slice_with_length(array.offset() * width, len * width)
+    };
+
+    let data = ArrayData::builder(array.data_type().clone())
+        .len(len)
+        .add_buffer(values)
+        .nulls(nulls.map(NullBuffer::new));
+    Some(make_array(
+        data.build().expect("the layout of the input array"),
+    ))
+}
+
+/// An integer lane that [`blend`] selects with.
+trait BlendLane:
+    ArrowNativeType + BitAnd<Output = Self> + BitOr<Output = Self> + Not<Output = Self>
+{
+    /// `mask` (0 or -1) at this width.
+    fn from_mask(mask: i32) -> Self;
+}
+
+macro_rules! blend_lane {
+    ($($t:ty),*) => {
+        $(impl BlendLane for $t {
+            #[inline]
+            fn from_mask(mask: i32) -> Self {
+                mask as $t
+            }
+        })*
+    };
+}
+blend_lane!(i8, i16, i64);
+
+impl BlendLane for i32 {
+    #[inline]
+    fn from_mask(mask: i32) -> Self {
+        mask
+    }
+}
+
+/// For each bit of `word`, an `i32` with all bits set if it is set, else clear.
+#[inline]
+fn lane_masks(word: u64) -> [i32; 64] {
+    // Every lane of a byte tests a constant bit of the same value: an AND and a compare of
+    // 32-bit lanes, which SSE2 and NEON have. Shifting the word by a different count per lane
+    // has no SIMD form before AVX2, and SSE2 has no 64-bit compare.

Review Comment:
   my bad, thanks for checking. I had verified this loop for branches, not for how the masks are computed, and LLVM turns the compare into a shift into the sign bit, which SSE2 emulates with pmuludq. Took your wording.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "Jefffrey (via GitHub) via github" <gi...@arrow.apache.org>.
Jefffrey commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4481428036


##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,174 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return output;
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<Result<ArrayRef, ArrowError>> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+            16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+            32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+            _ => return None,
+        }
+    } else {
+        array.buffers()[0].slice_with_length(array.offset() * width, len * width)

Review Comment:
   for the null scalar case, we could actually extend this to work for all datatypes (except runendencoded and unions) since we're just replacing the null buffer of the array; though we can do this in a followup PR



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mightsleep (via GitHub)" <gi...@apache.org>.
mightsleep commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4363354673


##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return Ok(output);
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,

Review Comment:
   Thanks for digging into this and for filing #11862. I'd like to keep this PR about speed. The fast path dispatches on width only, and an unaligned buffer falls back to zip_impl through the generic alignment check in rows, so the behavior for it doesn't change here. Handling DayTime alignment in each kernel with a fast path would spread the same workaround around, while the fix in MutableArrayData, as #10171 did for MonthDayNano, covers zip, merge and every other kernel that copies through it at once. I can pick up #11862 once one of my open PRs lands, unless someone gets to it first.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6434141714

   
   ```
   zip_8192_from_decimal128/array_vs_non_null_scalar/true_then_false    12.86    37.5±0.14µs        ? ?/sec     1.00      2.9±0.02µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/10pct_true             183.32   105.8±0.38µs        ? ?/sec    1.00    577.0±3.43ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/1pct_true              170.61    98.4±0.36µs        ? ?/sec    1.00    576.7±3.37ns        ? ?/sec
   ```
   
   oh wow  🤯 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mightsleep (via GitHub)" <gi...@apache.org>.
mightsleep commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6349928266

   Rebased on #11625


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4361268730


##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return Ok(output);
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,

Review Comment:
   What happens here for an `Interval(DayTime)` array whose values buffer is only 4-byte aligned? arrow-rs requires a fixed-width buffer to be aligned only to `align_of` of its native type ([`new_fixed_width`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-data/src/data.rs#L2058-L2063)), and [`IntervalDayTime`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-buffer/src/interval.rs#L348-L355) is two `i32`s, so its alignment is 4. The Arrow format only [recommends 8- or 64-byte alignment](https://github.com/apache/arrow/blob/373d64c02cb23663b63c5853099fca43a46c39fc/docs/source/format/Columnar.rst?plain=1#L264-L273) in memory.
   
   For such a buffer (I built one by slicing a `Buffer` 4 bytes in), `blend::<i64, 1>` returns `None`, and the fallback panics in `Buffer::typed_data`, because `MutableArrayData` [extends `DayTime` as `i64`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-data/src/transform/mod.rs#L259-L263). That panic also happens on `main` without this PR, so it needs its own fix.
   
   For this PR, what do you think about blending `DayTime` as two `i32` lanes? The aligned case would then never fall back, and as I read it, the alignment check in `rows` would no longer fail for any type that reaches `blend`. Could the test also add `Interval(DayTime)` to `data_types`, plus one case with a 4-byte-aligned buffer? `random_primitive` always produces aligned buffers. This needs `IntervalUnit` in the top-level `arrow_schema` import.
   
   ```suggestion
               8 if *array.data_type() == DataType::Interval(IntervalUnit::DayTime) => {
                   // arrow-rs aligns a `DayTime` buffer only to `align_of::<IntervalDayTime>()`, 4
                   blend::<i32, 2>(&from_array, &array, &scalar)?
               }
               8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383714117

   🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383511996)
   
   **Target:** zip&#95;kernels
   
   Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   env:
     BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
     RUSTFLAGS: -C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6
   resources:
     arch: amd64
   shards: 1
   
   ```
   
   </details>
   
   <details><summary>Details</summary>
   
   ```
   group                                                                base                                    changed
   -----                                                                ----                                    -------
   zip_8192_from_decimal128/array_vs_non_null_scalar/10pct_true         8.39     73.4±1.91µs        ? ?/sec     1.00      8.8±0.13µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/1pct_true          9.73     61.5±1.80µs        ? ?/sec     1.00      6.3±0.08µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_nulls        9.71     85.1±2.37µs        ? ?/sec     1.00      8.8±0.16µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_true         10.30    90.5±1.54µs        ? ?/sec     1.00      8.8±0.14µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/90pct_true         3.25     28.1±0.46µs        ? ?/sec     1.00      8.6±0.14µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/99pct_true         1.18      8.1±0.17µs        ? ?/sec     1.00      6.9±0.13µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/all_false          22.12    60.4±0.92µs        ? ?/sec     1.00      2.7±0.05µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/all_true           1.06      5.6±0.04µs        ? ?/sec     1.00      5.3±0.08µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/true_then_false    7.96     33.0±0.53µs        ? ?/sec     1.00      4.1±0.06µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/10pct_true             176.17    92.9±1.39µs        ? ?/sec    1.00    527.1±8.94ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/1pct_true              154.13    81.9±1.06µs        ? ?/sec    1.00   531.3±10.80ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/50pct_nulls            177.48   103.1±2.02µs        ? ?/sec    1.00   580.8±13.25ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/50pct_true             183.86    96.3±1.57µs        ? ?/sec    1.00   523.8±10.25ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/90pct_true             56.56    29.9±0.52µs        ? ?/sec     1.00   529.4±10.98ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/99pct_true             15.66     8.3±0.10µs        ? ?/sec     1.00   528.6±11.18ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/all_false              152.09    80.0±1.42µs        ? ?/sec    1.00    525.8±9.43ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/all_true               10.93     5.7±0.04µs        ? ?/sec     1.00    521.8±7.71ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/true_then_false        81.30    43.0±0.72µs        ? ?/sec     1.00   529.2±13.58ns        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/10pct_true         3.91     34.1±0.51µs        ? ?/sec     1.00      8.7±0.16µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/1pct_true          1.83     12.9±0.14µs        ? ?/sec     1.00      7.1±0.15µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_nulls        6.95     61.4±0.72µs        ? ?/sec     1.00      8.8±0.17µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_true         11.29    98.3±1.54µs        ? ?/sec     1.00      8.7±0.11µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/90pct_true         8.97     78.9±1.07µs        ? ?/sec     1.00      8.8±0.15µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/99pct_true         10.49    67.5±1.19µs        ? ?/sec     1.00      6.4±0.09µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/all_false          1.08      5.8±0.04µs        ? ?/sec     1.00      5.3±0.16µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/all_true           23.27    64.9±1.36µs        ? ?/sec     1.00      2.8±0.04µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/true_then_false    9.09     37.5±0.92µs        ? ?/sec     1.00      4.1±0.05µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/10pct_true                20.26    71.0±0.92µs        ? ?/sec     1.00      3.5±0.06µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/1pct_true                 24.91    62.2±1.49µs        ? ?/sec     1.00      2.5±0.05µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls               22.37    80.9±1.31µs        ? ?/sec     1.00      3.6±0.06µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_true                24.60    85.6±1.27µs        ? ?/sec     1.00      3.5±0.05µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/90pct_true                7.75     27.2±0.52µs        ? ?/sec     1.00      3.5±0.06µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/99pct_true                1.78      4.7±0.05µs        ? ?/sec     1.00      2.6±0.05µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_false                 61.65    60.8±0.86µs        ? ?/sec     1.00   987.0±21.69ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_true                  1.10  1809.6±24.72ns        ? ?/sec     1.00  1644.9±28.64ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/true_then_false           22.39    31.3±0.43µs        ? ?/sec     1.00  1396.8±55.86ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/10pct_true                    167.86    89.2±1.34µs        ? ?/sec    1.00   531.7±10.53ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/1pct_true                     156.29    83.2±1.08µs        ? ?/sec    1.00    532.3±9.06ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/50pct_nulls                   164.97    96.3±2.26µs        ? ?/sec    1.00    583.9±7.88ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/50pct_true                    180.66    96.0±1.50µs        ? ?/sec    1.00    531.5±7.92ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/90pct_true                    54.69    29.2±0.39µs        ? ?/sec     1.00    534.6±7.03ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/99pct_true                    9.53      5.1±0.08µs        ? ?/sec     1.00   531.8±10.07ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/all_false                     154.66    81.9±1.57µs        ? ?/sec    1.00    529.7±6.98ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/all_true                      3.56  1875.6±30.34ns        ? ?/sec     1.00    527.0±9.79ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/true_then_false               79.80    41.8±0.77µs        ? ?/sec     1.00    524.2±8.12ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/10pct_true                8.43     30.4±0.44µs        ? ?/sec     1.00      3.6±0.06µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/1pct_true                 2.54      6.8±0.10µs        ? ?/sec     1.00      2.7±0.14µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls               16.46    59.1±1.10µs        ? ?/sec     1.00      3.6±0.06µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_true                25.42    92.4±1.24µs        ? ?/sec     1.00      3.6±0.10µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/90pct_true                20.84    74.5±1.17µs        ? ?/sec     1.00      3.6±0.06µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/99pct_true                24.92    63.8±1.00µs        ? ?/sec     1.00      2.6±0.05µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_false                 1.13  1926.3±34.08ns        ? ?/sec     1.00  1704.3±25.36ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_true                  59.23    62.2±1.14µs        ? ?/sec     1.00  1049.8±17.05ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/true_then_false           23.62    32.7±0.39µs        ? ?/sec     1.00  1386.0±52.13ns        ? ?/sec
   
   ```
   
   </details>
   
   <details><summary>Per-runner information</summary>
   
   <details><summary>zip&#95;kernels — shard 1/1</summary>
   
   **Node:** gk3-benchmark-cluster-nap-10zn6ylp-8d12e677-rxgh
   
   **Instance:** c4-highmem-16 (12 vCPU / 65 GiB)
   
   **uname:**
   
   ```text
   Linux bench-c6383511996-3823-r7nm2 6.12.105+ #1 SMP Sat Sep  5 07:57:24 UTC 2026 x86_64 GNU/Linux
   ```
   
   **BENCH_COMMAND:**
   
   ```sh
   cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
   ```
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```text
   Architecture:                            x86_64
   CPU op-mode(s):                          32-bit, 64-bit
   Address sizes:                           52 bits physical, 57 bits virtual
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               GenuineIntel
   Model name:                              INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz
   CPU family:                              6
   Model:                                   207
   Thread(s) per core:                      2
   Core(s) per socket:                      8
   Socket(s):                               1
   Stepping:                                2
   BogoMIPS:                                4600.00
   Flags:                                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx_vnni avx512_bf16 wbnoinvd arat avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid cldemote movdiri movdir64b fsrm md_clear serialize tsxldtrk amx_bf16 avx512_fp16 amx_tile amx_int8 arch_capabilities
   Hypervisor vendor:                       KVM
   Virtualization type:                     full
   L1d cache:                               384 KiB (8 instances)
   L1i cache:                               256 KiB (8 instances)
   L2 cache:                                16 MiB (8 instances)
   L3 cache:                                260 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; usercopy/swapgs barriers and __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 530.1s |
   | Peak memory | 32.8 MiB |
   | Avg memory | 25.8 MiB |
   | CPU user | 525.4s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 525.1s |
   | Peak memory | 32.9 MiB |
   | Avg memory | 26.3 MiB |
   | CPU user | 521.6s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   
   </details>
   
   </details>
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6383511996:finish:479f8278ca9a49ebf909a09d7a8b0939 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4361268730


##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return Ok(output);
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,

Review Comment:
   What happens here for an `Interval(DayTime)` array whose values buffer is only 4-byte aligned? arrow-rs requires a fixed-width buffer to be aligned only to `align_of` of its native type ([`new_fixed_width`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-data/src/data.rs#L2058-L2063)), and [`IntervalDayTime`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-buffer/src/interval.rs#L348-L355) is two `i32`s, so its alignment is 4. The Arrow format only [recommends 8- or 64-byte alignment](https://github.com/apache/arrow/blob/373d64c02cb23663b63c5853099fca43a46c39fc/docs/source/format/Columnar.rst?plain=1#L264-L273) in memory.
   
   For such a buffer (I built one by slicing a `Buffer` 4 bytes in), `blend::<i64, 1>` returns `None`, and the fallback panics in `Buffer::typed_data`, because `MutableArrayData` [extends `DayTime` as `i64`](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/arrow-data/src/transform/mod.rs#L259-L263). That panic also happens on `main` without this PR, so it needs its own fix. Could you file an issue for it?
   
   For this PR, what do you think about blending `DayTime` as two `i32` lanes? The aligned case would then never fall back, and as I read it, the alignment check in `rows` would no longer fail for any type that reaches `blend`. Could the test also add `Interval(DayTime)` to `data_types`, plus one case with a 4-byte-aligned buffer? `random_primitive` always produces aligned buffers. This needs `IntervalUnit` in the top-level `arrow_schema` import.
   
   ```suggestion
               8 if *array.data_type() == DataType::Interval(IntervalUnit::DayTime) => {
                   // arrow-rs aligns a `DayTime` buffer only to `align_of::<IntervalDayTime>()`, 4
                   blend::<i32, 2>(&from_array, &array, &scalar)?
               }
               8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
   ```



##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return Ok(output);
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+            16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+            32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+            _ => return None,
+        }
+    } else {
+        array.buffers()[0].slice_with_length(array.offset() * width, len * width)
+    };
+
+    let data = ArrayData::builder(array.data_type().clone())
+        .len(len)
+        .add_buffer(values)
+        .nulls(nulls.map(NullBuffer::new));
+    Some(make_array(
+        data.build().expect("the layout of the input array"),
+    ))
+}
+
+/// An integer lane that [`blend`] selects with.
+trait BlendLane:
+    ArrowNativeType + BitAnd<Output = Self> + BitOr<Output = Self> + Not<Output = Self>
+{
+    /// `mask` (0 or -1) at this width.
+    fn from_mask(mask: i32) -> Self;
+}
+
+macro_rules! blend_lane {
+    ($($t:ty),*) => {
+        $(impl BlendLane for $t {
+            #[inline]
+            fn from_mask(mask: i32) -> Self {
+                mask as $t
+            }
+        })*
+    };
+}
+blend_lane!(i8, i16, i64);
+
+impl BlendLane for i32 {
+    #[inline]
+    fn from_mask(mask: i32) -> Self {
+        mask
+    }
+}
+
+/// For each bit of `word`, an `i32` with all bits set if it is set, else clear.
+#[inline]
+fn lane_masks(word: u64) -> [i32; 64] {
+    // Every lane of a byte tests a constant bit of the same value: an AND and a compare of
+    // 32-bit lanes, which SSE2 and NEON have. Shifting the word by a different count per lane
+    // has no SIMD form before AVX2, and SSE2 has no 64-bit compare.

Review Comment:
   Did you check which instructions this compiles to? On rustc 1.99.1 release, I don't see an AND and a compare on either target. On aarch64, `lane_masks` uses the per-lane variable shift that the comment says has no SIMD form, plus scalar bit extracts:
   
   ```
   ushl.4s v2, v1, v0     ; shift each lane by its own count
   cmlt.4s v2, v2, #0     ; mask = lane < 0
   sbfx    w11, w8, #5, #1
   ```
   
   On x86_64 (SSE2), it emulates the per-lane shift with multiplies, plus scalar `shl`/`sar`:
   
   ```
   pmuludq xmm4, xmm3     # multiply by per-lane powers of two
   psrad   xmm8, 31       # mask = sign bit
   ```
   
   Neither listing has a `pcmpeqd`, `cmeq` or `cmtst`. Both are branch-free, which is the property that matters here. Should the comment state that property instead, so that nobody later keeps the "AND and compare" form for a reason that doesn't hold?
   
   ```suggestion
       // Each mask tests a constant bit of one byte of the word. This compiles without a branch per
       // bit on x86_64 (SSE2) and aarch64.
   ```



##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return Ok(output);
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+            16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+            32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+            _ => return None,
+        }
+    } else {
+        array.buffers()[0].slice_with_length(array.offset() * width, len * width)
+    };
+
+    let data = ArrayData::builder(array.data_type().clone())
+        .len(len)
+        .add_buffer(values)
+        .nulls(nulls.map(NullBuffer::new));
+    Some(make_array(
+        data.build().expect("the layout of the input array"),
+    ))

Review Comment:
   Could this return the error instead of panicking? `zip` already returns a `Result`, and `PrimitiveScalarImpl` in this file propagates its build error ([line 633](https://github.com/apache/arrow-rs/blob/03bef9fd4622d8c78244b265a1653873e038d3de/arrow-select/src/zip.rs#L633)). If the function returns `Option<Result<ArrayRef, ArrowError>>` (line 171), the `?` on `blend` keeps working, and the caller at line 149 becomes `return output;`.
   
   ```suggestion
       Some(data.build().map(make_array))
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mightsleep (via GitHub)" <gi...@apache.org>.
mightsleep commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383181775

   Yes hello, added them in a commit before the change: decimal128 (an array against a scalar on either side, and against a null scalar) and array_vs_null_scalar for every type. Could you run the bot on zip_kernels @mbutrovich?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383508283

   🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383507725)
   
   **Target:** zip&#95;kernels
   
   **Sharding factor: 1** (1 workers).
   
   Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   env:
     BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
   shards: 1
   
   ```
   
   </details>
   
   Results will be posted when all workers finish.
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6383507725:start:4411e15c9be8f7789b94d24390bd300a -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383720375

   🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383519229)
   
   **Target:** zip&#95;kernels
   
   Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   env:
     BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
     RUSTFLAGS: -C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6
   shards: 1
   
   ```
   
   </details>
   
   <details><summary>Details</summary>
   
   ```
   group                                                                base                                    changed
   -----                                                                ----                                    -------
   zip_8192_from_decimal128/array_vs_non_null_scalar/10pct_true         10.72    80.9±0.30µs        ? ?/sec     1.00      7.5±0.13µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/1pct_true          12.17    72.0±0.23µs        ? ?/sec     1.00      5.9±0.03µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_nulls        11.60    88.6±0.33µs        ? ?/sec     1.00      7.6±0.17µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_true         11.42    88.5±0.32µs        ? ?/sec     1.00      7.7±0.19µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/90pct_true         4.07     30.9±0.35µs        ? ?/sec     1.00      7.6±0.22µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/99pct_true         1.34      7.3±0.04µs        ? ?/sec     1.00      5.5±0.07µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/all_false          30.79    70.6±0.28µs        ? ?/sec     1.00      2.3±0.01µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/all_true           1.02      3.6±0.05µs        ? ?/sec     1.00      3.5±0.06µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/true_then_false    12.86    37.5±0.14µs        ? ?/sec     1.00      2.9±0.02µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/10pct_true             183.32   105.8±0.38µs        ? ?/sec    1.00    577.0±3.43ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/1pct_true              170.61    98.4±0.36µs        ? ?/sec    1.00    576.7±3.37ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/50pct_nulls            168.75   110.3±0.33µs        ? ?/sec    1.00    653.4±3.94ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/50pct_true             178.74   103.0±0.27µs        ? ?/sec    1.00    576.2±3.28ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/90pct_true             58.84    33.9±0.09µs        ? ?/sec     1.00    576.4±3.33ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/99pct_true             13.34     7.7±0.02µs        ? ?/sec     1.00    576.3±3.55ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/all_false              170.55    97.3±0.31µs        ? ?/sec    1.00    570.5±2.96ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/all_true               6.37      3.6±0.05µs        ? ?/sec     1.00    570.7±2.77ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/true_then_false        88.98    51.1±0.24µs        ? ?/sec     1.00    574.1±3.08ns        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/10pct_true         4.84     37.1±0.42µs        ? ?/sec     1.00      7.7±0.23µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/1pct_true          2.39     13.0±0.13µs        ? ?/sec     1.00      5.5±0.05µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_nulls        8.69     66.4±0.74µs        ? ?/sec     1.00      7.6±0.16µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_true         13.22   101.4±0.78µs        ? ?/sec     1.00      7.7±0.24µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/90pct_true         11.38    88.8±0.28µs        ? ?/sec     1.00      7.8±0.25µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/99pct_true         13.06    78.3±0.17µs        ? ?/sec     1.00      6.0±0.02µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/all_false          1.07      3.8±0.06µs        ? ?/sec     1.00      3.6±0.05µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/all_true           32.50    76.4±0.19µs        ? ?/sec     1.00      2.4±0.01µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/true_then_false    14.12    42.4±0.19µs        ? ?/sec     1.00      3.0±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/10pct_true                29.18    78.8±0.21µs        ? ?/sec     1.00      2.7±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/1pct_true                 33.24    71.7±0.27µs        ? ?/sec     1.00      2.2±0.01µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls               31.11    86.9±0.11µs        ? ?/sec     1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_true                31.83    86.0±0.30µs        ? ?/sec     1.00      2.7±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/90pct_true                10.39    28.1±0.34µs        ? ?/sec     1.00      2.7±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/99pct_true                2.39      5.1±0.03µs        ? ?/sec     1.00      2.1±0.01µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_false                 58.91    70.4±0.40µs        ? ?/sec     1.00   1195.7±3.11ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_true                  1.14   1714.4±9.54ns        ? ?/sec     1.00   1502.3±4.41ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/true_then_false           26.67    36.2±0.12µs        ? ?/sec     1.00   1357.9±4.53ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/10pct_true                    176.41   103.6±0.29µs        ? ?/sec    1.00    587.2±5.68ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/1pct_true                     167.14    98.1±0.28µs        ? ?/sec    1.00    587.2±5.71ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/50pct_nulls                   163.62   107.5±0.22µs        ? ?/sec    1.00    657.1±2.85ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/50pct_true                    172.18    99.8±0.13µs        ? ?/sec    1.00    579.4±3.06ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/90pct_true                    52.26    30.7±0.05µs        ? ?/sec     1.00    586.9±5.85ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/99pct_true                    9.45      5.5±0.01µs        ? ?/sec     1.00    579.9±3.07ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/all_false                     168.79    97.0±0.28µs        ? ?/sec    1.00    574.8±2.58ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/all_true                      3.15   1806.9±5.63ns        ? ?/sec     1.00    574.3±2.55ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/true_then_false               86.36    49.6±0.15µs        ? ?/sec     1.00    574.3±2.62ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/10pct_true                11.33    31.4±0.35µs        ? ?/sec     1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/1pct_true                 3.34      7.5±0.03µs        ? ?/sec     1.00      2.3±0.01µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls               21.11    61.1±0.48µs        ? ?/sec     1.00      2.9±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_true                34.69    96.0±0.16µs        ? ?/sec     1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/90pct_true                30.46    85.1±0.12µs        ? ?/sec     1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/99pct_true                33.85    76.2±0.24µs        ? ?/sec     1.00      2.3±0.01µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_false                 1.15   1826.7±5.11ns        ? ?/sec     1.00  1589.1±10.15ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_true                  59.72    74.3±0.13µs        ? ?/sec     1.00   1244.4±6.41ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/true_then_false           26.81    39.0±0.12µs        ? ?/sec     1.00   1456.3±6.57ns        ? ?/sec
   
   ```
   
   </details>
   
   <details><summary>Per-runner information</summary>
   
   <details><summary>zip&#95;kernels — shard 1/1</summary>
   
   **Node:** gk3-benchmark-cluster-nap-2afsrt1r-6e04f20f-xqf5
   
   **Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
   
   **uname:**
   
   ```text
   Linux bench-c6383519229-3824-qjqzd 6.12.105+ #1 SMP Sat Sep  5 07:54:11 UTC 2026 aarch64 GNU/Linux
   ```
   
   **BENCH_COMMAND:**
   
   ```sh
   cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
   ```
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```text
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 530.1s |
   | Peak memory | 34.1 MiB |
   | Avg memory | 25.9 MiB |
   | CPU user | 523.3s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 535.1s |
   | Peak memory | 32.3 MiB |
   | Avg memory | 24.9 MiB |
   | CPU user | 530.6s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   
   </details>
   
   </details>
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6383519229:finish:0f1e58677ef2d11b365e28ccf179e575 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mightsleep (via GitHub) via github" <gi...@arrow.apache.org>.
mightsleep commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4485722953


##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,174 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return output;
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<Result<ArrayRef, ArrowError>> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+            16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+            32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+            _ => return None,
+        }
+    } else {
+        array.buffers()[0].slice_with_length(array.offset() * width, len * width)

Review Comment:
   yep, will do in a follow-up



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mightsleep (via GitHub)" <gi...@apache.org>.
mightsleep commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4363361523


##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return Ok(output);
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,
+            16 => blend::<i64, 2>(&from_array, &array, &scalar)?,
+            32 => blend::<i64, 4>(&from_array, &array, &scalar)?,
+            _ => return None,
+        }
+    } else {
+        array.buffers()[0].slice_with_length(array.offset() * width, len * width)
+    };
+
+    let data = ArrayData::builder(array.data_type().clone())
+        .len(len)
+        .add_buffer(values)
+        .nulls(nulls.map(NullBuffer::new));
+    Some(make_array(
+        data.build().expect("the layout of the input array"),
+    ))

Review Comment:
   Makes sense, applied. It can't fail for a layout taken from a valid input, but propagating the error matches PrimitiveScalarImpl in the same file.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6367490152

   run benchmark zip_kernels
   env:
     BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383713453

   🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383509349)
   
   **Target:** zip&#95;kernels
   
   Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   env:
     BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
   resources:
     arch: amd64
   shards: 1
   
   ```
   
   </details>
   
   <details><summary>Details</summary>
   
   ```
   group                                                                base                                    changed
   -----                                                                ----                                    -------
   zip_8192_from_decimal128/array_vs_non_null_scalar/10pct_true         11.50    87.7±2.27µs        ? ?/sec     1.00      7.6±0.19µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/1pct_true          12.52    73.2±1.65µs        ? ?/sec     1.00      5.8±0.10µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_nulls        10.83    86.4±4.01µs        ? ?/sec     1.00      8.0±0.43µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_true         13.65   104.0±3.02µs        ? ?/sec     1.00      7.6±0.16µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/90pct_true         4.44     34.0±1.14µs        ? ?/sec     1.00      7.7±0.22µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/99pct_true         1.34      8.6±0.28µs        ? ?/sec     1.00      6.4±0.11µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/all_false          21.70    64.5±3.13µs        ? ?/sec     1.00      3.0±0.10µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/all_true           1.26      6.3±0.17µs        ? ?/sec     1.00      5.0±0.20µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/true_then_false    9.75     40.5±1.39µs        ? ?/sec     1.00      4.2±0.08µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/10pct_true             176.44    94.6±1.82µs        ? ?/sec    1.00    536.2±9.16ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/1pct_true              157.79    84.4±3.37µs        ? ?/sec    1.00   534.9±12.04ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/50pct_nulls            175.73   103.3±4.66µs        ? ?/sec    1.00   588.0±17.29ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/50pct_true             214.40   117.0±2.94µs        ? ?/sec    1.00   545.5±26.50ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/90pct_true             68.81    36.8±1.04µs        ? ?/sec     1.00    535.0±5.58ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/99pct_true             17.31     9.3±0.23µs        ? ?/sec     1.00    536.1±7.74ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/all_false              159.26    84.5±2.82µs        ? ?/sec    1.00    530.4±3.77ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/all_true               11.62     6.2±0.21µs        ? ?/sec     1.00   537.4±25.94ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/true_then_false        84.50    45.0±1.51µs        ? ?/sec     1.00    532.2±8.31ns        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/10pct_true         5.41     41.4±1.62µs        ? ?/sec     1.00      7.7±0.14µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/1pct_true          2.29     15.2±0.63µs        ? ?/sec     1.00      6.6±0.37µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_nulls        9.72     77.1±1.05µs        ? ?/sec     1.00      7.9±0.05µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_true         13.03    99.7±3.94µs        ? ?/sec     1.00      7.6±0.12µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/90pct_true         10.93    83.7±3.08µs        ? ?/sec     1.00      7.7±0.16µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/99pct_true         12.30    72.4±4.02µs        ? ?/sec     1.00      5.9±0.08µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/all_false          1.23      6.4±0.16µs        ? ?/sec     1.00      5.2±0.10µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/all_true           22.54    67.8±2.26µs        ? ?/sec     1.00      3.0±0.05µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/true_then_false    11.16    46.2±1.18µs        ? ?/sec     1.00      4.1±0.04µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/10pct_true                18.02    78.4±6.23µs        ? ?/sec     1.00      4.3±0.14µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/1pct_true                 22.31    67.8±5.24µs        ? ?/sec     1.00      3.0±0.07µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls               19.94    88.6±5.69µs        ? ?/sec     1.00      4.4±0.11µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_true                20.86    88.8±3.14µs        ? ?/sec     1.00      4.3±0.14µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/90pct_true                6.54     28.3±1.16µs        ? ?/sec     1.00      4.3±0.14µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/99pct_true                1.55      5.0±0.24µs        ? ?/sec     1.00      3.2±0.09µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_false                 52.79    65.2±5.03µs        ? ?/sec     1.00  1234.9±27.13ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_true                  1.09  1903.8±39.32ns        ? ?/sec     1.00  1748.0±48.83ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/true_then_false           18.99    33.1±2.51µs        ? ?/sec     1.00  1743.9±52.35ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/10pct_true                    143.44    94.6±4.33µs        ? ?/sec    1.00   659.5±12.25ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/1pct_true                     157.36    86.2±2.85µs        ? ?/sec    1.00   548.0±15.02ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/50pct_nulls                   203.39   120.9±3.73µs        ? ?/sec    1.00   594.7±19.63ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/50pct_true                    155.84   100.2±4.43µs        ? ?/sec    1.00   642.7±15.67ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/90pct_true                    46.85    30.2±0.83µs        ? ?/sec     1.00   645.2±17.51ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/99pct_true                    8.83      5.7±0.50µs        ? ?/sec     1.00   645.4±15.37ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/all_false                     172.04    92.9±8.26µs        ? ?/sec    1.00   540.2±17.44ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/all_true                      3.64      2.3±0.06µs        ? ?/sec     1.00   634.2±15.52ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/true_then_false               71.45    45.0±2.28µs        ? ?/sec     1.00   630.1±18.81ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/10pct_true                9.62     36.3±1.14µs        ? ?/sec     1.00      3.8±0.14µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/1pct_true                 2.98      8.3±0.20µs        ? ?/sec     1.00      2.8±0.07µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls               15.98    72.3±2.37µs        ? ?/sec     1.00      4.5±0.10µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_true                30.28   114.8±2.34µs        ? ?/sec     1.00      3.8±0.17µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/90pct_true                24.49    91.7±4.48µs        ? ?/sec     1.00      3.7±0.10µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/99pct_true                26.97    72.1±6.30µs        ? ?/sec     1.00      2.7±0.12µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_false                 1.18      2.5±0.09µs        ? ?/sec     1.00      2.1±0.05µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_true                  51.03    68.0±4.84µs        ? ?/sec     1.00  1332.5±32.56ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/true_then_false           26.04    39.8±1.49µs        ? ?/sec     1.00  1528.1±39.62ns        ? ?/sec
   
   ```
   
   </details>
   
   <details><summary>Per-runner information</summary>
   
   <details><summary>zip&#95;kernels — shard 1/1</summary>
   
   **Node:** gk3-benchmark-cluster-nap-10zn6ylp-79de5834-b7fn
   
   **Instance:** c4-highmem-16 (12 vCPU / 65 GiB)
   
   **uname:**
   
   ```text
   Linux bench-c6383509349-3822-2cwms 6.12.105+ #1 SMP Sat Sep  5 07:57:24 UTC 2026 x86_64 GNU/Linux
   ```
   
   **BENCH_COMMAND:**
   
   ```sh
   cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
   ```
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```text
   Architecture:                            x86_64
   CPU op-mode(s):                          32-bit, 64-bit
   Address sizes:                           52 bits physical, 57 bits virtual
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               GenuineIntel
   Model name:                              INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz
   CPU family:                              6
   Model:                                   207
   Thread(s) per core:                      2
   Core(s) per socket:                      8
   Socket(s):                               1
   Stepping:                                2
   BogoMIPS:                                4600.00
   Flags:                                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx_vnni avx512_bf16 wbnoinvd arat avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid cldemote movdiri movdir64b fsrm md_clear serialize tsxldtrk amx_bf16 avx512_fp16 amx_tile amx_int8 arch_capabilities
   Hypervisor vendor:                       KVM
   Virtualization type:                     full
   L1d cache:                               384 KiB (8 instances)
   L1i cache:                               256 KiB (8 instances)
   L2 cache:                                16 MiB (8 instances)
   L3 cache:                                260 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; usercopy/swapgs barriers and __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 535.1s |
   | Peak memory | 31.7 MiB |
   | Avg memory | 24.8 MiB |
   | CPU user | 532.4s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 525.1s |
   | Peak memory | 31.7 MiB |
   | Avg memory | 24.8 MiB |
   | CPU user | 520.6s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   
   </details>
   
   </details>
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6383509349:finish:b99b3b4829c4e04cbb569a059e4f362c -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6186136950

   run benchmark zip_kernels
   env:
     BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383708566

   🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383507725)
   
   **Target:** zip&#95;kernels
   
   Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   env:
     BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
   shards: 1
   
   ```
   
   </details>
   
   <details><summary>Details</summary>
   
   ```
   group                                                                base                                    changed
   -----                                                                ----                                    -------
   zip_8192_from_decimal128/array_vs_non_null_scalar/10pct_true         10.34    82.6±0.76µs        ? ?/sec     1.00      8.0±0.10µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/1pct_true          11.60    73.6±0.33µs        ? ?/sec     1.00      6.3±0.07µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_nulls        10.91    89.2±0.37µs        ? ?/sec     1.00      8.2±0.27µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/50pct_true         11.41    91.9±0.58µs        ? ?/sec     1.00      8.1±0.17µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/90pct_true         4.20     33.5±0.41µs        ? ?/sec     1.00      8.0±0.06µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/99pct_true         1.25      7.6±0.10µs        ? ?/sec     1.00      6.1±0.07µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/all_false          31.15    71.8±0.28µs        ? ?/sec     1.00      2.3±0.01µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/all_true           1.07      4.6±0.09µs        ? ?/sec     1.00      4.3±0.08µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_non_null_scalar/true_then_false    12.44    39.5±0.17µs        ? ?/sec     1.00      3.2±0.06µs        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/10pct_true             187.09   108.5±0.35µs        ? ?/sec    1.00    579.8±3.68ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/1pct_true              171.53    99.6±0.36µs        ? ?/sec    1.00    580.7±3.85ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/50pct_nulls            171.73   114.4±0.32µs        ? ?/sec    1.00    665.9±4.66ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/50pct_true             183.71   106.5±0.30µs        ? ?/sec    1.00    579.9±3.82ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/90pct_true             62.60    36.3±0.76µs        ? ?/sec     1.00    580.1±3.63ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/99pct_true             13.69     8.0±0.09µs        ? ?/sec     1.00    583.3±3.67ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/all_false              170.70    98.4±0.35µs        ? ?/sec    1.00    576.7±3.34ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/all_true               6.64      3.8±0.23µs        ? ?/sec     1.00    577.9±3.69ns        ? ?/sec
   zip_8192_from_decimal128/array_vs_null_scalar/true_then_false        92.13    53.1±0.27µs        ? ?/sec     1.00    576.3±3.42ns        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/10pct_true         5.01     40.6±0.35µs        ? ?/sec     1.00      8.1±0.14µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/1pct_true          2.45     14.8±0.31µs        ? ?/sec     1.00      6.0±0.08µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_nulls        9.01     69.7±0.52µs        ? ?/sec     1.00      7.7±0.35µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/50pct_true         13.14   105.8±0.40µs        ? ?/sec     1.00      8.1±0.12µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/90pct_true         11.42    92.1±0.32µs        ? ?/sec     1.00      8.1±0.09µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/99pct_true         12.44    79.5±0.45µs        ? ?/sec     1.00      6.4±0.05µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/all_false          1.22      4.8±0.10µs        ? ?/sec     1.00      3.9±0.22µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/all_true           33.14    78.0±0.33µs        ? ?/sec     1.00      2.4±0.01µs        ? ?/sec
   zip_8192_from_decimal128/non_null_scalar_vs_array/true_then_false    14.83    45.2±0.27µs        ? ?/sec     1.00      3.0±0.06µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/10pct_true                28.85    79.9±0.36µs        ? ?/sec     1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/1pct_true                 33.58    72.8±0.33µs        ? ?/sec     1.00      2.2±0.01µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls               30.28    87.2±0.24µs        ? ?/sec     1.00      2.9±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_true                31.11    86.3±0.36µs        ? ?/sec     1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/90pct_true                10.20    28.1±0.27µs        ? ?/sec     1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/99pct_true                2.36      5.1±0.03µs        ? ?/sec     1.00      2.2±0.01µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_false                 59.24    71.0±0.33µs        ? ?/sec     1.00   1198.5±5.87ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_true                  1.15  1749.0±20.69ns        ? ?/sec     1.00   1519.2±8.10ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/true_then_false           27.48    37.0±0.23µs        ? ?/sec     1.00   1346.1±7.02ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/10pct_true                    177.17   103.9±0.28µs        ? ?/sec    1.00    586.2±7.80ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/1pct_true                     167.97    98.5±0.31µs        ? ?/sec    1.00    586.3±8.05ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/50pct_nulls                   162.26   107.8±0.37µs        ? ?/sec    1.00    664.1±4.12ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/50pct_true                    171.62   100.6±0.41µs        ? ?/sec    1.00    586.0±7.76ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/90pct_true                    52.66    30.9±0.09µs        ? ?/sec     1.00    585.9±8.09ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/99pct_true                    9.32      5.5±0.02µs        ? ?/sec     1.00    586.4±8.09ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/all_false                     168.68    97.9±0.25µs        ? ?/sec    1.00    580.4±7.75ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/all_true                      3.06   1787.4±9.07ns        ? ?/sec     1.00    583.5±8.63ns        ? ?/sec
   zip_8192_from_i32/array_vs_null_scalar/true_then_false               86.93    50.4±0.21µs        ? ?/sec     1.00    579.9±7.82ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/10pct_true                11.15    31.6±0.33µs        ? ?/sec     1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/1pct_true                 3.38      7.6±0.03µs        ? ?/sec     1.00      2.2±0.01µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls               20.89    61.5±0.40µs        ? ?/sec     1.00      2.9±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_true                34.14    96.8±0.37µs        ? ?/sec     1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/90pct_true                30.06    85.8±0.29µs        ? ?/sec     1.00      2.9±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/99pct_true                33.91    77.0±0.26µs        ? ?/sec     1.00      2.3±0.01µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_false                 1.20   1910.0±7.78ns        ? ?/sec     1.00   1593.1±6.78ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_true                  59.77    74.8±0.21µs        ? ?/sec     1.00   1250.8±6.85ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/true_then_false           26.74    39.4±0.18µs        ? ?/sec     1.00   1472.2±6.32ns        ? ?/sec
   
   ```
   
   </details>
   
   <details><summary>Per-runner information</summary>
   
   <details><summary>zip&#95;kernels — shard 1/1</summary>
   
   **Node:** gk3-benchmark-cluster-nap-hr1hsdiv-b6316889-j9b7
   
   **Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
   
   **uname:**
   
   ```text
   Linux bench-c6383507725-3821-4rhsq 6.12.105+ #1 SMP Sat Sep  5 07:54:11 UTC 2026 aarch64 GNU/Linux
   ```
   
   **BENCH_COMMAND:**
   
   ```sh
   cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
   ```
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```text
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 530.1s |
   | Peak memory | 33.2 MiB |
   | Avg memory | 25.0 MiB |
   | CPU user | 527.3s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 535.1s |
   | Peak memory | 31.1 MiB |
   | Avg memory | 23.6 MiB |
   | CPU user | 530.6s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   
   </details>
   
   </details>
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6383507725:finish:eb445b066884d247ad59304b9895dc8b -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383509349

   run benchmark zip_kernels
   ```
   env:
     BENCH_FILTER: "from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)"
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   resources:
     arch: amd64
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6367929060

   🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6367490152)
   
   **Target:** zip&#95;kernels
   
   Comparing zip-scalar-blend (03bef9fd4622d8c78244b265a1653873e038d3de) to df4dc3e (merge-base) [diff](https://github.com/apache/arrow-rs/compare/df4dc3ede59bd43b5f71847231212381d068bb72..03bef9fd4622d8c78244b265a1653873e038d3de)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   env:
     BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array
   shards: 1
   
   ```
   
   </details>
   
   <details><summary>Details</summary>
   
   ```
   group                                                                               base                                   changed
   -----                                                                               ----                                   -------
   zip_8192_from_i32/array_vs_non_null_scalar/10pct_true                               28.38    79.0±0.22µs        ? ?/sec    1.00      2.8±0.03µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/1pct_true                                32.39    72.0±0.29µs        ? ?/sec    1.00      2.2±0.01µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_nulls                              29.92    86.5±0.15µs        ? ?/sec    1.00      2.9±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/50pct_true                               30.72    85.9±0.34µs        ? ?/sec    1.00      2.8±0.03µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/90pct_true                               10.15    27.9±0.27µs        ? ?/sec    1.00      2.7±0.02µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/99pct_true                               2.32      5.1±0.03µs        ? ?/sec    1.00      2.2±0.01µs        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_false                                58.25    70.4±0.29µs        ? ?/sec    1.00   1209.3±3.18ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/all_true                                 1.09   1717.7±5.66ns        ? ?/sec    1.00   1579.3±5.19ns        ? ?/sec
   zip_8192_from_i32/array_vs_non_null_scalar/true_then_false                          25.39    36.3±0.14µs        ? ?/sec    1.00   1431.0±4.14ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/10pct_true                               11.11    31.4±0.29µs        ? ?/sec    1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/1pct_true                                3.31      7.6±0.02µs        ? ?/sec    1.00      2.3±0.01µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_nulls                              20.88    61.4±0.44µs        ? ?/sec    1.00      2.9±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/50pct_true                               33.93    96.7±0.26µs        ? ?/sec    1.00      2.8±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/90pct_true                               29.75    85.2±0.25µs        ? ?/sec    1.00      2.9±0.02µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/99pct_true                               32.72    76.1±0.26µs        ? ?/sec    1.00      2.3±0.01µs        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_false                                1.13   1872.4±6.10ns        ? ?/sec    1.00   1659.2±4.59ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/all_true                                 59.03    74.4±0.26µs        ? ?/sec    1.00   1260.2±5.62ns        ? ?/sec
   zip_8192_from_i32/non_null_scalar_vs_array/true_then_false                          25.35    39.1±0.15µs        ? ?/sec    1.00   1542.6±3.71ns        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/10pct_true         1.00     96.2±0.40µs        ? ?/sec    1.00     95.7±0.38µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/1pct_true          1.00     89.2±0.37µs        ? ?/sec    1.00     89.1±0.54µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/50pct_nulls        1.00    103.3±0.23µs        ? ?/sec    1.00    103.3±0.33µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/50pct_true         1.00    102.3±0.28µs        ? ?/sec    1.00    102.6±0.37µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/90pct_true         1.00     40.3±0.30µs        ? ?/sec    1.00     40.3±0.32µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/99pct_true         1.00     13.2±0.06µs        ? ?/sec    1.01     13.2±0.06µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/all_false          1.00     88.4±0.41µs        ? ?/sec    1.00     88.0±0.62µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/all_true           1.00      8.8±0.04µs        ? ?/sec    1.01      8.9±0.07µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/array_vs_non_null_scalar/true_then_false    1.01     48.8±0.19µs        ? ?/sec    1.00     48.3±0.26µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/10pct_true         1.00     46.0±0.37µs        ? ?/sec    1.00     45.9±0.39µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/1pct_true          1.00     18.0±0.05µs        ? ?/sec    1.00     17.9±0.05µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/50pct_nulls        1.00     78.4±0.62µs        ? ?/sec    1.00     78.2±0.62µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/50pct_true         1.00    116.1±0.37µs        ? ?/sec    1.00    116.1±0.28µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/90pct_true         1.00    105.2±0.30µs        ? ?/sec    1.00    105.0±0.38µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/99pct_true         1.00     95.9±0.35µs        ? ?/sec    1.00     95.5±0.35µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/all_false          1.04      9.4±0.27µs        ? ?/sec    1.00      9.0±0.04µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/all_true           1.00     94.2±0.40µs        ? ?/sec    1.00     93.9±0.34µs        ? ?/sec
   zip_8192_from_string_views size (3..20)/non_null_scalar_vs_array/true_then_false    1.01     53.6±0.31µs        ? ?/sec    1.00     53.3±0.20µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/10pct_true                   1.00    119.8±0.54µs        ? ?/sec    1.00    119.9±0.66µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/1pct_true                    1.00    110.9±0.71µs        ? ?/sec    1.00    111.0±0.86µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/50pct_nulls                  1.00    126.4±0.39µs        ? ?/sec    1.00    126.7±0.48µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/50pct_true                   1.00    123.2±0.49µs        ? ?/sec    1.00    123.5±0.53µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/90pct_true                   1.00     49.2±0.34µs        ? ?/sec    1.00     49.3±0.40µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/99pct_true                   1.00     16.3±0.04µs        ? ?/sec    1.02     16.6±0.04µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/all_false                    1.00    109.9±0.62µs        ? ?/sec    1.00    109.3±0.83µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/all_true                     1.00      8.6±0.02µs        ? ?/sec    1.01      8.7±0.03µs        ? ?/sec
   zip_8192_from_strings (3..20)/array_vs_non_null_scalar/true_then_false              1.00     60.6±0.36µs        ? ?/sec    1.00     60.4±0.38µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/10pct_true                   1.01     54.6±0.39µs        ? ?/sec    1.00     54.3±0.45µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/1pct_true                    1.02     20.5±0.06µs        ? ?/sec    1.00     20.0±0.04µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/50pct_nulls                  1.01     94.2±0.64µs        ? ?/sec    1.00     93.7±0.73µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/50pct_true                   1.01    139.4±0.35µs        ? ?/sec    1.00    138.6±0.47µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/90pct_true                   1.01    132.4±0.46µs        ? ?/sec    1.00    131.5±0.62µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/99pct_true                   1.01    119.0±0.53µs        ? ?/sec    1.00    118.3±0.72µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/all_false                    1.00      8.7±0.03µs        ? ?/sec    1.01      8.8±0.03µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/all_true                     1.00    116.0±0.56µs        ? ?/sec    1.00    116.5±0.68µs        ? ?/sec
   zip_8192_from_strings (3..20)/non_null_scalar_vs_array/true_then_false              1.00     65.7±0.29µs        ? ?/sec    1.00     65.7±0.34µs        ? ?/sec
   
   ```
   
   </details>
   
   <details><summary>Per-runner information</summary>
   
   <details><summary>zip&#95;kernels — shard 1/1</summary>
   
   **Node:** gk3-benchmark-cluster-nap-144p8rlm-9d0d7b21-v79b
   
   **Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
   
   **uname:**
   
   ```text
   Linux bench-c6367490152-3793-v7fsq 6.12.105+ #1 SMP Sat Sep  5 07:54:11 UTC 2026 aarch64 GNU/Linux
   ```
   
   **BENCH_COMMAND:**
   
   ```sh
   cargo bench --features=arrow,async,test_common,experimental,object_store --bench zip_kernels
   ```
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```text
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 525.1s |
   | Peak memory | 38.6 MiB |
   | Avg memory | 28.6 MiB |
   | CPU user | 521.4s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 525.1s |
   | Peak memory | 38.6 MiB |
   | Avg memory | 28.7 MiB |
   | CPU user | 521.6s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   
   </details>
   
   </details>
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6367490152:finish:74eeca4f2bf8f8caf6c151f95396b609 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "Jefffrey (via GitHub) via github" <gi...@arrow.apache.org>.
Jefffrey merged PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383509942

   🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383509349)
   
   **Target:** zip&#95;kernels
   
   **Sharding factor: 1** (1 workers).
   
   Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   env:
     BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
   resources:
     arch: amd64
   shards: 1
   
   ```
   
   </details>
   
   Results will be posted when all workers finish.
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6383509349:start:a537b171920a9813f5ca4fe83c8ee429 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4365060164


##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return Ok(output);
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,

Review Comment:
   Sounds good!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6367492342

   🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6367490152)
   
   **Target:** zip&#95;kernels
   
   **Sharding factor: 1** (1 workers).
   
   Comparing zip-scalar-blend (03bef9fd4622d8c78244b265a1653873e038d3de) to df4dc3e (merge-base) [diff](https://github.com/apache/arrow-rs/compare/df4dc3ede59bd43b5f71847231212381d068bb72..03bef9fd4622d8c78244b265a1653873e038d3de)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   env:
     BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array
   shards: 1
   
   ```
   
   </details>
   
   Results will be posted when all workers finish.
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6367490152:start:8d227a232276e9ddfee246398f40930c -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on code in PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#discussion_r4361462702


##########
arrow-select/src/zip.rs:
##########
@@ -139,12 +139,177 @@ pub fn zip(
         ));
     }
 
+    if truthy_is_scalar != falsy_is_scalar {
+        let (array, scalar) = if truthy_is_scalar {
+            (falsy, truthy)
+        } else {
+            (truthy, falsy)
+        };
+        if let Some(output) = zip_primitive_with_scalar(mask, array, scalar, truthy_is_scalar) {
+            return Ok(output);
+        }
+    }
+
     let falsy = falsy.to_data();
     let truthy = truthy.to_data();
 
     zip_impl(mask, &truthy, truthy_is_scalar, &falsy, falsy_is_scalar)
 }
 
+/// [`zip`] of a primitive `array` with a `scalar` on the other side, or `None` for the types it
+/// does not handle. `scalar_is_truthy` says which side the scalar is on.
+///
+/// The output values are a select per row between the array's value and the scalar's, and its
+/// null buffer has a closed form, so no row goes through [`MutableArrayData`]. With a null
+/// scalar the values under the scalar's rows are never read, so the array's values buffer is
+/// shared as it is.
+fn zip_primitive_with_scalar(
+    mask: &BooleanArray,
+    array: &dyn Array,
+    scalar: &dyn Array,
+    scalar_is_truthy: bool,
+) -> Option<ArrayRef> {
+    let width = array.data_type().primitive_width()?;
+    let array = array.to_data();
+    let scalar = scalar.to_data();
+    let len = array.len();
+
+    // The rows that take the array's value
+    let mut from_array = maybe_prep_null_mask_filter(mask);
+    if scalar_is_truthy {
+        from_array = !&from_array;
+    }
+
+    let scalar_valid = scalar.is_valid(0);
+    let nulls = match (scalar_valid, array.nulls()) {
+        (true, None) => None,
+        (true, Some(nulls)) => Some(nulls.inner() | &!&from_array),
+        (false, None) => Some(from_array.clone()),
+        (false, Some(nulls)) => Some(nulls.inner() & &from_array),
+    };
+
+    let values = if scalar_valid {
+        match width {
+            1 => blend::<i8, 1>(&from_array, &array, &scalar)?,
+            2 => blend::<i16, 1>(&from_array, &array, &scalar)?,
+            4 => blend::<i32, 1>(&from_array, &array, &scalar)?,
+            8 => blend::<i64, 1>(&from_array, &array, &scalar)?,

Review Comment:
   Filed https://github.com/apache/arrow-rs/issues/11862.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mightsleep (via GitHub) via github" <gi...@arrow.apache.org>.
mightsleep commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6670111217

   Yes hello, merged up.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6186138278

   🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6186136950)
   
   **Target:** zip&#95;kernels
   
   **Sharding factor: 1** (1 workers).
   
   Comparing zip-scalar-blend (deecbb71446efd61c0963b5f5802731b894edc10) to 4dc6bc2 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/4dc6bc25f351dd0b8474339ca342d488723f1368..deecbb71446efd61c0963b5f5802731b894edc10)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   env:
     BENCH_FILTER: array_vs_non_null_scalar|non_null_scalar_vs_array
   shards: 1
   
   ```
   
   </details>
   
   Results will be posted when all workers finish.
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6186136950:start:7cc9556a794cb9d3bb3b8ed1e621ffcd -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383512513

   🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383511996)
   
   **Target:** zip&#95;kernels
   
   **Sharding factor: 1** (1 workers).
   
   Comparing zip-scalar-blend (2ac0674dddee08c0567028b483d398d8d8565c41) to 0debf10fc85c36c5fb01cbbb4cad3863d4497748 [diff](https://github.com/apache/arrow-rs/compare/0debf10fc85c36c5fb01cbbb4cad3863d4497748..2ac0674dddee08c0567028b483d398d8d8565c41)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark zip_kernels
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   env:
     BENCH_FILTER: from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)
     RUSTFLAGS: -C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6
   resources:
     arch: amd64
   shards: 1
   
   ```
   
   </details>
   
   Results will be posted when all workers finish.
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6383511996:start:0e7339e0b52f432c1e07a2b971a7fd6f -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383511996

   run benchmark zip_kernels
   ```
   env:
     BENCH_FILTER: "from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)"
     RUSTFLAGS: "-C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6"
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   resources:
     arch: amd64
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383519229

   run benchmark zip_kernels
   ```
   env:
     BENCH_FILTER: "from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)"
     RUSTFLAGS: "-C llvm-args=-align-all-functions=6 -C llvm-args=-align-all-nofallthru-blocks=6"
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6383507725

   run benchmark zip_kernels
   ```
   env:
     BENCH_FILTER: "from_(i32|decimal128)/(array_vs_non_null_scalar|non_null_scalar_vs_array|array_vs_null_scalar)"
   baseline:
     ref: 0debf10fc85c36c5fb01cbbb4cad3863d4497748
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] perf(arrow-select): zip a primitive array with a scalar without MutableArrayData [arrow-rs]

Posted by "Jefffrey (via GitHub) via github" <gi...@arrow.apache.org>.
Jefffrey commented on PR #11559:
URL: https://github.com/apache/arrow-rs/pull/11559#issuecomment-6671153796

   thanks, some great numbers 🚀 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org