You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alexeigor (via GitHub)" <gi...@apache.org> on 2026/09/02 16:44:00 UTC
[PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
alexeigor opened a new pull request, #10958:
URL: https://github.com/apache/arrow-rs/pull/10958
# Which issue does this PR close?
- Closes #10955.
# Rationale for this change
`cast_numeric_arrays` currently sends every numeric pair through `num_cast`, including total conversions such as `Int32 -> Int64`. The safe path rebuilds validity through `unary_opt`; the unsafe path retains a fallible operation and walks valid indices. Both prevent the straight-line widening loop used by `PrimitiveArray::unary`.
# What changes are included in this PR?
- Identify the 35 numeric pairs for which `num_cast` is total and route them through `unary` with `AsPrimitive::as_`.
- Keep every narrowing or otherwise fallible conversion on the existing checked paths. The match is fail-closed, so an omitted pair only misses an optimization.
- Add a correctness matrix covering every fast-path pair, edge values, spread samples, floating-point special values, and an `i64 -> i32` negative control.
- Add Criterion coverage for safe, `safe: false`, and direct `unary` paths at 0% and 10% null density over 100,000 values.
Float16 conversions are intentionally left out of the fast path because they cannot use a primitive `as` cast and need separate correctness reasoning.
# Are these changes tested?
Yes. Criterion results on Apple arm64 with rustc 1.98.0:
| nulls | path | stock | patched | Criterion change |
|---:|---|---:|---:|---:|
| 0% | safe | 10.91 us | 9.13 us | -18.0% to -22.5% |
| 0% | `safe: false` | 12.00 us | 9.12 us | -18.6% to -23.0% |
| 10% | safe | 55.92 us | 9.14 us | -83.4% to -83.7% |
| 10% | `safe: false` | 56.20 us | 9.14 us | -83.8% to -84.0% |
| 10% | direct `unary` | 9.36 us | 9.14 us | reference |
A DataFusion 55 reproduction over 9,994,240 rows improves `SUM(Int32)` from 7.40 ms to 3.08 ms (2.40x), while the `SUM(Int64)` control stays at 2.14 ms.
Validation against current `main` using its pinned Rust toolchain:
- `cargo test -p arrow-cast --lib` (379 passed)
- `cargo fmt --all -- --check`
- `cargo clippy -p arrow-cast --lib --tests -- -D warnings`
- `cargo clippy -p arrow --bench cast_kernels --features test_utils -- -D warnings`
- `cargo bench -p arrow --bench cast_kernels --features test_utils --no-run`
AI assistance disclosure: Codex assisted with the reproduction scaffolding, test matrix, benchmark, and initial patch. The resulting code was validated with the checks above.
# Are there any user-facing changes?
No API or behavior changes are intended. Infallible numeric casts become faster and retain the input null buffer instead of rebuilding it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "alexeigor (via GitHub)" <gi...@apache.org>.
alexeigor commented on code in PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#discussion_r3917484677
##########
arrow/benches/cast_kernels.rs:
##########
@@ -252,13 +252,59 @@ fn build_string_dict_array(size: usize, distinct: usize) -> ArrayRef {
))
}
+/// Array lengths the primitive to primitive cast benchmarks are run at.
+const NUMERIC_CAST_SIZES: [usize; 4] = [512, 1_024, 8_192, 1_048_576];
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5515652408
🤔 looking the this again Im not sure we really need this many new benchmarks either
```
cast int64 to int32 1024 1.00 1619.9±1.64ns ? ?/sec
cast int64 to int32 512 1.00 834.5±3.08ns ? ?/sec 1.14 951.0±1.66ns ? ?/sec
cast int64 to int32 65536 1.00 90.0±3.18µs ? ?/sec
cast int64 to int32 8192 1.00 11.1±0.01µs ? ?/sec
```
is there anyway to reduce this to just two benchmarks be type cast?
this is really more of a personal nit than anything else.
I'll try and take a second look at the PR this evening
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "alexeigor (via GitHub)" <gi...@apache.org>.
alexeigor commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5514841029
Thanks @Rich-T-kid — both points addressed.
**Benchmark sizes.** Now `[512, 1024, 8192, 65536]` as suggested. The `512` names are unchanged, so those rows stay comparable with historical runs.
**Trimming.** The PR goes from `+438 −31` to `+362 −29`.
Benchmarks — removed the `infallible numeric cast i32 to i64` group (6 benchmark IDs). The size sweep over the existing primitive cast benchmarks covers the same ground, and unlike a newly added group it can actually be compared against the merge base. That group was the one showing a blank `main` column in the run above, so it was contributing bulk without contributing a comparison. The imports it needed are reverted too, so the import diff is gone.
Tests — the end-to-end cast test went from all 34 integer-source pairs to one per conversion class (signed widening, signed to float, unsigned widening, unsigned to wider signed, unsigned to float), and `Float32 -> Float64` folded into it rather than having a test of its own. Four test functions down to three.
The reasoning for keeping a sample rather than the matrix: the property test already covers all 35 pairs numerically, so re-enumerating them end to end adds no numeric coverage. The end-to-end test is there to check what `unary` could get wrong — length, validity and array offsets — and a representative sample exercises that identically.
I left two things intact deliberately: the predicate's correctness matrix, and the negative control asserting narrowing / unsigned-to-signed-of-the-same-width / signed-to-unsigned still return null under `safe` and an error under `safe: false`. Those are what actually guard against a wrong pair entering the fast path — the negative control catches a bad `u32 -> i32` entry that the older assertions did not.
**Also in this push:** a fix for the `MIRI (3)` failure on the previous commit. The `Float32 -> Float64` test compared the converted value bitwise including for NaN; only NaN-ness is guaranteed across a float conversion, and Miri randomises the sign and payload deliberately, so it saw `0xFFF8000000000000` against `0x7FF8000000000000`. It now asserts `is_nan()` for NaN and keeps the bitwise comparison everywhere else, so the signed zero case still means something.
Locally: `cargo test -p arrow-cast --lib` (381 passed), `cargo fmt --all -- --check`, `cargo clippy -p arrow-cast --lib --tests -- -D warnings`, `cargo clippy -p arrow --bench cast_kernels --features test_utils -- -D warnings`, and the three tests under `cargo miri test` with `-Zmiri-many-seeds=0..8`.
Ready for another look when you have a moment. A re-run of `cast_kernels` would also fill in the new 8192 and 65536 columns on neutral hardware.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5515978510
🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10958#issuecomment-5515627667)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
Comparing perf/infallible-numeric-casts (935ffa7f5ebda56914227ddb537493738fcbbdbb) to 8c49c71 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/8c49c7132f8a55bb609d00685887987b2b1c9c56..935ffa7f5ebda56914227ddb537493738fcbbdbb)
<details><summary>Run configuration</summary>
```yaml
run benchmark cast_kernels
```
</details>
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
group main perf_infallible-numeric-casts
----- ---- -----------------------------
"cast decimal128 to float64" 1.00 27.1±0.02µs ? ?/sec 1.00 27.1±0.03µs ? ?/sec
"cast decimal128 to int64" 1.00 47.9±0.54µs ? ?/sec 1.01 48.3±0.77µs ? ?/sec
"cast decimal128 to int8" 1.00 62.3±1.84µs ? ?/sec 1.01 63.1±2.29µs ? ?/sec
"cast decimal256 to float64" 1.00 68.6±0.09µs ? ?/sec 1.00 68.6±0.09µs ? ?/sec
"cast decimal256 to int64" 1.00 146.4±1.17µs ? ?/sec 1.01 147.5±0.82µs ? ?/sec
"cast float64 to decimal128(32, 3)" 1.00 32.2±0.05µs ? ?/sec 1.00 32.3±0.04µs ? ?/sec
"cast invalid float64 to to decimal128(32, 3)" 1.00 15.7±0.01µs ? ?/sec 1.00 15.7±0.01µs ? ?/sec
"cast string to decimal128(38, 3)" 1.02 97.7±0.36µs ? ?/sec 1.00 95.8±0.54µs ? ?/sec
"cast string to decimal256(76, 3)" 1.00 150.3±0.48µs ? ?/sec 1.01 151.3±0.48µs ? ?/sec
cast binary dict to string view (sparse) 1.04 49.4±1.40µs ? ?/sec 1.00 47.5±3.19µs ? ?/sec
cast binary to dict high cardinality 1.00 185.7±0.32µs ? ?/sec 1.00 186.4±0.35µs ? ?/sec
cast binary to dict high cardinality no nulls 1.00 162.9±0.27µs ? ?/sec 1.00 163.4±0.28µs ? ?/sec
cast binary to dict low cardinality 1.01 154.4±0.29µs ? ?/sec 1.00 153.0±0.28µs ? ?/sec
cast binary to dict low cardinality no nulls 1.00 130.4±0.23µs ? ?/sec 1.00 129.8±0.20µs ? ?/sec
cast binary to dict medium cardinality 1.00 155.7±0.27µs ? ?/sec 1.00 155.2±0.29µs ? ?/sec
cast binary to dict medium cardinality no nulls 1.00 132.2±0.20µs ? ?/sec 1.00 131.9±0.21µs ? ?/sec
cast binary view to string 1.00 67.5±0.73µs ? ?/sec 1.01 68.1±3.20µs ? ?/sec
cast binary view to string view 1.00 63.5±2.06µs ? ?/sec 1.00 63.7±2.13µs ? ?/sec
cast binary view to wide string 1.00 67.8±0.77µs ? ?/sec 1.03 69.7±3.12µs ? ?/sec
cast date32 to date64 512 1.02 336.0±0.83ns ? ?/sec 1.00 330.5±0.49ns ? ?/sec
cast date64 to date32 512 1.00 825.9±1.37ns ? ?/sec 1.13 930.1±2.09ns ? ?/sec
cast decimal128 to decimal128 512 1.00 6.9±0.01µs ? ?/sec 1.00 6.9±0.01µs ? ?/sec
cast decimal128 to decimal128 512 lower precision 1.00 19.9±0.02µs ? ?/sec 1.00 19.9±0.02µs ? ?/sec
cast decimal128 to decimal128 512 with lower scale (infallible) 1.00 45.9±0.08µs ? ?/sec 1.00 45.9±0.05µs ? ?/sec
cast decimal128 to decimal128 512 with same scale 1.00 74.9±0.58ns ? ?/sec 1.01 75.8±0.86ns ? ?/sec
cast decimal128 to decimal256 512 1.02 26.2±0.02µs ? ?/sec 1.00 25.8±0.07µs ? ?/sec
cast decimal256 to decimal128 512 1.00 317.8±0.33µs ? ?/sec 1.00 317.5±0.20µs ? ?/sec
cast decimal256 to decimal256 512 1.00 82.4±0.10µs ? ?/sec 1.00 82.3±0.10µs ? ?/sec
cast decimal256 to decimal256 512 with same scale 1.00 75.1±0.90ns ? ?/sec 1.01 75.6±1.23ns ? ?/sec
cast dict to string view 1.00 10.6±0.01µs ? ?/sec 1.00 10.6±0.02µs ? ?/sec
cast dict to string view (sparse) 1.00 5.3±0.08µs ? ?/sec 1.00 5.3±0.07µs ? ?/sec
cast f32 to string 512 1.00 12.0±0.04µs ? ?/sec 1.02 12.2±0.06µs ? ?/sec
cast f64 to string 512 1.00 15.3±0.03µs ? ?/sec 1.02 15.6±0.04µs ? ?/sec
cast float32 to int32 1024 1.00 1001.8±1.50ns ? ?/sec
cast float32 to int32 512 1.19 778.8±0.76ns ? ?/sec 1.00 654.8±1.15ns ? ?/sec
cast float32 to int32 65536 1.00 48.5±0.04µs ? ?/sec
cast float32 to int32 8192 1.00 6.3±0.00µs ? ?/sec
cast float64 to float32 1024 1.00 965.4±3.36ns ? ?/sec
cast float64 to float32 512 1.03 643.9±1.90ns ? ?/sec 1.00 627.2±1.64ns ? ?/sec
cast float64 to float32 65536 1.00 48.7±0.05µs ? ?/sec
cast float64 to float32 8192 1.00 5.9±0.01µs ? ?/sec
cast float64 to uint64 1024 1.00 1026.9±2.01ns ? ?/sec
cast float64 to uint64 512 1.03 673.0±1.50ns ? ?/sec 1.00 652.2±1.09ns ? ?/sec
cast float64 to uint64 65536 1.00 51.8±3.08µs ? ?/sec
cast float64 to uint64 8192 1.00 6.5±0.01µs ? ?/sec
cast i64 to string 512 1.00 8.7±0.04µs ? ?/sec 1.04 9.0±0.03µs ? ?/sec
cast int32 to float32 1024 1.00 212.4±0.72ns ? ?/sec
cast int32 to float32 512 3.84 628.7±4.12ns ? ?/sec 1.00 163.6±0.56ns ? ?/sec
cast int32 to float32 65536 1.00 5.8±0.00µs ? ?/sec
cast int32 to float32 8192 1.00 873.6±1.15ns ? ?/sec
cast int32 to float64 1024 1.00 259.7±0.33ns ? ?/sec
cast int32 to float64 512 3.32 651.4±1.15ns ? ?/sec 1.00 196.1±0.31ns ? ?/sec
cast int32 to float64 65536 1.00 8.6±0.01µs ? ?/sec
cast int32 to float64 8192 1.00 1242.2±1.44ns ? ?/sec
cast int32 to int32 1024 1.00 174.9±1.80ns ? ?/sec
cast int32 to int32 512 1.00 173.8±1.01ns ? ?/sec 1.04 181.0±6.54ns ? ?/sec
cast int32 to int32 65536 1.00 174.8±1.80ns ? ?/sec
cast int32 to int32 8192 1.00 174.6±1.72ns ? ?/sec
cast int32 to int64 1024 1.00 216.0±0.44ns ? ?/sec
cast int32 to int64 512 3.75 648.5±2.99ns ? ?/sec 1.00 173.0±0.36ns ? ?/sec
cast int32 to int64 65536 1.00 6.0±0.01µs ? ?/sec
cast int32 to int64 8192 1.00 821.6±0.69ns ? ?/sec
cast int32 to uint32 1024 1.00 1467.3±28.42ns ? ?/sec
cast int32 to uint32 512 1.04 899.0±0.87ns ? ?/sec 1.00 864.0±0.79ns ? ?/sec
cast int32 to uint32 65536 1.00 213.9±0.35µs ? ?/sec
cast int32 to uint32 8192 1.00 11.3±0.10µs ? ?/sec
cast int64 to decimal32(9, -1) 512 1.00 3.4±0.00µs ? ?/sec 1.00 3.4±0.00µs ? ?/sec
cast int64 to decimal32(9, 0) 512 1.00 1258.7±1.35ns ? ?/sec 1.01 1269.6±3.89ns ? ?/sec
cast int64 to int32 1024 1.00 1623.3±3.25ns ? ?/sec
cast int64 to int32 512 1.00 838.9±0.85ns ? ?/sec 1.14 955.9±0.88ns ? ?/sec
cast int64 to int32 65536 1.00 86.8±0.18µs ? ?/sec
cast int64 to int32 8192 1.00 11.1±0.01µs ? ?/sec
cast nested dict to dict 1.01 4.6±0.03µs ? ?/sec 1.00 4.5±0.02µs ? ?/sec
cast no runs of int32s to ree<int32> 1.00 57.7±2.06µs ? ?/sec 1.01 58.5±0.93µs ? ?/sec
cast runs of 10 string to ree<int32> 1.00 8.6±0.02µs ? ?/sec 1.02 8.8±0.04µs ? ?/sec
cast runs of 1000 int32s to ree<int32> 1.00 3.4±0.01µs ? ?/sec 1.01 3.4±0.01µs ? ?/sec
cast string single run to ree<int32> 1.00 27.5±0.04µs ? ?/sec 1.00 27.5±0.02µs ? ?/sec
cast string to binary view 512 1.00 2.1±0.01µs ? ?/sec 1.00 2.1±0.00µs ? ?/sec
cast string view to binary view 1.07 93.2±3.55ns ? ?/sec 1.00 86.7±3.78ns ? ?/sec
cast string view to dict 1.00 154.9±0.33µs ? ?/sec 1.00 155.2±0.33µs ? ?/sec
cast string view to string 1.00 42.9±0.82µs ? ?/sec 1.00 43.1±0.77µs ? ?/sec
cast string view to wide string 1.00 42.8±0.85µs ? ?/sec 1.00 43.1±0.84µs ? ?/sec
cast time32s to time32ms 512 1.00 846.0±1.21ns ? ?/sec 1.00 846.2±2.76ns ? ?/sec
cast time32s to time64us 512 1.00 332.9±0.38ns ? ?/sec 1.00 333.4±1.27ns ? ?/sec
cast time64ns to time32s 512 1.01 418.5±0.35ns ? ?/sec 1.00 415.5±1.70ns ? ?/sec
cast timestamp_ms to i64 512 1.00 254.8±3.00ns ? ?/sec 1.00 254.7±4.86ns ? ?/sec
cast timestamp_ms to timestamp_ns 512 1.00 1285.3±3.06ns ? ?/sec 1.03 1319.2±4.92ns ? ?/sec
cast timestamp_ns to timestamp_s 512 1.01 176.6±1.93ns ? ?/sec 1.00 174.2±2.50ns ? ?/sec
cast utf8 to date32 512 1.00 6.9±0.05µs ? ?/sec 1.00 6.9±0.05µs ? ?/sec
cast utf8 to date64 512 1.04 33.5±0.15µs ? ?/sec 1.00 32.3±0.13µs ? ?/sec
cast utf8 to f32 1.01 5.6±0.03µs ? ?/sec 1.00 5.6±0.03µs ? ?/sec
cast utf8 to i32 1.01 5.3±0.06µs ? ?/sec 1.00 5.3±0.06µs ? ?/sec
cast wide string to binary view 512 1.00 4.0±0.07µs ? ?/sec 1.02 4.1±0.07µs ? ?/sec
```
</p>
</details>
<details><summary>Resource Usage</summary>
**base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 655.2s |
| Peak memory | 18.3 MiB |
| Avg memory | 17.0 MiB |
| CPU user | 648.5s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
**branch**
| Metric | Value |
|--------|-------|
| Wall time | 895.2s |
| Peak memory | 19.2 MiB |
| Avg memory | 17.7 MiB |
| CPU user | 892.0s |
| CPU sys | 0.1s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5514100769
🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10958#issuecomment-5513758436)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
Comparing perf/infallible-numeric-casts (54c55dcf8183b4a7455abf764039a0d048d3008a) to 8c49c71 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/8c49c7132f8a55bb609d00685887987b2b1c9c56..54c55dcf8183b4a7455abf764039a0d048d3008a)
<details><summary>Run configuration</summary>
```yaml
run benchmark cast_kernels
```
</details>
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
group main perf_infallible-numeric-casts
----- ---- -----------------------------
"cast decimal128 to float64" 1.00 27.1±0.03µs ? ?/sec 1.00 27.1±0.03µs ? ?/sec
"cast decimal128 to int64" 1.00 48.0±0.61µs ? ?/sec 1.00 48.2±0.26µs ? ?/sec
"cast decimal128 to int8" 1.00 62.3±1.84µs ? ?/sec 1.01 63.0±2.40µs ? ?/sec
"cast decimal256 to float64" 1.00 68.7±0.09µs ? ?/sec 1.00 68.7±0.13µs ? ?/sec
"cast decimal256 to int64" 1.00 147.0±1.04µs ? ?/sec 1.00 146.8±1.06µs ? ?/sec
"cast float64 to decimal128(32, 3)" 1.00 32.2±0.01µs ? ?/sec 1.01 32.6±0.02µs ? ?/sec
"cast invalid float64 to to decimal128(32, 3)" 1.00 15.7±0.01µs ? ?/sec 1.00 15.7±0.01µs ? ?/sec
"cast string to decimal128(38, 3)" 1.00 97.8±0.45µs ? ?/sec 1.00 98.0±0.46µs ? ?/sec
"cast string to decimal256(76, 3)" 1.00 150.4±0.47µs ? ?/sec 1.01 152.2±0.51µs ? ?/sec
cast binary dict to string view (sparse) 1.08 49.1±2.19µs ? ?/sec 1.00 45.6±1.36µs ? ?/sec
cast binary to dict high cardinality 1.00 186.5±0.34µs ? ?/sec 1.00 186.3±0.32µs ? ?/sec
cast binary to dict high cardinality no nulls 1.00 163.1±0.28µs ? ?/sec 1.00 163.0±0.29µs ? ?/sec
cast binary to dict low cardinality 1.00 154.2±0.25µs ? ?/sec 1.01 155.7±0.37µs ? ?/sec
cast binary to dict low cardinality no nulls 1.00 130.3±0.22µs ? ?/sec 1.00 130.3±0.21µs ? ?/sec
cast binary to dict medium cardinality 1.01 156.7±0.31µs ? ?/sec 1.00 155.6±0.33µs ? ?/sec
cast binary to dict medium cardinality no nulls 1.00 132.1±0.24µs ? ?/sec 1.00 132.1±0.20µs ? ?/sec
cast binary view to string 1.03 69.4±0.83µs ? ?/sec 1.00 67.2±0.62µs ? ?/sec
cast binary view to string view 1.00 63.5±2.04µs ? ?/sec 1.01 64.0±1.08µs ? ?/sec
cast binary view to wide string 1.01 67.8±0.81µs ? ?/sec 1.00 67.4±0.68µs ? ?/sec
cast date32 to date64 512 1.00 338.0±3.17ns ? ?/sec 1.00 337.2±1.37ns ? ?/sec
cast date64 to date32 512 1.01 845.8±6.38ns ? ?/sec 1.00 840.4±1.63ns ? ?/sec
cast decimal128 to decimal128 512 1.00 6.9±0.01µs ? ?/sec 1.00 6.9±0.01µs ? ?/sec
cast decimal128 to decimal128 512 lower precision 1.01 20.1±0.22µs ? ?/sec 1.00 19.9±0.03µs ? ?/sec
cast decimal128 to decimal128 512 with lower scale (infallible) 1.00 46.0±0.10µs ? ?/sec 1.00 45.8±0.05µs ? ?/sec
cast decimal128 to decimal128 512 with same scale 1.00 74.8±0.59ns ? ?/sec 1.00 74.5±0.51ns ? ?/sec
cast decimal128 to decimal256 512 1.00 26.2±0.02µs ? ?/sec 1.00 26.2±0.01µs ? ?/sec
cast decimal256 to decimal128 512 1.00 318.1±0.21µs ? ?/sec 1.00 317.1±0.25µs ? ?/sec
cast decimal256 to decimal256 512 1.00 82.4±0.12µs ? ?/sec 1.00 82.7±0.12µs ? ?/sec
cast decimal256 to decimal256 512 with same scale 1.00 74.9±0.81ns ? ?/sec 1.00 74.6±0.89ns ? ?/sec
cast dict to string view 1.00 10.9±0.24µs ? ?/sec 1.12 12.2±0.01µs ? ?/sec
cast dict to string view (sparse) 1.00 5.3±0.08µs ? ?/sec 1.04 5.5±0.08µs ? ?/sec
cast f32 to string 512 1.00 11.9±0.05µs ? ?/sec 1.00 12.0±0.04µs ? ?/sec
cast f64 to string 512 1.01 15.5±0.03µs ? ?/sec 1.00 15.3±0.04µs ? ?/sec
cast float32 to int32 512 1.00 765.5±5.57ns ? ?/sec 1.04 792.9±1.92ns ? ?/sec
cast float64 to float32 512 1.00 622.6±5.09ns ? ?/sec 1.06 659.8±1.83ns ? ?/sec
cast float64 to uint64 512 1.02 672.4±6.01ns ? ?/sec 1.00 659.0±2.09ns ? ?/sec
cast i64 to string 512 1.00 8.7±0.03µs ? ?/sec 1.01 8.8±0.03µs ? ?/sec
cast int32 to float32 512 4.06 626.0±2.22ns ? ?/sec 1.00 154.3±1.44ns ? ?/sec
cast int32 to float64 512 3.35 654.1±5.18ns ? ?/sec 1.00 195.2±1.35ns ? ?/sec
cast int32 to int32 512 1.01 174.7±1.21ns ? ?/sec 1.00 173.7±1.75ns ? ?/sec
cast int32 to int64 512 3.88 643.2±5.30ns ? ?/sec 1.00 165.8±1.38ns ? ?/sec
cast int32 to uint32 512 1.13 896.5±1.15ns ? ?/sec 1.00 792.7±1.80ns ? ?/sec
cast int64 to decimal32(9, -1) 512 1.00 3.4±0.01µs ? ?/sec 1.01 3.4±0.00µs ? ?/sec
cast int64 to decimal32(9, 0) 512 1.00 1264.6±6.00ns ? ?/sec 1.01 1274.9±1.93ns ? ?/sec
cast int64 to int32 512 1.01 855.7±4.74ns ? ?/sec 1.00 845.9±1.66ns ? ?/sec
cast nested dict to dict 1.12 5.2±0.04µs ? ?/sec 1.00 4.6±0.01µs ? ?/sec
cast no runs of int32s to ree<int32> 1.00 56.6±1.87µs ? ?/sec 1.02 57.5±2.12µs ? ?/sec
cast runs of 10 string to ree<int32> 1.00 8.6±0.02µs ? ?/sec 1.02 8.8±0.05µs ? ?/sec
cast runs of 1000 int32s to ree<int32> 1.00 3.4±0.01µs ? ?/sec 1.01 3.4±0.02µs ? ?/sec
cast string single run to ree<int32> 1.00 27.5±0.02µs ? ?/sec 1.00 27.5±0.05µs ? ?/sec
cast string to binary view 512 1.00 2.1±0.00µs ? ?/sec 1.00 2.1±0.00µs ? ?/sec
cast string view to binary view 1.12 91.0±1.44ns ? ?/sec 1.00 81.0±1.07ns ? ?/sec
cast string view to dict 1.00 154.5±0.49µs ? ?/sec 1.00 154.0±0.34µs ? ?/sec
cast string view to string 1.00 42.8±0.82µs ? ?/sec 1.00 43.0±0.71µs ? ?/sec
cast string view to wide string 1.00 42.9±0.84µs ? ?/sec 1.00 42.9±0.77µs ? ?/sec
cast time32s to time32ms 512 1.00 861.8±5.46ns ? ?/sec 1.01 866.7±2.59ns ? ?/sec
cast time32s to time64us 512 1.00 335.4±3.01ns ? ?/sec 1.00 336.7±1.47ns ? ?/sec
cast time64ns to time32s 512 1.00 415.4±2.88ns ? ?/sec 1.01 420.6±1.57ns ? ?/sec
cast timestamp_ms to i64 512 1.02 256.0±2.84ns ? ?/sec 1.00 252.1±1.35ns ? ?/sec
cast timestamp_ms to timestamp_ns 512 1.00 1292.7±4.39ns ? ?/sec 1.01 1305.8±4.99ns ? ?/sec
cast timestamp_ns to timestamp_s 512 1.00 173.0±1.41ns ? ?/sec 1.01 174.0±2.63ns ? ?/sec
cast utf8 to date32 512 1.01 7.0±0.06µs ? ?/sec 1.00 6.9±0.03µs ? ?/sec
cast utf8 to date64 512 1.00 32.0±0.15µs ? ?/sec 1.00 32.1±0.13µs ? ?/sec
cast utf8 to f32 1.00 5.6±0.03µs ? ?/sec 1.00 5.6±0.04µs ? ?/sec
cast utf8 to i32 1.00 5.3±0.06µs ? ?/sec 1.00 5.3±0.05µs ? ?/sec
cast wide string to binary view 512 1.01 4.0±0.07µs ? ?/sec 1.00 4.0±0.07µs ? ?/sec
infallible numeric cast i32 to i64/cast safe/0 1.00 9.6±0.08µs 9.7 GElem/sec
infallible numeric cast i32 to i64/cast safe/0.1 1.00 9.7±0.05µs 9.6 GElem/sec
infallible numeric cast i32 to i64/cast safe=false/0 1.00 9.7±0.06µs 9.6 GElem/sec
infallible numeric cast i32 to i64/cast safe=false/0.1 1.00 9.8±0.02µs 9.5 GElem/sec
infallible numeric cast i32 to i64/unary/0 1.00 10.1±0.10µs 9.3 GElem/sec
infallible numeric cast i32 to i64/unary/0.1 1.00 10.5±0.08µs 8.9 GElem/sec
```
</p>
</details>
<details><summary>Resource Usage</summary>
**base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 650.1s |
| Peak memory | 18.2 MiB |
| Avg memory | 17.1 MiB |
| CPU user | 647.5s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
**branch**
| Metric | Value |
|--------|-------|
| Wall time | 710.2s |
| Peak memory | 18.8 MiB |
| Avg memory | 16.9 MiB |
| CPU user | 708.8s |
| CPU sys | 0.1s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5515658456
🤖 Arrow criterion benchmark running (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10958#issuecomment-5515627667)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5515627667-2107-5wdtv 6.12.94+ #1 SMP Fri Jul 17 09:42:57 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing perf/infallible-numeric-casts (935ffa7f5ebda56914227ddb537493738fcbbdbb) to 8c49c71 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/8c49c7132f8a55bb609d00685887987b2b1c9c56..935ffa7f5ebda56914227ddb537493738fcbbdbb)
<details><summary>Run configuration</summary>
```yaml
run benchmark cast_kernels
```
</details>
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench cast_kernels
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on code in PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#discussion_r3926941157
##########
arrow-cast/src/cast/mod.rs:
##########
@@ -2570,6 +2586,45 @@ where
}
}
+/// Returns `true` when the per-element checked conversion can be replaced by a
+/// plain widening conversion, that is when
Review Comment:
nit: this isnt 100% true right? it allows precision-losing integer-to-float cases such as UInt64 -> Float32
I think we can say the operation is infallible or something of the sort
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5515245466
🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10958#issuecomment-5514861887)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
Comparing perf/infallible-numeric-casts (935ffa7f5ebda56914227ddb537493738fcbbdbb) to 8c49c71 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/8c49c7132f8a55bb609d00685887987b2b1c9c56..935ffa7f5ebda56914227ddb537493738fcbbdbb)
<details><summary>Run configuration</summary>
```yaml
run benchmark cast_kernels
```
</details>
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
group main perf_infallible-numeric-casts
----- ---- -----------------------------
"cast decimal128 to float64" 1.00 27.1±0.02µs ? ?/sec 1.00 27.2±0.01µs ? ?/sec
"cast decimal128 to int64" 1.00 48.4±0.89µs ? ?/sec 1.00 48.3±0.84µs ? ?/sec
"cast decimal128 to int8" 1.00 63.1±2.34µs ? ?/sec 1.00 63.3±2.24µs ? ?/sec
"cast decimal256 to float64" 1.00 68.7±0.08µs ? ?/sec 1.00 68.6±0.08µs ? ?/sec
"cast decimal256 to int64" 1.00 145.9±0.94µs ? ?/sec 1.01 147.2±0.64µs ? ?/sec
"cast float64 to decimal128(32, 3)" 1.00 32.2±0.02µs ? ?/sec 1.00 32.4±0.06µs ? ?/sec
"cast invalid float64 to to decimal128(32, 3)" 1.00 15.6±0.01µs ? ?/sec 1.00 15.6±0.02µs ? ?/sec
"cast string to decimal128(38, 3)" 1.03 98.1±0.54µs ? ?/sec 1.00 95.7±0.45µs ? ?/sec
"cast string to decimal256(76, 3)" 1.00 150.3±0.41µs ? ?/sec 1.01 151.2±0.41µs ? ?/sec
cast binary dict to string view (sparse) 1.03 46.4±0.37µs ? ?/sec 1.00 45.0±0.10µs ? ?/sec
cast binary to dict high cardinality 1.00 185.9±0.36µs ? ?/sec 1.00 185.4±0.38µs ? ?/sec
cast binary to dict high cardinality no nulls 1.00 162.9±0.31µs ? ?/sec 1.00 163.2±0.33µs ? ?/sec
cast binary to dict low cardinality 1.01 154.2±0.29µs ? ?/sec 1.00 153.1±0.30µs ? ?/sec
cast binary to dict low cardinality no nulls 1.00 130.2±0.24µs ? ?/sec 1.00 130.7±0.24µs ? ?/sec
cast binary to dict medium cardinality 1.01 156.5±0.34µs ? ?/sec 1.00 154.9±0.34µs ? ?/sec
cast binary to dict medium cardinality no nulls 1.00 132.1±0.24µs ? ?/sec 1.00 132.0±0.56µs ? ?/sec
cast binary view to string 1.00 67.4±0.76µs ? ?/sec 1.02 68.6±0.75µs ? ?/sec
cast binary view to string view 1.00 62.3±1.41µs ? ?/sec 1.02 63.9±1.98µs ? ?/sec
cast binary view to wide string 1.00 67.6±0.77µs ? ?/sec 1.00 67.9±0.93µs ? ?/sec
cast date32 to date64 512 1.06 352.1±3.90ns ? ?/sec 1.00 331.7±0.72ns ? ?/sec
cast date64 to date32 512 1.00 851.8±8.88ns ? ?/sec 1.11 941.4±1.89ns ? ?/sec
cast decimal128 to decimal128 512 1.00 6.9±0.01µs ? ?/sec 1.00 6.9±0.00µs ? ?/sec
cast decimal128 to decimal128 512 lower precision 1.00 19.8±0.02µs ? ?/sec 1.01 19.9±0.02µs ? ?/sec
cast decimal128 to decimal128 512 with lower scale (infallible) 1.00 45.9±0.06µs ? ?/sec 1.00 45.8±0.06µs ? ?/sec
cast decimal128 to decimal128 512 with same scale 1.02 76.6±4.20ns ? ?/sec 1.00 75.0±0.59ns ? ?/sec
cast decimal128 to decimal256 512 1.01 26.2±0.01µs ? ?/sec 1.00 25.8±0.09µs ? ?/sec
cast decimal256 to decimal128 512 1.00 317.9±0.17µs ? ?/sec 1.00 317.1±0.21µs ? ?/sec
cast decimal256 to decimal256 512 1.00 82.3±0.10µs ? ?/sec 1.00 82.3±0.10µs ? ?/sec
cast decimal256 to decimal256 512 with same scale 1.13 85.1±5.72ns ? ?/sec 1.00 75.3±1.00ns ? ?/sec
cast dict to string view 1.02 10.8±0.16µs ? ?/sec 1.00 10.6±0.02µs ? ?/sec
cast dict to string view (sparse) 1.00 5.3±0.09µs ? ?/sec 1.00 5.3±0.09µs ? ?/sec
cast f32 to string 512 1.00 12.0±0.06µs ? ?/sec 1.01 12.1±0.06µs ? ?/sec
cast f64 to string 512 1.00 15.5±0.04µs ? ?/sec 1.00 15.5±0.03µs ? ?/sec
cast float32 to int32 1024 1.00 983.2±1.57ns ? ?/sec
cast float32 to int32 512 1.21 804.3±2.60ns ? ?/sec 1.00 664.8±1.59ns ? ?/sec
cast float32 to int32 65536 1.00 48.5±0.04µs ? ?/sec
cast float32 to int32 8192 1.00 6.3±0.01µs ? ?/sec
cast float64 to float32 1024 1.00 949.1±3.77ns ? ?/sec
cast float64 to float32 512 1.03 639.3±2.62ns ? ?/sec 1.00 618.3±1.27ns ? ?/sec
cast float64 to float32 65536 1.00 48.7±0.04µs ? ?/sec
cast float64 to float32 8192 1.00 5.9±0.01µs ? ?/sec
cast float64 to uint64 1024 1.00 1007.3±2.16ns ? ?/sec
cast float64 to uint64 512 1.03 675.9±2.97ns ? ?/sec 1.00 655.4±1.37ns ? ?/sec
cast float64 to uint64 65536 1.00 56.3±3.19µs ? ?/sec
cast float64 to uint64 8192 1.00 6.7±0.11µs ? ?/sec
cast i64 to string 512 1.00 8.8±0.04µs ? ?/sec 1.02 9.0±0.03µs ? ?/sec
cast int32 to float32 1024 1.00 211.8±0.77ns ? ?/sec
cast int32 to float32 512 3.87 632.1±5.53ns ? ?/sec 1.00 163.2±0.47ns ? ?/sec
cast int32 to float32 65536 1.00 5.6±0.00µs ? ?/sec
cast int32 to float32 8192 1.00 881.4±1.38ns ? ?/sec
cast int32 to float64 1024 1.00 259.8±0.67ns ? ?/sec
cast int32 to float64 512 3.29 655.9±2.74ns ? ?/sec 1.00 199.2±1.22ns ? ?/sec
cast int32 to float64 65536 1.00 8.4±0.00µs ? ?/sec
cast int32 to float64 8192 1.00 1248.2±1.26ns ? ?/sec
cast int32 to int32 1024 1.00 174.2±1.62ns ? ?/sec
cast int32 to int32 512 1.01 176.2±4.14ns ? ?/sec 1.00 174.5±1.42ns ? ?/sec
cast int32 to int32 65536 1.00 174.5±1.77ns ? ?/sec
cast int32 to int32 8192 1.00 174.5±1.74ns ? ?/sec
cast int32 to int64 1024 1.00 216.1±0.72ns ? ?/sec
cast int32 to int64 512 3.62 643.2±2.86ns ? ?/sec 1.00 177.9±1.47ns ? ?/sec
cast int32 to int64 65536 1.00 5.7±0.02µs ? ?/sec
cast int32 to int64 8192 1.00 821.3±1.09ns ? ?/sec
cast int32 to uint32 1024 1.00 1423.3±1.59ns ? ?/sec
cast int32 to uint32 512 1.04 903.1±2.24ns ? ?/sec 1.00 864.3±0.87ns ? ?/sec
cast int32 to uint32 65536 1.00 214.1±0.36µs ? ?/sec
cast int32 to uint32 8192 1.00 11.2±0.05µs ? ?/sec
cast int64 to decimal32(9, -1) 512 1.00 3.4±0.00µs ? ?/sec 1.01 3.4±0.00µs ? ?/sec
cast int64 to decimal32(9, 0) 512 1.00 1269.6±9.25ns ? ?/sec 1.01 1283.5±1.48ns ? ?/sec
cast int64 to int32 1024 1.00 1619.9±1.64ns ? ?/sec
cast int64 to int32 512 1.00 834.5±3.08ns ? ?/sec 1.14 951.0±1.66ns ? ?/sec
cast int64 to int32 65536 1.00 90.0±3.18µs ? ?/sec
cast int64 to int32 8192 1.00 11.1±0.01µs ? ?/sec
cast nested dict to dict 1.00 4.5±0.01µs ? ?/sec 1.05 4.7±0.01µs ? ?/sec
cast no runs of int32s to ree<int32> 1.00 56.4±1.76µs ? ?/sec 1.02 57.6±1.20µs ? ?/sec
cast runs of 10 string to ree<int32> 1.00 8.7±0.02µs ? ?/sec 1.02 8.8±0.04µs ? ?/sec
cast runs of 1000 int32s to ree<int32> 1.01 3.4±0.02µs ? ?/sec 1.00 3.4±0.01µs ? ?/sec
cast string single run to ree<int32> 1.00 27.4±0.02µs ? ?/sec 1.00 27.4±0.01µs ? ?/sec
cast string to binary view 512 1.00 2.1±0.01µs ? ?/sec 1.00 2.1±0.00µs ? ?/sec
cast string view to binary view 1.16 94.5±5.06ns ? ?/sec 1.00 81.2±1.34ns ? ?/sec
cast string view to dict 1.00 154.2±0.36µs ? ?/sec 1.01 155.1±0.59µs ? ?/sec
cast string view to string 1.00 42.8±0.79µs ? ?/sec 1.00 42.7±0.79µs ? ?/sec
cast string view to wide string 1.00 42.8±0.84µs ? ?/sec 1.00 42.7±0.85µs ? ?/sec
cast time32s to time32ms 512 1.02 872.8±9.48ns ? ?/sec 1.00 857.5±1.14ns ? ?/sec
cast time32s to time64us 512 1.02 338.6±3.50ns ? ?/sec 1.00 332.6±0.86ns ? ?/sec
cast time64ns to time32s 512 1.02 422.2±3.38ns ? ?/sec 1.00 413.4±0.80ns ? ?/sec
cast timestamp_ms to i64 512 1.00 254.5±4.27ns ? ?/sec 1.00 253.4±1.61ns ? ?/sec
cast timestamp_ms to timestamp_ns 512 1.00 1288.0±5.84ns ? ?/sec 1.02 1315.1±4.16ns ? ?/sec
cast timestamp_ns to timestamp_s 512 1.00 177.7±3.95ns ? ?/sec 1.01 180.3±5.64ns ? ?/sec
cast utf8 to date32 512 1.00 6.9±0.04µs ? ?/sec 1.00 6.9±0.04µs ? ?/sec
cast utf8 to date64 512 1.00 31.9±0.13µs ? ?/sec 1.05 33.5±0.14µs ? ?/sec
cast utf8 to f32 1.01 5.6±0.02µs ? ?/sec 1.00 5.6±0.03µs ? ?/sec
cast utf8 to i32 1.01 5.3±0.05µs ? ?/sec 1.00 5.3±0.05µs ? ?/sec
cast wide string to binary view 512 1.00 4.1±0.08µs ? ?/sec 1.00 4.1±0.08µs ? ?/sec
```
</p>
</details>
<details><summary>Resource Usage</summary>
**base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 655.1s |
| Peak memory | 18.3 MiB |
| Avg memory | 17.0 MiB |
| CPU user | 650.5s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
**branch**
| Metric | Value |
|--------|-------|
| Wall time | 915.2s |
| Peak memory | 19.2 MiB |
| Avg memory | 17.6 MiB |
| CPU user | 910.0s |
| CPU sys | 0.0s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "albertlockett (via GitHub)" <gi...@apache.org>.
albertlockett commented on code in PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#discussion_r3970281144
##########
arrow-cast/src/cast/mod.rs:
##########
@@ -2570,6 +2586,45 @@ where
}
}
+/// Returns `true` when the per-element checked conversion can be replaced by a
+/// plain widening conversion, that is when
+///
+/// ```text
+/// num_cast::<FROM, TO>(v) == Some(v.as_())
+/// ```
+///
+/// holds for *every* value of `FROM`.
+///
+/// Note that the required invariant is agreement with [`AsPrimitive::as_`], not
+/// merely that `num_cast` is total; a pair belongs here only if it also has a
+/// case in `test_infallible_numeric_casts`, which asserts exactly that equality.
+///
+/// The match is fail-closed: an unlisted pair keeps the existing checked path, so
+/// an omission costs performance and never correctness. The set is deliberately
+/// conservative and does not attempt to be exhaustive. In particular `Float16` is
+/// excluded, because `f16` has no primitive `as` conversion and so needs separate
+/// correctness reasoning.
+///
+/// Called with associated constants, so each monomorphisation folds this to a
+/// constant and the branch disappears.
+fn is_infallible_numeric_cast(from: &DataType, to: &DataType) -> bool {
Review Comment:
I'm wondering if this could/should be a method on [`ArrowPrimitiveType`](https://github.com/apache/arrow-rs/blob/b7e5977133eadfe652eee4d4dc814a4c9d135533/arrow-array/src/types.rs#L59-L67)?
The reason I'm asking specifically is: I'd like to use this in the dictionary builders to optimize converting keys in these `try_new_from_builder` methods:
https://github.com/apache/arrow-rs/blob/b7e5977133eadfe652eee4d4dc814a4c9d135533/arrow-array/src/builder/generic_bytes_dictionary_builder.rs#L198-L207
The canonical use case for these mtethods is: as a dictionary array is being built, if it turns out it would overflow, it can be upgraded efficiently to a builder with a wider key, which would typically be an infallible. It would be nice if we could reuse this same logic to know when we can do the same optimization.
e.g., wondering if it'd be acceptable to do something like:
```rs
pub trait ArrowPrimitiveType: primitive::PrimitiveTypeSealed + 'static {
// ...
// the new function:
fn can_widen_to<K: ArrowPrimitiveType>() -> bool {
matches!(
(Self::DATA_TYPE, K::DATA_TYPE),
// ... the widening match cases
)
}
```
Then in `cast_numeric_arrays` I think the the check becomes `FROM::can_widen_to::<TO>()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5873241767
both PR's seem to be pretty similar. I think we should go with whichever one is faster 😆. unless @alexeigor has a different opinion since he did open the initial PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5513505827
🤖 Arrow criterion benchmark running (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10958#issuecomment-5513466634)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5513466634-2086-9s9x9 6.12.94+ #1 SMP Fri Jul 17 09:42:57 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing perf/infallible-numeric-casts (54c55dcf8183b4a7455abf764039a0d048d3008a) to 8c49c71 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/8c49c7132f8a55bb609d00685887987b2b1c9c56..54c55dcf8183b4a7455abf764039a0d048d3008a)
<details><summary>Run configuration</summary>
```yaml
run benchmark arrow-cast
```
</details>
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow-cast
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5513507107
Benchmark for [this request](https://github.com/apache/arrow-rs/pull/10958#issuecomment-5513466634) failed before finishing (Kubernetes reason: `BackoffLimitExceeded`).
Benchmarks requested: `arrow-cast`
<details><summary>Kubernetes message</summary>
```
Job has reached the specified backoff limit
```
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "alexeigor (via GitHub)" <gi...@apache.org>.
alexeigor commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5513651992
Thanks for kicking off the benchmark @Rich-T-kid. That run failed on the trigger argument rather than on anything in the branch — `arrow-cast` is a package name, not a bench target, so cargo bailed out before compiling anything:
```console
$ cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow-cast
error: no bench target named `arrow-cast` in default-run packages
help: available bench targets:
aggregate_kernels
arithmetic_kernels
...
```
which is what the 6-second `BackoffLimitExceeded` is.
The `arrow-cast` package's own benches are `parse_timestamp`, `parse_time`, `parse_date`, `parse_decimal` and `format_temporal`, none of which reach `cast_numeric_arrays`. The bench this PR touches is **`cast_kernels`**, in the `arrow` package (`arrow/benches/cast_kernels.rs`). The same command with that target resolves and builds, with the feature list unchanged:
```console
$ cargo bench --features=arrow,async,test_common,experimental,object_store --bench cast_kernels --no-run
Finished `bench` profile [optimized] target(s) in 21.60s
Executable benches/cast_kernels.rs
```
Could someone re-run it as `run benchmark cast_kernels`? That covers the `infallible numeric cast i32 to i64` group this PR adds (safe / `safe: false` / direct `unary`, at 0% and 10% null density), plus the pre-existing cast benchmarks as a regression check. Numbers from neutral hardware would be much better than the laptop figures in the description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "alexeigor (via GitHub)" <gi...@apache.org>.
alexeigor commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5515600738
@Rich-T-kid please run above workflows
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5513758436
run benchmark cast_kernels
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5514215033
```
cast i64 to string 512 1.00 8.7±0.03µs ? ?/sec 1.01 8.8±0.03µs ? ?/sec
cast int32 to float32 512 4.06 626.0±2.22ns ? ?/sec 1.00 154.3±1.44ns ? ?/sec
cast int32 to float64 512 3.35 654.1±5.18ns ? ?/sec 1.00 195.2±1.35ns ? ?/sec
cast int32 to int32 512 1.01 174.7±1.21ns ? ?/sec 1.00 173.7±1.75ns ? ?/sec
cast int32 to int64 512 3.88 643.2±5.30ns ? ?/sec 1.00 165.8±1.38ns ? ?/sec
cast int32 to uint32 512 1.13 896.5±1.15ns ? ?/sec 1.00 792.7±1.80ns ? ?/sec
```
nice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5514901039
🤖 Arrow criterion benchmark running (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10958#issuecomment-5514861887)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5514861887-2106-njkbw 6.12.94+ #1 SMP Fri Jul 17 09:42:57 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing perf/infallible-numeric-casts (935ffa7f5ebda56914227ddb537493738fcbbdbb) to 8c49c71 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/8c49c7132f8a55bb609d00685887987b2b1c9c56..935ffa7f5ebda56914227ddb537493738fcbbdbb)
<details><summary>Run configuration</summary>
```yaml
run benchmark cast_kernels
```
</details>
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench cast_kernels
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5513466634
run benchmark arrow-cast
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5513788910
🤖 Arrow criterion benchmark running (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10958#issuecomment-5513758436)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5513758436-2094-fnmmg 6.12.94+ #1 SMP Fri Jul 17 09:42:57 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing perf/infallible-numeric-casts (54c55dcf8183b4a7455abf764039a0d048d3008a) to 8c49c71 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/8c49c7132f8a55bb609d00685887987b2b1c9c56..54c55dcf8183b4a7455abf764039a0d048d3008a)
<details><summary>Run configuration</summary>
```yaml
run benchmark cast_kernels
```
</details>
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench cast_kernels
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5872903248
@neilconway came up with another version of this PR here
- https://github.com/apache/arrow-rs/pull/11219
Do you have any preference between that PR and this PR @alexeigor or @Rich-T-kid ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5514861887
run benchmark cast_kernels
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on code in PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#discussion_r3917385255
##########
arrow/benches/cast_kernels.rs:
##########
@@ -252,13 +252,59 @@ fn build_string_dict_array(size: usize, distinct: usize) -> ArrayRef {
))
}
+/// Array lengths the primitive to primitive cast benchmarks are run at.
+const NUMERIC_CAST_SIZES: [usize; 4] = [512, 1_024, 8_192, 1_048_576];
Review Comment:
I think [512,1k,8k,64k] should be fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on PR #10958:
URL: https://github.com/apache/arrow-rs/pull/10958#issuecomment-5515627667
run benchmark cast_kernels
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(arrow-cast): speed up infallible numeric casts [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed pull request #10958: perf(arrow-cast): speed up infallible numeric casts
URL: https://github.com/apache/arrow-rs/pull/10958
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org