You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2026/03/31 19:23:50 UTC

[PR] Alamb/test encoding speed [arrow-rs]

alamb opened a new pull request, #9636:
URL: https://github.com/apache/arrow-rs/pull/9636

   This is a dummy PR that has all the changes in  that I can automatically run the benchmarks on
    - https://github.com/apache/arrow-rs/pull/9447
   
   The benchmark runner currently doesn't support branches called `main`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4164943996

   🤖 Arrow criterion benchmark running (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/9636#issuecomment-4164922910)
   **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4164922910-635-4lzw4 6.12.55+ #1 SMP Sun Feb  1 08:59:41 UTC 2026 aarch64 GNU/Linux`
   <details><summary>CPU Details (lscpu)</summary>
   
   ```
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   Comparing alamb/test_encoding_speed (4d2010cf3e7e9c19344ac6b541d8621447f643a3) to 77e4d05 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/77e4d05fe0f199ccfaad578e58278329534a9c3d..4d2010cf3e7e9c19344ac6b541d8621447f643a3)
   BENCH_NAME=arrow_writer
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_writer
   BENCH_FILTER=
   Results will be posted here when complete
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4164939554

   🤖 Arrow criterion benchmark running (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/9636#issuecomment-4164926733)
   **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4164926733-636-z2g6l 6.12.55+ #1 SMP Sun Feb  1 08:59:41 UTC 2026 aarch64 GNU/Linux`
   <details><summary>CPU Details (lscpu)</summary>
   
   ```
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   Comparing alamb/test_encoding_speed (4d2010cf3e7e9c19344ac6b541d8621447f643a3) to 77e4d05 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/77e4d05fe0f199ccfaad578e58278329534a9c3d..4d2010cf3e7e9c19344ac6b541d8621447f643a3)
   BENCH_NAME=arrow_writer
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_writer
   BENCH_FILTER=
   Results will be posted here when complete
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4171256054

   🤖 Arrow criterion benchmark running (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/9636#issuecomment-4171240149)
   **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4171240149-649-2m5k7 6.12.55+ #1 SMP Sun Feb  1 08:59:41 UTC 2026 aarch64 GNU/Linux`
   <details><summary>CPU Details (lscpu)</summary>
   
   ```
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   Comparing alamb/test_encoding_speed (d55ca607945662d8d47e500ddf1bf3d750ef4a1e) to 1f07e54 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/1f07e54c1209a512f3ce15f4e576f0ca5d1c8d97..d55ca607945662d8d47e500ddf1bf3d750ef4a1e)
   BENCH_NAME=arrow_writer
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_writer
   BENCH_FILTER=
   Results will be posted here when complete
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4171612362

   🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/9636#issuecomment-4171240149)
   
   **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Details</summary>
   <p>
   
   ```
   group                                     alamb_test_encoding_speed              main
   -----                                     -------------------------              ----
   bool/bloom_filter                         1.05     14.9±0.12ms    16.8 MB/sec    1.00     14.1±0.18ms    17.7 MB/sec
   bool/default                              1.06     12.8±0.13ms    19.5 MB/sec    1.00     12.1±0.18ms    20.7 MB/sec
   bool/parquet_2                            1.02     15.7±0.11ms    15.9 MB/sec    1.00     15.5±0.19ms    16.1 MB/sec
   bool/zstd                                 1.06     13.3±0.12ms    18.8 MB/sec    1.00     12.6±0.18ms    19.9 MB/sec
   bool/zstd_parquet_2                       1.02     16.2±0.12ms    15.5 MB/sec    1.00     15.9±0.20ms    15.7 MB/sec
   bool_non_null/bloom_filter                1.01      6.9±0.03ms    18.2 MB/sec    1.00      6.8±0.02ms    18.4 MB/sec
   bool_non_null/default                     1.02      4.2±0.02ms    29.6 MB/sec    1.00      4.2±0.03ms    30.1 MB/sec
   bool_non_null/parquet_2                   1.00      8.0±0.02ms    15.6 MB/sec    1.08      8.6±0.02ms    14.5 MB/sec
   bool_non_null/zstd                        1.02      4.6±0.02ms    27.3 MB/sec    1.00      4.5±0.02ms    27.8 MB/sec
   bool_non_null/zstd_parquet_2              1.00      8.4±0.03ms    14.8 MB/sec    1.07      9.1±0.03ms    13.8 MB/sec
   float_with_nans/bloom_filter              1.01     94.1±1.90ms   148.8 MB/sec    1.00     93.5±0.38ms   149.8 MB/sec
   float_with_nans/default                   1.01     75.6±1.77ms   185.3 MB/sec    1.00     74.8±0.18ms   187.3 MB/sec
   float_with_nans/parquet_2                 1.00     97.3±1.80ms   143.9 MB/sec    1.00     97.2±0.28ms   144.0 MB/sec
   float_with_nans/zstd                      1.01    113.2±1.85ms   123.7 MB/sec    1.00    112.4±0.21ms   124.6 MB/sec
   float_with_nans/zstd_parquet_2            1.00    134.6±1.84ms   104.0 MB/sec    1.00    134.2±0.23ms   104.3 MB/sec
   list_primitive/bloom_filter               1.00    358.2±1.90ms  1522.7 MB/sec    1.00    357.3±3.51ms  1526.3 MB/sec
   list_primitive/default                    1.01    280.4±1.04ms  1945.2 MB/sec    1.00    276.8±1.82ms  1970.3 MB/sec
   list_primitive/parquet_2                  1.01    285.6±7.37ms  1909.7 MB/sec    1.00    281.5±8.69ms  1937.2 MB/sec
   list_primitive/zstd                       1.00    520.2±5.58ms  1048.5 MB/sec    1.03    533.3±4.02ms  1022.6 MB/sec
   list_primitive/zstd_parquet_2             1.00    498.3±4.73ms  1094.4 MB/sec    1.00    498.3±3.36ms  1094.4 MB/sec
   list_primitive_non_null/bloom_filter      1.00   467.0±37.24ms  1165.5 MB/sec    1.03   481.0±12.43ms  1131.4 MB/sec
   list_primitive_non_null/default           1.01   331.1±26.07ms  1643.7 MB/sec    1.00    328.8±8.80ms  1655.3 MB/sec
   list_primitive_non_null/parquet_2         1.01    312.0±7.20ms  1744.6 MB/sec    1.00   307.8±34.43ms  1767.9 MB/sec
   list_primitive_non_null/zstd              1.02   736.3±21.31ms   739.2 MB/sec    1.00   723.7±16.01ms   752.0 MB/sec
   list_primitive_non_null/zstd_parquet_2    1.00   730.8±20.55ms   744.8 MB/sec    1.00   732.5±24.41ms   743.0 MB/sec
   primitive/bloom_filter                    1.02    164.0±0.70ms   273.7 MB/sec    1.00    161.5±0.99ms   277.9 MB/sec
   primitive/default                         1.04    133.2±1.04ms   336.9 MB/sec    1.00    127.8±0.99ms   351.1 MB/sec
   primitive/parquet_2                       1.03    146.7±0.69ms   306.0 MB/sec    1.00    142.8±1.13ms   314.2 MB/sec
   primitive/zstd                            1.02    161.9±0.64ms   277.2 MB/sec    1.00    158.2±1.05ms   283.6 MB/sec
   primitive/zstd_parquet_2                  1.02    180.5±0.81ms   248.6 MB/sec    1.00    176.6±1.01ms   254.2 MB/sec
   primitive_non_null/bloom_filter           1.02    114.1±0.38ms   385.7 MB/sec    1.00    112.0±0.72ms   392.9 MB/sec
   primitive_non_null/default                1.00     68.8±0.23ms   639.8 MB/sec    1.01     69.5±0.31ms   633.2 MB/sec
   primitive_non_null/parquet_2              1.00     90.7±0.20ms   485.2 MB/sec    1.01     91.3±0.37ms   481.7 MB/sec
   primitive_non_null/zstd                   1.05    105.9±0.23ms   415.6 MB/sec    1.00    100.4±0.42ms   438.2 MB/sec
   primitive_non_null/zstd_parquet_2         1.04    130.5±1.91ms   337.2 MB/sec    1.00    125.6±0.25ms   350.4 MB/sec
   string/bloom_filter                       1.00    210.4±6.91ms     2.4 GB/sec    1.02   214.1±17.09ms     2.4 GB/sec
   string/default                            1.00    121.5±1.72ms     4.2 GB/sec    1.07   130.1±17.64ms     3.9 GB/sec
   string/parquet_2                          1.49    188.9±0.48ms     2.7 GB/sec    1.00    126.7±0.59ms     4.0 GB/sec
   string/zstd                               1.00   440.8±17.23ms  1189.2 MB/sec    1.02   450.9±20.06ms  1162.7 MB/sec
   string/zstd_parquet_2                     1.04   413.5±19.51ms  1267.8 MB/sec    1.00    396.3±2.64ms  1322.7 MB/sec
   string_and_binary_view/bloom_filter       1.01     67.6±0.32ms   477.4 MB/sec    1.00     66.8±0.37ms   482.6 MB/sec
   string_and_binary_view/default            1.03     51.8±0.32ms   622.8 MB/sec    1.00     50.4±0.37ms   639.6 MB/sec
   string_and_binary_view/parquet_2          1.02     62.4±0.35ms   516.7 MB/sec    1.00     61.1±0.35ms   527.5 MB/sec
   string_and_binary_view/zstd               1.01     88.3±0.35ms   365.1 MB/sec    1.00     87.6±0.81ms   367.9 MB/sec
   string_and_binary_view/zstd_parquet_2     1.01     76.4±0.34ms   422.4 MB/sec    1.00     75.8±0.72ms   425.7 MB/sec
   string_dictionary/bloom_filter            1.00     90.2±0.66ms     2.9 GB/sec    1.26    114.0±2.32ms     2.3 GB/sec
   string_dictionary/default                 1.00     50.0±0.44ms     5.2 GB/sec    1.44     71.9±0.97ms     3.6 GB/sec
   string_dictionary/parquet_2               1.29     96.5±0.39ms     2.7 GB/sec    1.00     74.8±1.34ms     3.5 GB/sec
   string_dictionary/zstd                    1.00   223.4±15.01ms  1182.1 MB/sec    1.01    225.8±3.45ms  1169.8 MB/sec
   string_dictionary/zstd_parquet_2          1.03    206.1±1.37ms  1281.5 MB/sec    1.00    200.4±0.38ms  1317.9 MB/sec
   string_non_null/bloom_filter              1.00   245.0±14.59ms     2.1 GB/sec    1.00   245.7±14.35ms     2.1 GB/sec
   string_non_null/default                   1.08   139.0±12.46ms     3.7 GB/sec    1.00    129.0±9.54ms     4.0 GB/sec
   string_non_null/parquet_2                 1.07    142.3±7.26ms     3.6 GB/sec    1.00    132.8±3.96ms     3.9 GB/sec
   string_non_null/zstd                      1.00    556.6±5.03ms   941.5 MB/sec    1.02   566.5±14.69ms   925.0 MB/sec
   string_non_null/zstd_parquet_2            1.00    514.7±2.66ms  1018.1 MB/sec    1.02   522.6±11.45ms  1002.7 MB/sec
   ```
   
   </p>
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 1342.0s |
   | Peak memory | 4.9 GiB |
   | Avg memory | 4.6 GiB |
   | CPU user | 1258.9s |
   | CPU sys | 82.7s |
   | Disk read | 0 B |
   | Disk write | 1.7 GiB |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 1354.7s |
   | Peak memory | 4.9 GiB |
   | Avg memory | 4.6 GiB |
   | CPU user | 1268.0s |
   | CPU sys | 86.6s |
   | Disk read | 0 B |
   | Disk write | 4.1 MiB |
   </details>
   
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4171508882

   run benchmark arrow_writer
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4165289179

   > The benchmark failure appears to happen on main as well; I will file a follow on ticket
   
   - https://github.com/apache/arrow-rs/issues/9637


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4171845774

   🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/9636#issuecomment-4171508882)
   
   **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Details</summary>
   <p>
   
   ```
   group                                     alamb_test_encoding_speed              main
   -----                                     -------------------------              ----
   bool/bloom_filter                         1.00     14.1±0.04ms    17.8 MB/sec    1.00     14.1±0.18ms    17.7 MB/sec
   bool/default                              1.00     12.1±0.04ms    20.7 MB/sec    1.00     12.1±0.19ms    20.7 MB/sec
   bool/parquet_2                            1.00     15.1±0.04ms    16.6 MB/sec    1.03     15.4±0.18ms    16.2 MB/sec
   bool/zstd                                 1.00     12.6±0.05ms    19.9 MB/sec    1.00     12.6±0.18ms    19.8 MB/sec
   bool/zstd_parquet_2                       1.00     15.5±0.05ms    16.2 MB/sec    1.02     15.8±0.19ms    15.8 MB/sec
   bool_non_null/bloom_filter                1.00      6.8±0.03ms    18.3 MB/sec    1.01      6.9±0.03ms    18.1 MB/sec
   bool_non_null/default                     1.00      4.2±0.02ms    30.1 MB/sec    1.02      4.2±0.02ms    29.6 MB/sec
   bool_non_null/parquet_2                   1.00      8.0±0.02ms    15.6 MB/sec    1.08      8.6±0.04ms    14.5 MB/sec
   bool_non_null/zstd                        1.00      4.5±0.02ms    27.7 MB/sec    1.01      4.6±0.02ms    27.4 MB/sec
   bool_non_null/zstd_parquet_2              1.00      8.4±0.02ms    14.8 MB/sec    1.07      9.0±0.04ms    13.8 MB/sec
   float_with_nans/bloom_filter              1.00     93.0±0.17ms   150.5 MB/sec    1.01     93.7±0.30ms   149.4 MB/sec
   float_with_nans/default                   1.00     74.6±0.12ms   187.6 MB/sec    1.01     75.3±0.14ms   185.9 MB/sec
   float_with_nans/parquet_2                 1.00     96.7±0.18ms   144.7 MB/sec    1.00     97.0±0.22ms   144.3 MB/sec
   float_with_nans/zstd                      1.00    112.6±0.13ms   124.4 MB/sec    1.00    112.6±0.14ms   124.3 MB/sec
   float_with_nans/zstd_parquet_2            1.00    134.0±0.16ms   104.5 MB/sec    1.01    134.7±0.29ms   103.9 MB/sec
   list_primitive/bloom_filter               1.04    343.9±0.81ms  1585.7 MB/sec    1.00   331.2±11.71ms  1646.8 MB/sec
   list_primitive/default                    1.07    266.9±1.14ms  2043.0 MB/sec    1.00    250.5±7.12ms     2.1 GB/sec
   list_primitive/parquet_2                  1.03    286.5±0.90ms  1903.4 MB/sec    1.00    278.2±7.22ms  1960.5 MB/sec
   list_primitive/zstd                       1.00    500.6±2.97ms  1089.3 MB/sec    1.01    504.9±8.86ms  1080.2 MB/sec
   list_primitive/zstd_parquet_2             1.00    491.4±0.42ms  1109.9 MB/sec    1.01    495.5±7.46ms  1100.7 MB/sec
   list_primitive_non_null/bloom_filter      1.00   445.0±11.83ms  1223.0 MB/sec    1.03   458.0±10.54ms  1188.4 MB/sec
   list_primitive_non_null/default           1.00   313.3±10.67ms  1737.0 MB/sec    1.01    315.9±9.37ms  1722.8 MB/sec
   list_primitive_non_null/parquet_2         1.10    359.5±1.60ms  1513.9 MB/sec    1.00   325.6±18.76ms  1671.4 MB/sec
   list_primitive_non_null/zstd              1.00   717.6±19.85ms   758.4 MB/sec    1.01   722.6±16.68ms   753.1 MB/sec
   list_primitive_non_null/zstd_parquet_2    1.05    727.1±1.97ms   748.5 MB/sec    1.00    690.4±5.68ms   788.3 MB/sec
   primitive/bloom_filter                    1.00    159.0±0.36ms   282.3 MB/sec    1.03    163.1±4.86ms   275.1 MB/sec
   primitive/default                         1.00    127.2±0.41ms   352.7 MB/sec    1.02    130.1±4.97ms   344.9 MB/sec
   primitive/parquet_2                       1.00    142.6±0.43ms   314.6 MB/sec    1.02    144.8±4.79ms   310.0 MB/sec
   primitive/zstd                            1.00    157.0±0.45ms   285.8 MB/sec    1.02    159.6±4.76ms   281.1 MB/sec
   primitive/zstd_parquet_2                  1.00    175.8±0.29ms   255.3 MB/sec    1.02    178.5±4.77ms   251.4 MB/sec
   primitive_non_null/bloom_filter           1.04    114.6±0.29ms   384.1 MB/sec    1.00    110.4±0.56ms   398.4 MB/sec
   primitive_non_null/default                1.00     68.7±0.18ms   640.7 MB/sec    1.01     69.3±0.21ms   635.0 MB/sec
   primitive_non_null/parquet_2              1.00     90.9±0.18ms   483.8 MB/sec    1.01     91.5±0.28ms   480.9 MB/sec
   primitive_non_null/zstd                   1.00    106.2±0.19ms   414.2 MB/sec    1.00    106.8±0.22ms   412.2 MB/sec
   primitive_non_null/zstd_parquet_2         1.00    131.0±2.05ms   335.8 MB/sec    1.00    131.2±2.07ms   335.3 MB/sec
   string/bloom_filter                       1.00   215.1±19.81ms     2.4 GB/sec    1.00   215.9±12.28ms     2.4 GB/sec
   string/default                            1.00    120.9±8.52ms     4.2 GB/sec    1.08   130.6±15.14ms     3.9 GB/sec
   string/parquet_2                          1.06   136.4±21.09ms     3.8 GB/sec    1.00    129.3±0.67ms     4.0 GB/sec
   string/zstd                               1.00    430.3±7.23ms  1218.4 MB/sec    1.01    434.7±4.61ms  1206.0 MB/sec
   string/zstd_parquet_2                     1.02    407.5±7.02ms  1286.5 MB/sec    1.00    398.4±0.72ms  1315.8 MB/sec
   string_and_binary_view/bloom_filter       1.00     65.3±0.19ms   493.8 MB/sec    1.06     69.2±0.46ms   465.9 MB/sec
   string_and_binary_view/default            1.00     50.3±0.15ms   641.6 MB/sec    1.02     51.5±0.38ms   626.6 MB/sec
   string_and_binary_view/parquet_2          1.00     61.1±0.13ms   527.4 MB/sec    1.02     62.2±0.39ms   518.2 MB/sec
   string_and_binary_view/zstd               1.00     87.1±0.15ms   370.4 MB/sec    1.01     88.3±0.40ms   365.4 MB/sec
   string_and_binary_view/zstd_parquet_2     1.00     75.0±0.15ms   430.3 MB/sec    1.01     76.0±0.39ms   424.3 MB/sec
   string_dictionary/bloom_filter            1.42    133.4±9.87ms  1979.9 MB/sec    1.00     93.9±0.49ms     2.7 GB/sec
   string_dictionary/default                 1.88     96.6±0.51ms     2.7 GB/sec    1.00     51.3±0.79ms     5.0 GB/sec
   string_dictionary/parquet_2               1.80     99.9±0.27ms     2.6 GB/sec    1.00     55.5±0.22ms     4.6 GB/sec
   string_dictionary/zstd                    1.05   223.3±10.13ms  1182.9 MB/sec    1.00    212.0±0.32ms  1246.1 MB/sec
   string_dictionary/zstd_parquet_2          1.00    199.6±0.17ms  1323.3 MB/sec    1.00    200.5±0.23ms  1317.1 MB/sec
   string_non_null/bloom_filter              1.00   247.0±13.83ms     2.1 GB/sec    1.04   258.1±10.47ms  2030.6 MB/sec
   string_non_null/default                   1.00   134.7±12.82ms     3.8 GB/sec    1.04   140.6±12.96ms     3.6 GB/sec
   string_non_null/parquet_2                 1.05    139.0±5.34ms     3.7 GB/sec    1.00    132.4±7.21ms     3.9 GB/sec
   string_non_null/zstd                      1.07   568.4±14.66ms   921.8 MB/sec    1.00    533.5±1.95ms   982.1 MB/sec
   string_non_null/zstd_parquet_2            1.02   521.1±10.06ms  1005.6 MB/sec    1.00    510.6±2.63ms  1026.3 MB/sec
   ```
   
   </p>
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 1313.6s |
   | Peak memory | 4.9 GiB |
   | Avg memory | 4.6 GiB |
   | CPU user | 1267.5s |
   | CPU sys | 46.0s |
   | Disk read | 0 B |
   | Disk write | 633.6 MiB |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 1341.0s |
   | Peak memory | 4.9 GiB |
   | Avg memory | 4.6 GiB |
   | CPU user | 1246.7s |
   | CPU sys | 94.2s |
   | Disk read | 0 B |
   | Disk write | 4.1 MiB |
   </details>
   
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4165051971

   Benchmark for [this request](https://github.com/apache/arrow-rs/pull/9636#issuecomment-4164922910) failed.
   
   Last 20 lines of output:
   <details><summary>Click to expand</summary>
   
   ```
   Benchmarking list_primitive/parquet_2: Analyzing
   Benchmarking list_primitive/zstd
   Benchmarking list_primitive/zstd: Warming up for 3.0000 s
   
   Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 49.2s, or reduce sample count to 10.
   Benchmarking list_primitive/zstd: Collecting 100 samples in estimated 49.218 s (100 iterations)
   Benchmarking list_primitive/zstd: Analyzing
   Benchmarking list_primitive/zstd_parquet_2
   Benchmarking list_primitive/zstd_parquet_2: Warming up for 3.0000 s
   
   Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 48.7s, or reduce sample count to 10.
   Benchmarking list_primitive/zstd_parquet_2: Collecting 100 samples in estimated 48.713 s (100 iterations)
   Benchmarking list_primitive/zstd_parquet_2: Analyzing
   Benchmarking list_primitive/cdc
   Benchmarking list_primitive/cdc: Warming up for 3.0000 s
   
   thread 'main' (15467) panicked at parquet/src/column/writer/mod.rs:720:39:
   range end index 59344 out of range for slice of length 58905
   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
   error: bench failed, to rerun pass `-p parquet --bench arrow_writer`
   ```
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4165120346

   
   The benchmark failure appears to happen on main as well; I will file a follow on ticket


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed pull request #9636: Alamb/test encoding speed
URL: https://github.com/apache/arrow-rs/pull/9636


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4165049853

   Benchmark for [this request](https://github.com/apache/arrow-rs/pull/9636#issuecomment-4164926733) failed.
   
   Last 20 lines of output:
   <details><summary>Click to expand</summary>
   
   ```
   Benchmarking list_primitive/parquet_2: Analyzing
   Benchmarking list_primitive/zstd
   Benchmarking list_primitive/zstd: Warming up for 3.0000 s
   
   Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 49.6s, or reduce sample count to 10.
   Benchmarking list_primitive/zstd: Collecting 100 samples in estimated 49.588 s (100 iterations)
   Benchmarking list_primitive/zstd: Analyzing
   Benchmarking list_primitive/zstd_parquet_2
   Benchmarking list_primitive/zstd_parquet_2: Warming up for 3.0000 s
   
   Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 48.8s, or reduce sample count to 10.
   Benchmarking list_primitive/zstd_parquet_2: Collecting 100 samples in estimated 48.753 s (100 iterations)
   Benchmarking list_primitive/zstd_parquet_2: Analyzing
   Benchmarking list_primitive/cdc
   Benchmarking list_primitive/cdc: Warming up for 3.0000 s
   
   thread 'main' (16198) panicked at parquet/src/column/writer/mod.rs:720:39:
   range end index 59344 out of range for slice of length 58905
   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
   error: bench failed, to rerun pass `-p parquet --bench arrow_writer`
   ```
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4164926733

   
   run benchmark arrow_writer
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4164922910

   run benchmark arrow_writer


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4171240149

   
   run benchmark arrow_writer
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Alamb/test encoding speed [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #9636:
URL: https://github.com/apache/arrow-rs/pull/9636#issuecomment-4171528584

   🤖 Arrow criterion benchmark running (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/9636#issuecomment-4171508882)
   **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4171508882-650-9t7tb 6.12.55+ #1 SMP Sun Feb  1 08:59:41 UTC 2026 aarch64 GNU/Linux`
   <details><summary>CPU Details (lscpu)</summary>
   
   ```
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   Comparing alamb/test_encoding_speed (d55ca607945662d8d47e500ddf1bf3d750ef4a1e) to 1f07e54 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/1f07e54c1209a512f3ce15f4e576f0ca5d1c8d97..d55ca607945662d8d47e500ddf1bf3d750ef4a1e)
   BENCH_NAME=arrow_writer
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_writer
   BENCH_FILTER=
   Results will be posted here when complete
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org